Sep 07 11:45:20.123504 ip-10-0-135-173 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Sep 07 11:45:20.123513 ip-10-0-135-173 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Sep 07 11:45:20.123520 ip-10-0-135-173 systemd[1]: kubelet.service: Failed with result 'resources'.
Sep 07 11:45:20.123664 ip-10-0-135-173 systemd[1]: Failed to start Kubernetes Kubelet.
Sep 07 11:45:30.373285 ip-10-0-135-173 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Sep 07 11:45:30.373304 ip-10-0-135-173 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot 7306a6aed61c46a7a988d73439818d69 --
Sep 07 11:47:07.165569 ip-10-0-135-173 systemd[1]: Starting Kubernetes Kubelet...
Sep 07 11:47:07.546541 ip-10-0-135-173 kubenswrapper[2563]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 07 11:47:07.546541 ip-10-0-135-173 kubenswrapper[2563]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Sep 07 11:47:07.546541 ip-10-0-135-173 kubenswrapper[2563]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 07 11:47:07.546541 ip-10-0-135-173 kubenswrapper[2563]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Sep 07 11:47:07.546541 ip-10-0-135-173 kubenswrapper[2563]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 07 11:47:07.547623 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:07.547541    2563 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Sep 07 11:47:07.551230 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.551216    2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 07 11:47:07.551230 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.551230    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 07 11:47:07.551329 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.551234    2563 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 07 11:47:07.551329 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.551237    2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 07 11:47:07.551329 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.551240    2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 07 11:47:07.551329 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.551243    2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 07 11:47:07.551329 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.551251    2563 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 07 11:47:07.551329 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.551255    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 07 11:47:07.551329 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.551270    2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 07 11:47:07.551329 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.551275    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 07 11:47:07.551329 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.551280    2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 07 11:47:07.551329 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.551283    2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 07 11:47:07.551329 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.551285    2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 07 11:47:07.551329 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.551288    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 07 11:47:07.551329 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.551291    2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 07 11:47:07.551329 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.551295    2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 07 11:47:07.551329 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.551300    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 07 11:47:07.551329 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.551303    2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 07 11:47:07.551329 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.551306    2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 07 11:47:07.551329 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.551309    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 07 11:47:07.551329 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.551311    2563 feature_gate.go:328] unrecognized feature gate: Example
Sep 07 11:47:07.551329 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.551314    2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 07 11:47:07.551821 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.551316    2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 07 11:47:07.551821 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.551319    2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 07 11:47:07.551821 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.551322    2563 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 07 11:47:07.551821 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.551324    2563 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 07 11:47:07.551821 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.551327    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 07 11:47:07.551821 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.551330    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 07 11:47:07.551821 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.551333    2563 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 07 11:47:07.551821 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.551336    2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 07 11:47:07.551821 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.551339    2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 07 11:47:07.551821 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.551341    2563 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 07 11:47:07.551821 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.551344    2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 07 11:47:07.551821 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.551347    2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 07 11:47:07.551821 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.551350    2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 07 11:47:07.551821 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.551353    2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 07 11:47:07.551821 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.551357    2563 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 07 11:47:07.551821 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.551361    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 07 11:47:07.551821 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.551365    2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 07 11:47:07.551821 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.551368    2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 07 11:47:07.551821 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.551371    2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 07 11:47:07.552280 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.551373    2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 07 11:47:07.552280 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.551376    2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 07 11:47:07.552280 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.551378    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 07 11:47:07.552280 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.551380    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 07 11:47:07.552280 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.551383    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 07 11:47:07.552280 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.551385    2563 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 07 11:47:07.552280 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.551388    2563 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 07 11:47:07.552280 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.551390    2563 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 07 11:47:07.552280 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.551393    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 07 11:47:07.552280 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.551395    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 07 11:47:07.552280 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.551398    2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 07 11:47:07.552280 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.551400    2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 07 11:47:07.552280 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.551402    2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 07 11:47:07.552280 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.551406    2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 07 11:47:07.552280 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.551410    2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 07 11:47:07.552280 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.551414    2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 07 11:47:07.552280 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.551417    2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 07 11:47:07.552280 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.551420    2563 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 07 11:47:07.552280 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.551422    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 07 11:47:07.552751 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.551425    2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 07 11:47:07.552751 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.551428    2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 07 11:47:07.552751 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.551430    2563 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 07 11:47:07.552751 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.551433    2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 07 11:47:07.552751 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.551436    2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 07 11:47:07.552751 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.551438    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 07 11:47:07.552751 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.551441    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 07 11:47:07.552751 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.551443    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 07 11:47:07.552751 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.551446    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 07 11:47:07.552751 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.551449    2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 07 11:47:07.552751 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.551452    2563 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 07 11:47:07.552751 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.551458    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 07 11:47:07.552751 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.551461    2563 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 07 11:47:07.552751 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.551463    2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 07 11:47:07.552751 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.551466    2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 07 11:47:07.552751 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.551468    2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 07 11:47:07.552751 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.551470    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 07 11:47:07.552751 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.551473    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 07 11:47:07.552751 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.551475    2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 07 11:47:07.552751 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.551478    2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 07 11:47:07.553229 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.551480    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 07 11:47:07.553229 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.551483    2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 07 11:47:07.553229 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.551485    2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 07 11:47:07.553229 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.551488    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 07 11:47:07.553229 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.551490    2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 07 11:47:07.553229 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.551493    2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 07 11:47:07.553229 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.551495    2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 07 11:47:07.553229 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.551499    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 07 11:47:07.553229 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.551502    2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 07 11:47:07.553229 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.551504    2563 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 07 11:47:07.553229 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.551507    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 07 11:47:07.553229 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.551509    2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 07 11:47:07.553229 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.551512    2563 feature_gate.go:328] unrecognized feature gate: Example2
Sep 07 11:47:07.553229 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.551515    2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 07 11:47:07.553229 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.551517    2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 07 11:47:07.553229 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.551519    2563 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 07 11:47:07.553229 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.551522    2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 07 11:47:07.553229 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.551524    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 07 11:47:07.553229 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.551527    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 07 11:47:07.553229 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.552896    2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 07 11:47:07.553780 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.552904    2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 07 11:47:07.553780 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.552907    2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 07 11:47:07.553780 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.552910    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 07 11:47:07.553780 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.552913    2563 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 07 11:47:07.553780 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.552919    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 07 11:47:07.553780 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.552922    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 07 11:47:07.553780 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.552925    2563 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 07 11:47:07.553780 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.552928    2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 07 11:47:07.553780 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.552930    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 07 11:47:07.553780 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.552933    2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 07 11:47:07.553780 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.552935    2563 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 07 11:47:07.553780 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.552938    2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 07 11:47:07.553780 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.552941    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 07 11:47:07.553780 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.552943    2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 07 11:47:07.553780 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.552946    2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 07 11:47:07.553780 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.552948    2563 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 07 11:47:07.553780 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.552951    2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 07 11:47:07.553780 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.552953    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 07 11:47:07.553780 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.552958    2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 07 11:47:07.554253 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.552963    2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 07 11:47:07.554253 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.552966    2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 07 11:47:07.554253 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.552969    2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 07 11:47:07.554253 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.552972    2563 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 07 11:47:07.554253 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.552975    2563 feature_gate.go:328] unrecognized feature gate: Example2
Sep 07 11:47:07.554253 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.552978    2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 07 11:47:07.554253 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.552980    2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 07 11:47:07.554253 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.552982    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 07 11:47:07.554253 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.552985    2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 07 11:47:07.554253 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.552988    2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 07 11:47:07.554253 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.552990    2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 07 11:47:07.554253 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.552993    2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 07 11:47:07.554253 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.552997    2563 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 07 11:47:07.554253 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.553000    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 07 11:47:07.554253 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.553002    2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 07 11:47:07.554253 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.553005    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 07 11:47:07.554253 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.553008    2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 07 11:47:07.554253 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.553011    2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 07 11:47:07.554253 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.553018    2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 07 11:47:07.554253 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.553021    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 07 11:47:07.554845 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.553024    2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 07 11:47:07.554845 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.553027    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 07 11:47:07.554845 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.553029    2563 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 07 11:47:07.554845 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.553032    2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 07 11:47:07.554845 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.553034    2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 07 11:47:07.554845 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.553037    2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 07 11:47:07.554845 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.553039    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 07 11:47:07.554845 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.553042    2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 07 11:47:07.554845 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.553046    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 07 11:47:07.554845 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.553048    2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 07 11:47:07.554845 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.553051    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 07 11:47:07.554845 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.553054    2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 07 11:47:07.554845 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.553057    2563 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 07 11:47:07.554845 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.553059    2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 07 11:47:07.554845 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.553062    2563 feature_gate.go:328] unrecognized feature gate: Example
Sep 07 11:47:07.554845 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.553065    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 07 11:47:07.554845 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.553068    2563 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 07 11:47:07.554845 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.553070    2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 07 11:47:07.554845 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.553073    2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 07 11:47:07.554845 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.553075    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 07 11:47:07.555367 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.553078    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 07 11:47:07.555367 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.553080    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 07 11:47:07.555367 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.553082    2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 07 11:47:07.555367 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.553085    2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 07 11:47:07.555367 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.553088    2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 07 11:47:07.555367 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.553090    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 07 11:47:07.555367 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.553094    2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 07 11:47:07.555367 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.553098    2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 07 11:47:07.555367 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.553101    2563 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 07 11:47:07.555367 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.553104    2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 07 11:47:07.555367 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.553107    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 07 11:47:07.555367 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.553114    2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 07 11:47:07.555367 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.553117    2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 07 11:47:07.555367 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.553120    2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 07 11:47:07.555367 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.553122    2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 07 11:47:07.555367 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.553125    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 07 11:47:07.555367 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.553127    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 07 11:47:07.555367 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.553129    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 07 11:47:07.555367 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.553132    2563 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 07 11:47:07.555840 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.553134    2563 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 07 11:47:07.555840 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.553137    2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 07 11:47:07.555840 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.553139    2563 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 07 11:47:07.555840 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.553142    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 07 11:47:07.555840 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.553146    2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 07 11:47:07.555840 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.553149    2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 07 11:47:07.555840 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.553151    2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 07 11:47:07.555840 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.553154    2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 07 11:47:07.555840 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.553156    2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 07 11:47:07.555840 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.553159    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 07 11:47:07.555840 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.553161    2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 07 11:47:07.555840 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.553164    2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 07 11:47:07.555840 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.553167    2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 07 11:47:07.555840 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.553169    2563 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 07 11:47:07.555840 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.553172    2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 07 11:47:07.555840 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.553174    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 07 11:47:07.555840 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.553177    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 07 11:47:07.555840 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.553179    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 07 11:47:07.555840 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.553182    2563 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 07 11:47:07.555840 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.553184    2563 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 07 11:47:07.556353 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.553570    2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 07 11:47:07.556353 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.553575    2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 07 11:47:07.556353 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.553578    2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 07 11:47:07.556353 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.553581    2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 07 11:47:07.556353 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.553584    2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 07 11:47:07.556353 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.553587    2563 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 07 11:47:07.556353 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.553590    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 07 11:47:07.556353 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.553592    2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 07 11:47:07.556353 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.553595    2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 07 11:47:07.556353 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.553598    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 07 11:47:07.556353 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.553601    2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 07 11:47:07.556353 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.553605    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 07 11:47:07.556353 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.553607    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 07 11:47:07.556353 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.553610    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 07 11:47:07.556353 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.553614    2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 07 11:47:07.556353 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.553618    2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 07 11:47:07.556353 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.553621    2563 feature_gate.go:328] unrecognized feature gate: Example2
Sep 07 11:47:07.556353 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.553624    2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 07 11:47:07.556353 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.553627    2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 07 11:47:07.556817 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.553630    2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 07 11:47:07.556817 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.553633    2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 07 11:47:07.556817 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.553635    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 07 11:47:07.556817 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.553638    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 07 11:47:07.556817 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.553640    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 07 11:47:07.556817 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.553643    2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 07 11:47:07.556817 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.553646    2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 07 11:47:07.556817 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.553648    2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 07 11:47:07.556817 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.553650    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 07 11:47:07.556817 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.553653    2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 07 11:47:07.556817 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.553656    2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 07 11:47:07.556817 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.553658    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 07 11:47:07.556817 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.553660    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 07 11:47:07.556817 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.553664    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 07 11:47:07.556817 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.553666    2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 07 11:47:07.556817 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.553669    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 07 11:47:07.556817 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.553671    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 07 11:47:07.556817 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.553674    2563 feature_gate.go:328] unrecognized feature gate: Example
Sep 07 11:47:07.556817 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.553677    2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 07 11:47:07.556817 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.553680    2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 07 11:47:07.557319 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.553682    2563 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 07 11:47:07.557319 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.553685    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 07 11:47:07.557319 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.553687    2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 07 11:47:07.557319 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.553690    2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 07 11:47:07.557319 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.553692    2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 07 11:47:07.557319 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.553695    2563 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 07 11:47:07.557319 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.553697    2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 07 11:47:07.557319 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.553700    2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 07 11:47:07.557319 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.553702    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 07 11:47:07.557319 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.553705    2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 07 11:47:07.557319 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.553708    2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 07 11:47:07.557319 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.553711    2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 07 11:47:07.557319 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.553713    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 07 11:47:07.557319 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.553716    2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 07 11:47:07.557319 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.553718    2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 07 11:47:07.557319 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.553721    2563 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 07 11:47:07.557319 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.553723    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 07 11:47:07.557319 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.553726    2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 07 11:47:07.557319 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.553729    2563 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 07 11:47:07.557319 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.553731    2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 07 11:47:07.557818 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.553734    2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 07 11:47:07.557818 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.553736    2563 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 07 11:47:07.557818 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.553739    2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 07 11:47:07.557818 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.553741    2563 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 07 11:47:07.557818 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.553744    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 07 11:47:07.557818 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.553746    2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 07 11:47:07.557818 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.553749    2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 07 11:47:07.557818 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.553751    2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 07 11:47:07.557818 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.553754    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 07 11:47:07.557818 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.553757    2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 07 11:47:07.557818 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.553759    2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 07 11:47:07.557818 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.553762    2563 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 07 11:47:07.557818 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.553764    2563 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 07 11:47:07.557818 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.553767    2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 07 11:47:07.557818 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.553769    2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 07 11:47:07.557818 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.553771    2563 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 07 11:47:07.557818 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.553774    2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 07 11:47:07.557818 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.553776    2563 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 07 11:47:07.557818 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.553779    2563 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 07 11:47:07.557818 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.553781    2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 07 11:47:07.558334 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.553784    2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 07 11:47:07.558334 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.553788    2563 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 07 11:47:07.558334 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.553790    2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 07 11:47:07.558334 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.553793    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 07 11:47:07.558334 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.553796    2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 07 11:47:07.558334 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.553798    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 07 11:47:07.558334 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.553801    2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 07 11:47:07.558334 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.553803    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 07 11:47:07.558334 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.553806    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 07 11:47:07.558334 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.553808    2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 07 11:47:07.558334 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.553811    2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 07 11:47:07.558334 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.553813    2563 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 07 11:47:07.558334 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.553817    2563 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 07 11:47:07.558334 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.553820    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 07 11:47:07.558334 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.553823    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 07 11:47:07.558334 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.553826    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 07 11:47:07.558334 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.553829    2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 07 11:47:07.558334 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.553832    2563 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 07 11:47:07.558334 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.553835    2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 07 11:47:07.558827 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.553837    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 07 11:47:07.558827 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:07.553908    2563 flags.go:64] FLAG: --address="0.0.0.0"
Sep 07 11:47:07.558827 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:07.553937    2563 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Sep 07 11:47:07.558827 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:07.553943    2563 flags.go:64] FLAG: --anonymous-auth="true"
Sep 07 11:47:07.558827 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:07.553948    2563 flags.go:64] FLAG: --application-metrics-count-limit="100"
Sep 07 11:47:07.558827 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:07.553953    2563 flags.go:64] FLAG: --authentication-token-webhook="false"
Sep 07 11:47:07.558827 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:07.553956    2563 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Sep 07 11:47:07.558827 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:07.553960    2563 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Sep 07 11:47:07.558827 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:07.553964    2563 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Sep 07 11:47:07.558827 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:07.553967    2563 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Sep 07 11:47:07.558827 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:07.553970    2563 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Sep 07 11:47:07.558827 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:07.553974    2563 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Sep 07 11:47:07.558827 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:07.553977    2563 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Sep 07 11:47:07.558827 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:07.553980    2563 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Sep 07 11:47:07.558827 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:07.553983    2563 flags.go:64] FLAG: --cgroup-root=""
Sep 07 11:47:07.558827 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:07.553986    2563 flags.go:64] FLAG: --cgroups-per-qos="true"
Sep 07 11:47:07.558827 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:07.553989    2563 flags.go:64] FLAG: --client-ca-file=""
Sep 07 11:47:07.558827 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:07.553992    2563 flags.go:64] FLAG: --cloud-provider="external"
Sep 07 11:47:07.558827 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:07.553995    2563 flags.go:64] FLAG: --cluster-dns="[]"
Sep 07 11:47:07.558827 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:07.553999    2563 flags.go:64] FLAG: --cluster-domain=""
Sep 07 11:47:07.558827 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:07.554002    2563 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Sep 07 11:47:07.558827 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:07.554005    2563 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Sep 07 11:47:07.558827 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:07.554008    2563 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Sep 07 11:47:07.558827 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:07.554011    2563 flags.go:64] FLAG: --container-log-max-files="5"
Sep 07 11:47:07.559423 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:07.554014    2563 flags.go:64] FLAG: --container-log-max-size="10Mi"
Sep 07 11:47:07.559423 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:07.554017    2563 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Sep 07 11:47:07.559423 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:07.554020    2563 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Sep 07 11:47:07.559423 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:07.554023    2563 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Sep 07 11:47:07.559423 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:07.554026    2563 flags.go:64] FLAG: --contention-profiling="false"
Sep 07 11:47:07.559423 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:07.554029    2563 flags.go:64] FLAG: --cpu-cfs-quota="true"
Sep 07 11:47:07.559423 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:07.554032    2563 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Sep 07 11:47:07.559423 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:07.554035    2563 flags.go:64] FLAG: --cpu-manager-policy="none"
Sep 07 11:47:07.559423 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:07.554038    2563 flags.go:64] FLAG: --cpu-manager-policy-options=""
Sep 07 11:47:07.559423 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:07.554041    2563 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Sep 07 11:47:07.559423 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:07.554044    2563 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Sep 07 11:47:07.559423 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:07.554046    2563 flags.go:64] FLAG: --enable-debugging-handlers="true"
Sep 07 11:47:07.559423 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:07.554049    2563 flags.go:64] FLAG: --enable-load-reader="false"
Sep 07 11:47:07.559423 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:07.554053    2563 flags.go:64] FLAG: --enable-server="true"
Sep 07 11:47:07.559423 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:07.554056    2563 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Sep 07 11:47:07.559423 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:07.554060    2563 flags.go:64] FLAG: --event-burst="100"
Sep 07 11:47:07.559423 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:07.554063    2563 flags.go:64] FLAG: --event-qps="50"
Sep 07 11:47:07.559423 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:07.554066    2563 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Sep 07 11:47:07.559423 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:07.554069    2563 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Sep 07 11:47:07.559423 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:07.554072    2563 flags.go:64] FLAG: --eviction-hard=""
Sep 07 11:47:07.559423 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:07.554075    2563 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Sep 07 11:47:07.559423 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:07.554078    2563 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Sep 07 11:47:07.559423 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:07.554083    2563 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Sep 07 11:47:07.559423 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:07.554086    2563 flags.go:64] FLAG: --eviction-soft=""
Sep 07 11:47:07.559423 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:07.554089    2563 flags.go:64] FLAG: --eviction-soft-grace-period=""
Sep 07 11:47:07.560021 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:07.554092    2563 flags.go:64] FLAG: --exit-on-lock-contention="false"
Sep 07 11:47:07.560021 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:07.554095    2563 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Sep 07 11:47:07.560021 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:07.554098    2563 flags.go:64] FLAG: --experimental-mounter-path=""
Sep 07 11:47:07.560021 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:07.554100    2563 flags.go:64] FLAG: --fail-cgroupv1="false"
Sep 07 11:47:07.560021 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:07.554103    2563 flags.go:64] FLAG: --fail-swap-on="true"
Sep 07 11:47:07.560021 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:07.554106    2563 flags.go:64] FLAG: --feature-gates=""
Sep 07 11:47:07.560021 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:07.554110    2563 flags.go:64] FLAG: --file-check-frequency="20s"
Sep 07 11:47:07.560021 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:07.554113    2563 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Sep 07 11:47:07.560021 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:07.554116    2563 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Sep 07 11:47:07.560021 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:07.554119    2563 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Sep 07 11:47:07.560021 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:07.554122    2563 flags.go:64] FLAG: --healthz-port="10248"
Sep 07 11:47:07.560021 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:07.554125    2563 flags.go:64] FLAG: --help="false"
Sep 07 11:47:07.560021 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:07.554127    2563 flags.go:64] FLAG: --hostname-override="ip-10-0-135-173.ec2.internal"
Sep 07 11:47:07.560021 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:07.554132    2563 flags.go:64] FLAG: --housekeeping-interval="10s"
Sep 07 11:47:07.560021 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:07.554135    2563 flags.go:64] FLAG: --http-check-frequency="20s"
Sep 07 11:47:07.560021 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:07.554138    2563 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Sep 07 11:47:07.560021 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:07.554141    2563 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Sep 07 11:47:07.560021 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:07.554144    2563 flags.go:64] FLAG: --image-gc-high-threshold="85"
Sep 07 11:47:07.560021 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:07.554147    2563 flags.go:64] FLAG: --image-gc-low-threshold="80"
Sep 07 11:47:07.560021 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:07.554150    2563 flags.go:64] FLAG: --image-service-endpoint=""
Sep 07 11:47:07.560021 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:07.554152    2563 flags.go:64] FLAG: --kernel-memcg-notification="false"
Sep 07 11:47:07.560021 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:07.554155    2563 flags.go:64] FLAG: --kube-api-burst="100"
Sep 07 11:47:07.560021 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:07.554159    2563 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Sep 07 11:47:07.560622 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:07.554162    2563 flags.go:64] FLAG: --kube-api-qps="50"
Sep 07 11:47:07.560622 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:07.554165    2563 flags.go:64] FLAG: --kube-reserved=""
Sep 07 11:47:07.560622 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:07.554167    2563 flags.go:64] FLAG: --kube-reserved-cgroup=""
Sep 07 11:47:07.560622 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:07.554170    2563 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Sep 07 11:47:07.560622 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:07.554173    2563 flags.go:64] FLAG: --kubelet-cgroups=""
Sep 07 11:47:07.560622 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:07.554176    2563 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Sep 07 11:47:07.560622 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:07.554181    2563 flags.go:64] FLAG: --lock-file=""
Sep 07 11:47:07.560622 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:07.554183    2563 flags.go:64] FLAG: --log-cadvisor-usage="false"
Sep 07 11:47:07.560622 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:07.554186    2563 flags.go:64] FLAG: --log-flush-frequency="5s"
Sep 07 11:47:07.560622 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:07.554189    2563 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Sep 07 11:47:07.560622 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:07.554195    2563 flags.go:64] FLAG: --log-json-split-stream="false"
Sep 07 11:47:07.560622 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:07.554197    2563 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Sep 07 11:47:07.560622 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:07.554200    2563 flags.go:64] FLAG: --log-text-split-stream="false"
Sep 07 11:47:07.560622 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:07.554203    2563 flags.go:64] FLAG: --logging-format="text"
Sep 07 11:47:07.560622 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:07.554206    2563 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Sep 07 11:47:07.560622 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:07.554209    2563 flags.go:64] FLAG: --make-iptables-util-chains="true"
Sep 07 11:47:07.560622 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:07.554212    2563 flags.go:64] FLAG: --manifest-url=""
Sep 07 11:47:07.560622 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:07.554215    2563 flags.go:64] FLAG: --manifest-url-header=""
Sep 07 11:47:07.560622 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:07.554219    2563 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Sep 07 11:47:07.560622 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:07.554222    2563 flags.go:64] FLAG: --max-open-files="1000000"
Sep 07 11:47:07.560622 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:07.554225    2563 flags.go:64] FLAG: --max-pods="110"
Sep 07 11:47:07.560622 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:07.554228    2563 flags.go:64] FLAG: --maximum-dead-containers="-1"
Sep 07 11:47:07.560622 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:07.554232    2563 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Sep 07 11:47:07.560622 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:07.554235    2563 flags.go:64] FLAG: --memory-manager-policy="None"
Sep 07 11:47:07.560622 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:07.554238    2563 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Sep 07 11:47:07.561271 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:07.554241    2563 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Sep 07 11:47:07.561271 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:07.554244    2563 flags.go:64] FLAG: --node-ip="0.0.0.0"
Sep 07 11:47:07.561271 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:07.554247    2563 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Sep 07 11:47:07.561271 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:07.554253    2563 flags.go:64] FLAG: --node-status-max-images="50"
Sep 07 11:47:07.561271 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:07.554256    2563 flags.go:64] FLAG: --node-status-update-frequency="10s"
Sep 07 11:47:07.561271 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:07.554276    2563 flags.go:64] FLAG: --oom-score-adj="-999"
Sep 07 11:47:07.561271 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:07.554280    2563 flags.go:64] FLAG: --pod-cidr=""
Sep 07 11:47:07.561271 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:07.554284    2563 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8ccbacc999dbf9abad8f4009240188312b683f36ced4763eed166c1500c164b1"
Sep 07 11:47:07.561271 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:07.554289    2563 flags.go:64] FLAG: --pod-manifest-path=""
Sep 07 11:47:07.561271 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:07.554293    2563 flags.go:64] FLAG: --pod-max-pids="-1"
Sep 07 11:47:07.561271 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:07.554296    2563 flags.go:64] FLAG: --pods-per-core="0"
Sep 07 11:47:07.561271 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:07.554298    2563 flags.go:64] FLAG: --port="10250"
Sep 07 11:47:07.561271 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:07.554301    2563 flags.go:64] FLAG: --protect-kernel-defaults="false"
Sep 07 11:47:07.561271 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:07.554304    2563 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0c2638340cb7cb0c3"
Sep 07 11:47:07.561271 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:07.554309    2563 flags.go:64] FLAG: --qos-reserved=""
Sep 07 11:47:07.561271 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:07.554312    2563 flags.go:64] FLAG: --read-only-port="10255"
Sep 07 11:47:07.561271 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:07.554315    2563 flags.go:64] FLAG: --register-node="true"
Sep 07 11:47:07.561271 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:07.554318    2563 flags.go:64] FLAG: --register-with-taints=""
Sep 07 11:47:07.561271 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:07.554321    2563 flags.go:64] FLAG: --registry-burst="10"
Sep 07 11:47:07.561271 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:07.554324    2563 flags.go:64] FLAG: --registry-qps="5"
Sep 07 11:47:07.561271 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:07.554327    2563 flags.go:64] FLAG: --reserved-cpus=""
Sep 07 11:47:07.561271 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:07.554330    2563 flags.go:64] FLAG: --reserved-memory=""
Sep 07 11:47:07.561271 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:07.554333    2563 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Sep 07 11:47:07.561271 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:07.554336    2563 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Sep 07 11:47:07.561271 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:07.554339    2563 flags.go:64] FLAG: --rotate-certificates="false"
Sep 07 11:47:07.561901 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:07.554342    2563 flags.go:64] FLAG: --rotate-server-certificates="false"
Sep 07 11:47:07.561901 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:07.554344    2563 flags.go:64] FLAG: --runonce="false"
Sep 07 11:47:07.561901 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:07.554347    2563 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Sep 07 11:47:07.561901 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:07.554350    2563 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Sep 07 11:47:07.561901 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:07.554355    2563 flags.go:64] FLAG: --seccomp-default="false"
Sep 07 11:47:07.561901 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:07.554358    2563 flags.go:64] FLAG: --serialize-image-pulls="true"
Sep 07 11:47:07.561901 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:07.554360    2563 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Sep 07 11:47:07.561901 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:07.554363    2563 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Sep 07 11:47:07.561901 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:07.554366    2563 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Sep 07 11:47:07.561901 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:07.554370    2563 flags.go:64] FLAG: --storage-driver-password="root"
Sep 07 11:47:07.561901 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:07.554372    2563 flags.go:64] FLAG: --storage-driver-secure="false"
Sep 07 11:47:07.561901 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:07.554375    2563 flags.go:64] FLAG: --storage-driver-table="stats"
Sep 07 11:47:07.561901 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:07.554378    2563 flags.go:64] FLAG: --storage-driver-user="root"
Sep 07 11:47:07.561901 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:07.554381    2563 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Sep 07 11:47:07.561901 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:07.554384    2563 flags.go:64] FLAG: --sync-frequency="1m0s"
Sep 07 11:47:07.561901 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:07.554392    2563 flags.go:64] FLAG: --system-cgroups=""
Sep 07 11:47:07.561901 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:07.554395    2563 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Sep 07 11:47:07.561901 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:07.554400    2563 flags.go:64] FLAG: --system-reserved-cgroup=""
Sep 07 11:47:07.561901 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:07.554403    2563 flags.go:64] FLAG: --tls-cert-file=""
Sep 07 11:47:07.561901 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:07.554405    2563 flags.go:64] FLAG: --tls-cipher-suites="[]"
Sep 07 11:47:07.561901 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:07.554409    2563 flags.go:64] FLAG: --tls-min-version=""
Sep 07 11:47:07.561901 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:07.554413    2563 flags.go:64] FLAG: --tls-private-key-file=""
Sep 07 11:47:07.561901 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:07.554416    2563 flags.go:64] FLAG: --topology-manager-policy="none"
Sep 07 11:47:07.561901 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:07.554419    2563 flags.go:64] FLAG: --topology-manager-policy-options=""
Sep 07 11:47:07.561901 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:07.554422    2563 flags.go:64] FLAG: --topology-manager-scope="container"
Sep 07 11:47:07.562566 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:07.554425    2563 flags.go:64] FLAG: --v="2"
Sep 07 11:47:07.562566 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:07.554428    2563 flags.go:64] FLAG: --version="false"
Sep 07 11:47:07.562566 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:07.554432    2563 flags.go:64] FLAG: --vmodule=""
Sep 07 11:47:07.562566 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:07.554435    2563 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Sep 07 11:47:07.562566 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:07.554439    2563 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Sep 07 11:47:07.562566 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.554544    2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 07 11:47:07.562566 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.554548    2563 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 07 11:47:07.562566 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.554551    2563 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 07 11:47:07.562566 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.554554    2563 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 07 11:47:07.562566 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.554556    2563 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 07 11:47:07.562566 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.554559    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 07 11:47:07.562566 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.554563    2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 07 11:47:07.562566 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.554566    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 07 11:47:07.562566 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.554568    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 07 11:47:07.562566 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.554571    2563 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 07 11:47:07.562566 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.554573    2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 07 11:47:07.562566 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.554576    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 07 11:47:07.562566 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.554579    2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 07 11:47:07.562566 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.554581    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 07 11:47:07.562566 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.554584    2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 07 11:47:07.562566 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.554586    2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 07 11:47:07.563074 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.554589    2563 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 07 11:47:07.563074 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.554591    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 07 11:47:07.563074 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.554594    2563 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 07 11:47:07.563074 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.554596    2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 07 11:47:07.563074 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.554599    2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 07 11:47:07.563074 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.554602    2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 07 11:47:07.563074 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.554604    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 07 11:47:07.563074 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.554608    2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 07 11:47:07.563074 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.554612    2563 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 07 11:47:07.563074 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.554615    2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 07 11:47:07.563074 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.554618    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 07 11:47:07.563074 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.554621    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 07 11:47:07.563074 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.554624    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 07 11:47:07.563074 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.554627    2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 07 11:47:07.563074 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.554630    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 07 11:47:07.563074 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.554632    2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 07 11:47:07.563074 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.554635    2563 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 07 11:47:07.563074 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.554638    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 07 11:47:07.563074 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.554640    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 07 11:47:07.563565 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.554644    2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 07 11:47:07.563565 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.554647    2563 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 07 11:47:07.563565 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.554650    2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 07 11:47:07.563565 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.554654    2563 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 07 11:47:07.563565 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.554656    2563 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 07 11:47:07.563565 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.554659    2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 07 11:47:07.563565 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.554662    2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 07 11:47:07.563565 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.554665    2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 07 11:47:07.563565 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.554668    2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 07 11:47:07.563565 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.554670    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 07 11:47:07.563565 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.554673    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 07 11:47:07.563565 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.554675    2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 07 11:47:07.563565 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.554679    2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 07 11:47:07.563565 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.554682    2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 07 11:47:07.563565 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.554685    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 07 11:47:07.563565 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.554687    2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 07 11:47:07.563565 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.554690    2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 07 11:47:07.563565 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.554693    2563 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 07 11:47:07.563565 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.554695    2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 07 11:47:07.564026 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.554698    2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 07 11:47:07.564026 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.554702    2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 07 11:47:07.564026 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.554705    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 07 11:47:07.564026 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.554707    2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 07 11:47:07.564026 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.554710    2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 07 11:47:07.564026 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.554712    2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 07 11:47:07.564026 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.554715    2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 07 11:47:07.564026 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.554718    2563 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 07 11:47:07.564026 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.554721    2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 07 11:47:07.564026 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.554723    2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 07 11:47:07.564026 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.554726    2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 07 11:47:07.564026 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.554728    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 07 11:47:07.564026 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.554731    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 07 11:47:07.564026 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.554734    2563 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 07 11:47:07.564026 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.554736    2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 07 11:47:07.564026 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.554738    2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 07 11:47:07.564026 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.554742    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 07 11:47:07.564026 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.554745    2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 07 11:47:07.564026 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.554747    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 07 11:47:07.564026 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.554750    2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 07 11:47:07.564520 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.554752    2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 07 11:47:07.564520 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.554755    2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 07 11:47:07.564520 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.554758    2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 07 11:47:07.564520 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.554760    2563 feature_gate.go:328] unrecognized feature gate: Example
Sep 07 11:47:07.564520 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.554762    2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 07 11:47:07.564520 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.554765    2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 07 11:47:07.564520 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.554768    2563 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 07 11:47:07.564520 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.554770    2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 07 11:47:07.564520 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.554773    2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 07 11:47:07.564520 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.554775    2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 07 11:47:07.564520 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.554778    2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 07 11:47:07.564520 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.554780    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 07 11:47:07.564520 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.554783    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 07 11:47:07.564520 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.554787    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 07 11:47:07.564520 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.554789    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 07 11:47:07.564520 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.554792    2563 feature_gate.go:328] unrecognized feature gate: Example2
Sep 07 11:47:07.564520 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.554795    2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 07 11:47:07.564520 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.554798    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 07 11:47:07.564520 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.554800    2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 07 11:47:07.564520 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.554803    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 07 11:47:07.565027 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.554805    2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 07 11:47:07.565027 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.554808    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 07 11:47:07.565027 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.554810    2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 07 11:47:07.565027 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.554813    2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 07 11:47:07.565027 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.554816    2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 07 11:47:07.565027 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:07.555380    2563 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 07 11:47:07.565027 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:07.562629    2563 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Sep 07 11:47:07.565027 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:07.562646    2563 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Sep 07 11:47:07.565027 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:07.563432    2563 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Sep 07 11:47:07.565027 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:07.563439    2563 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Sep 07 11:47:07.565027 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.563514    2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 07 11:47:07.565027 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.563520    2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 07 11:47:07.565027 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.563523    2563 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 07 11:47:07.565027 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.563526    2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 07 11:47:07.565397 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.563530    2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 07 11:47:07.565397 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.563535    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 07 11:47:07.565397 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.563538    2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 07 11:47:07.565397 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.563542    2563 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 07 11:47:07.565397 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.563545    2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 07 11:47:07.565397 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.563548    2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 07 11:47:07.565397 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.563551    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 07 11:47:07.565397 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.563553    2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 07 11:47:07.565397 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.563556    2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 07 11:47:07.565397 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.563559    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 07 11:47:07.565397 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.563561    2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 07 11:47:07.565397 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.563564    2563 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 07 11:47:07.565397 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.563566    2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 07 11:47:07.565397 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.563569    2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 07 11:47:07.565397 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.563572    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 07 11:47:07.565397 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.563574    2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 07 11:47:07.565397 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.563577    2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 07 11:47:07.565397 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.563580    2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 07 11:47:07.565397 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.563582    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 07 11:47:07.565903 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.563585    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 07 11:47:07.565903 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.563588    2563 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 07 11:47:07.565903 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.563591    2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 07 11:47:07.565903 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.563593    2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 07 11:47:07.565903 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.563596    2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 07 11:47:07.565903 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.563599    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 07 11:47:07.565903 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.563607    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 07 11:47:07.565903 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.563610    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 07 11:47:07.565903 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.563613    2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 07 11:47:07.565903 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.563615    2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 07 11:47:07.565903 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.563618    2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 07 11:47:07.565903 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.563620    2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 07 11:47:07.565903 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.563623    2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 07 11:47:07.565903 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.563626    2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 07 11:47:07.565903 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.563628    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 07 11:47:07.565903 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.563631    2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 07 11:47:07.565903 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.563633    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 07 11:47:07.565903 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.563636    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 07 11:47:07.565903 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.563638    2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 07 11:47:07.565903 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.563641    2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 07 11:47:07.566398 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.563645    2563 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 07 11:47:07.566398 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.563648    2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 07 11:47:07.566398 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.563651    2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 07 11:47:07.566398 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.563653    2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 07 11:47:07.566398 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.563656    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 07 11:47:07.566398 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.563659    2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 07 11:47:07.566398 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.563661    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 07 11:47:07.566398 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.563664    2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 07 11:47:07.566398 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.563666    2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 07 11:47:07.566398 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.563669    2563 feature_gate.go:328] unrecognized feature gate: Example2
Sep 07 11:47:07.566398 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.563672    2563 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 07 11:47:07.566398 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.563674    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 07 11:47:07.566398 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.563677    2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 07 11:47:07.566398 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.563680    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 07 11:47:07.566398 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.563683    2563 feature_gate.go:328] unrecognized feature gate: Example
Sep 07 11:47:07.566398 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.563685    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 07 11:47:07.566398 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.563688    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 07 11:47:07.566398 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.563690    2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 07 11:47:07.566398 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.563693    2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 07 11:47:07.566867 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.563702    2563 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 07 11:47:07.566867 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.563704    2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 07 11:47:07.566867 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.563707    2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 07 11:47:07.566867 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.563710    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 07 11:47:07.566867 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.563712    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 07 11:47:07.566867 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.563715    2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 07 11:47:07.566867 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.563717    2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 07 11:47:07.566867 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.563720    2563 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 07 11:47:07.566867 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.563722    2563 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 07 11:47:07.566867 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.563725    2563 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 07 11:47:07.566867 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.563727    2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 07 11:47:07.566867 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.563730    2563 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 07 11:47:07.566867 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.563732    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 07 11:47:07.566867 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.563735    2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 07 11:47:07.566867 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.563738    2563 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 07 11:47:07.566867 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.563740    2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 07 11:47:07.566867 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.563743    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 07 11:47:07.566867 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.563745    2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 07 11:47:07.566867 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.563748    2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 07 11:47:07.566867 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.563751    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 07 11:47:07.567387 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.563753    2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 07 11:47:07.567387 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.563756    2563 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 07 11:47:07.567387 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.563758    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 07 11:47:07.567387 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.563761    2563 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 07 11:47:07.567387 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.563763    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 07 11:47:07.567387 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.563766    2563 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 07 11:47:07.567387 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.563769    2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 07 11:47:07.567387 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.563771    2563 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 07 11:47:07.567387 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.563774    2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 07 11:47:07.567387 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.563777    2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 07 11:47:07.567387 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.563779    2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 07 11:47:07.567387 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.563783    2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 07 11:47:07.567387 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.563785    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 07 11:47:07.567387 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.563793    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 07 11:47:07.567387 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.563797    2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 07 11:47:07.567387 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.563801    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 07 11:47:07.567387 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.563803    2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 07 11:47:07.567794 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:07.563808    2563 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 07 11:47:07.567794 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.563955    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 07 11:47:07.567794 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.563959    2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 07 11:47:07.567794 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.563963    2563 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 07 11:47:07.567794 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.563966    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 07 11:47:07.567794 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.563969    2563 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 07 11:47:07.567794 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.563972    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 07 11:47:07.567794 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.563975    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 07 11:47:07.567794 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.563977    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 07 11:47:07.567794 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.563980    2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 07 11:47:07.567794 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.563983    2563 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 07 11:47:07.567794 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.563986    2563 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 07 11:47:07.567794 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.563988    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 07 11:47:07.567794 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.563991    2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 07 11:47:07.567794 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.563994    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 07 11:47:07.568172 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.563997    2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 07 11:47:07.568172 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.563999    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 07 11:47:07.568172 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.564003    2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 07 11:47:07.568172 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.564005    2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 07 11:47:07.568172 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.564008    2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 07 11:47:07.568172 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.564010    2563 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 07 11:47:07.568172 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.564013    2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 07 11:47:07.568172 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.564021    2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 07 11:47:07.568172 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.564024    2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 07 11:47:07.568172 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.564027    2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 07 11:47:07.568172 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.564030    2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 07 11:47:07.568172 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.564033    2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 07 11:47:07.568172 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.564037    2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 07 11:47:07.568172 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.564040    2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 07 11:47:07.568172 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.564050    2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 07 11:47:07.568172 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.564053    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 07 11:47:07.568172 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.564056    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 07 11:47:07.568172 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.564058    2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 07 11:47:07.568172 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.564061    2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 07 11:47:07.568656 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.564064    2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 07 11:47:07.568656 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.564066    2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 07 11:47:07.568656 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.564069    2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 07 11:47:07.568656 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.564071    2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 07 11:47:07.568656 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.564074    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 07 11:47:07.568656 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.564076    2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 07 11:47:07.568656 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.564079    2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 07 11:47:07.568656 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.564081    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 07 11:47:07.568656 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.564084    2563 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 07 11:47:07.568656 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.564086    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 07 11:47:07.568656 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.564089    2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 07 11:47:07.568656 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.564091    2563 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 07 11:47:07.568656 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.564094    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 07 11:47:07.568656 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.564096    2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 07 11:47:07.568656 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.564099    2563 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 07 11:47:07.568656 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.564101    2563 feature_gate.go:328] unrecognized feature gate: Example
Sep 07 11:47:07.568656 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.564103    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 07 11:47:07.568656 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.564106    2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 07 11:47:07.568656 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.564108    2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 07 11:47:07.568656 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.564111    2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 07 11:47:07.569138 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.564114    2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 07 11:47:07.569138 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.564117    2563 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 07 11:47:07.569138 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.564119    2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 07 11:47:07.569138 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.564122    2563 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 07 11:47:07.569138 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.564125    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 07 11:47:07.569138 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.564127    2563 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 07 11:47:07.569138 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.564130    2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 07 11:47:07.569138 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.564132    2563 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 07 11:47:07.569138 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.564149    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 07 11:47:07.569138 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.564152    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 07 11:47:07.569138 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.564154    2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 07 11:47:07.569138 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.564157    2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 07 11:47:07.569138 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.564159    2563 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 07 11:47:07.569138 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.564163    2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 07 11:47:07.569138 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.564166    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 07 11:47:07.569138 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.564168    2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 07 11:47:07.569138 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.564171    2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 07 11:47:07.569138 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.564173    2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 07 11:47:07.569138 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.564176    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 07 11:47:07.569705 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.564178    2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 07 11:47:07.569705 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.564181    2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 07 11:47:07.569705 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.564183    2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 07 11:47:07.569705 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.564186    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 07 11:47:07.569705 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.564188    2563 feature_gate.go:328] unrecognized feature gate: Example2
Sep 07 11:47:07.569705 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.564191    2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 07 11:47:07.569705 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.564195    2563 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 07 11:47:07.569705 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.564198    2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 07 11:47:07.569705 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.564201    2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 07 11:47:07.569705 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.564204    2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 07 11:47:07.569705 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.564206    2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 07 11:47:07.569705 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.564209    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 07 11:47:07.569705 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.564211    2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 07 11:47:07.569705 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.564214    2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 07 11:47:07.569705 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.564217    2563 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 07 11:47:07.569705 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.564219    2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 07 11:47:07.569705 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.564222    2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 07 11:47:07.569705 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.564225    2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 07 11:47:07.569705 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.564227    2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 07 11:47:07.569705 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.564230    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 07 11:47:07.570192 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.564232    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 07 11:47:07.570192 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.564235    2563 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 07 11:47:07.570192 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.564243    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 07 11:47:07.570192 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.564246    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 07 11:47:07.570192 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.564248    2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 07 11:47:07.570192 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.564251    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 07 11:47:07.570192 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:07.564253    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 07 11:47:07.570192 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:07.564270    2563 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 07 11:47:07.570192 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:07.564377    2563 server.go:962] "Client rotation is on, will bootstrap in background"
Sep 07 11:47:07.570192 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:07.566600    2563 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Sep 07 11:47:07.570192 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:07.567552    2563 server.go:1019] "Starting client certificate rotation"
Sep 07 11:47:07.570192 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:07.567651    2563 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 07 11:47:07.570192 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:07.567689    2563 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 07 11:47:07.570541 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:07.570529    2563 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 07 11:47:07.578638 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:07.578612    2563 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 07 11:47:07.590974 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:07.590954    2563 log.go:25] "Validated CRI v1 runtime API"
Sep 07 11:47:07.594121 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:07.594106    2563 log.go:25] "Validated CRI v1 image API"
Sep 07 11:47:07.595005 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:07.594986    2563 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Sep 07 11:47:07.598382 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:07.598364    2563 fs.go:135] Filesystem UUIDs: map[384ee0f6-3b16-4a2e-aa60-d8c9793b91e1:/dev/nvme0n1p4 5894113c-9769-4e6a-b2c5-d342ad5ed167:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2]
Sep 07 11:47:07.598453 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:07.598381    2563 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Sep 07 11:47:07.598911 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:07.598894    2563 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 07 11:47:07.603390 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:07.603286    2563 manager.go:216] Machine: {Timestamp:2026-09-07 11:47:07.601864965 +0000 UTC m=+0.325752227 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3108823 MemoryCapacity:33164480512 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec29efcff806e1509e07131a9d0915bb SystemUUID:ec29efcf-f806-e150-9e07-131a9d0915bb BootID:7306a6ae-d61c-46a7-a988-d73439818d69 Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582238208 Type:vfs Inodes:4048398 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582242304 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:42:c0:65:bb:07 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:42:c0:65:bb:07 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:5e:64:c3:4f:49:d4 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164480512 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Sep 07 11:47:07.603390 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:07.603385    2563 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Sep 07 11:47:07.603507 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:07.603456    2563 manager.go:232] Version: {KernelVersion:5.14.0-570.138.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260901-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Sep 07 11:47:07.606608 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:07.606583    2563 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Sep 07 11:47:07.606731 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:07.606610    2563 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-135-173.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Sep 07 11:47:07.607373 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:07.607364    2563 topology_manager.go:138] "Creating topology manager with none policy"
Sep 07 11:47:07.607404 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:07.607376    2563 container_manager_linux.go:309] "Creating device plugin manager"
Sep 07 11:47:07.607404 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:07.607397    2563 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 07 11:47:07.608178 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:07.608169    2563 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 07 11:47:07.609507 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:07.609497    2563 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Sep 07 11:47:07.613245 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:07.613235    2563 state_mem.go:36] "Initialized new in-memory state store"
Sep 07 11:47:07.613361 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:07.613351    2563 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Sep 07 11:47:07.613401 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:07.613380    2563 kubelet.go:486] "Attempting to sync node with API server"
Sep 07 11:47:07.613401 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:07.613389    2563 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Sep 07 11:47:07.613401 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:07.613399    2563 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Sep 07 11:47:07.613517 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:07.613408    2563 kubelet.go:398] "Adding apiserver pod source"
Sep 07 11:47:07.613517 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:07.613416    2563 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Sep 07 11:47:07.613827 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:07.613813    2563 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 07 11:47:07.613917 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:07.613842    2563 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 07 11:47:07.615758 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:07.615731    2563 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.12-5.rhaos4.21.gitd1ec63c.el9" apiVersion="v1"
Sep 07 11:47:07.617982 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:07.617967    2563 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Sep 07 11:47:07.618039 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:07.617997    2563 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Sep 07 11:47:07.618152 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:07.618142    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Sep 07 11:47:07.618193 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:07.618157    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Sep 07 11:47:07.618193 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:07.618163    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Sep 07 11:47:07.618193 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:07.618168    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Sep 07 11:47:07.618193 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:07.618176    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Sep 07 11:47:07.618193 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:07.618181    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Sep 07 11:47:07.618789 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:07.618779    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Sep 07 11:47:07.618789 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:07.618788    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Sep 07 11:47:07.618854 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:07.618796    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Sep 07 11:47:07.618854 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:07.618801    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Sep 07 11:47:07.618854 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:07.618809    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Sep 07 11:47:07.618854 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:07.618816    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Sep 07 11:47:07.619328 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:07.619312    2563 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-wf4gj"
Sep 07 11:47:07.619474 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:07.619466    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Sep 07 11:47:07.619514 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:07.619475    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Sep 07 11:47:07.623757 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:07.623731    2563 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-173.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Sep 07 11:47:07.623894 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:07.623880    2563 server.go:1268] "Started kubelet"
Sep 07 11:47:07.623987 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:07.623963    2563 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Sep 07 11:47:07.624038 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:07.623962    2563 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Sep 07 11:47:07.624038 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:07.624016    2563 server_v1.go:49] "podresources" method="list" useActivePods=true
Sep 07 11:47:07.624350 ip-10-0-135-173 kubenswrapper[2563]: E0907 11:47:07.624319    2563 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-135-173.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Sep 07 11:47:07.624350 ip-10-0-135-173 kubenswrapper[2563]: E0907 11:47:07.624320    2563 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Sep 07 11:47:07.624709 ip-10-0-135-173 systemd[1]: Started Kubernetes Kubelet.
Sep 07 11:47:07.625202 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:07.625186    2563 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Sep 07 11:47:07.626189 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:07.626170    2563 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-wf4gj"
Sep 07 11:47:07.628755 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:07.628737    2563 server.go:310] "Adding debug handlers to kubelet server"
Sep 07 11:47:07.630932 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:07.630913    2563 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Sep 07 11:47:07.631543 ip-10-0-135-173 kubenswrapper[2563]: E0907 11:47:07.630047    2563 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-135-173.ec2.internal.18d30769b743e017  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-135-173.ec2.internal,UID:ip-10-0-135-173.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-135-173.ec2.internal,},FirstTimestamp:2026-09-07 11:47:07.623858199 +0000 UTC m=+0.347745461,LastTimestamp:2026-09-07 11:47:07.623858199 +0000 UTC m=+0.347745461,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-135-173.ec2.internal,}"
Sep 07 11:47:07.631543 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:07.631533    2563 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Sep 07 11:47:07.631692 ip-10-0-135-173 kubenswrapper[2563]: E0907 11:47:07.631574    2563 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Sep 07 11:47:07.631692 ip-10-0-135-173 kubenswrapper[2563]: E0907 11:47:07.631596    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-173.ec2.internal\" not found"
Sep 07 11:47:07.632146 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:07.632130    2563 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Sep 07 11:47:07.632146 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:07.632132    2563 volume_manager.go:311] "The desired_state_of_world populator starts"
Sep 07 11:47:07.632300 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:07.632156    2563 volume_manager.go:313] "Starting Kubelet Volume Manager"
Sep 07 11:47:07.632300 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:07.632250    2563 factory.go:55] Registering systemd factory
Sep 07 11:47:07.632300 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:07.632274    2563 reconstruct.go:97] "Volume reconstruction finished"
Sep 07 11:47:07.632300 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:07.632281    2563 factory.go:223] Registration of the systemd container factory successfully
Sep 07 11:47:07.632300 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:07.632285    2563 reconciler.go:29] "Reconciler: start to sync state"
Sep 07 11:47:07.632509 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:07.632495    2563 factory.go:153] Registering CRI-O factory
Sep 07 11:47:07.632553 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:07.632513    2563 factory.go:223] Registration of the crio container factory successfully
Sep 07 11:47:07.632586 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:07.632557    2563 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Sep 07 11:47:07.632624 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:07.632588    2563 factory.go:103] Registering Raw factory
Sep 07 11:47:07.632624 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:07.632604    2563 manager.go:1196] Started watching for new ooms in manager
Sep 07 11:47:07.632921 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:07.632911    2563 manager.go:349] Starting recovery of all containers
Sep 07 11:47:07.637187 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:07.637162    2563 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 07 11:47:07.641683 ip-10-0-135-173 kubenswrapper[2563]: E0907 11:47:07.641284    2563 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-135-173.ec2.internal\" not found" node="ip-10-0-135-173.ec2.internal"
Sep 07 11:47:07.641683 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:07.641503    2563 manager.go:354] Recovery completed
Sep 07 11:47:07.648453 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:07.648440    2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 07 11:47:07.650229 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:07.650214    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-173.ec2.internal" event="NodeHasSufficientMemory"
Sep 07 11:47:07.650322 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:07.650243    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-173.ec2.internal" event="NodeHasNoDiskPressure"
Sep 07 11:47:07.650322 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:07.650276    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-173.ec2.internal" event="NodeHasSufficientPID"
Sep 07 11:47:07.650919 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:07.650903    2563 cpu_manager.go:222] "Starting CPU manager" policy="none"
Sep 07 11:47:07.650985 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:07.650920    2563 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Sep 07 11:47:07.650985 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:07.650923    2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 07 11:47:07.650985 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:07.650939    2563 state_mem.go:36] "Initialized new in-memory state store"
Sep 07 11:47:07.651769 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:07.651752    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-173.ec2.internal" event="NodeHasSufficientMemory"
Sep 07 11:47:07.651848 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:07.651782    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-173.ec2.internal" event="NodeHasNoDiskPressure"
Sep 07 11:47:07.651848 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:07.651795    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-173.ec2.internal" event="NodeHasSufficientPID"
Sep 07 11:47:07.732486 ip-10-0-135-173 kubenswrapper[2563]: E0907 11:47:07.732462    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-173.ec2.internal\" not found"
Sep 07 11:47:07.796759 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:07.796706    2563 policy_none.go:49] "None policy: Start"
Sep 07 11:47:07.796759 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:07.796725    2563 memory_manager.go:187] "Starting memorymanager" policy="None"
Sep 07 11:47:07.796759 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:07.796737    2563 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Sep 07 11:47:07.798695 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:07.798679    2563 policy_none.go:47] "Start"
Sep 07 11:47:07.827454 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:07.827431    2563 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Sep 07 11:47:07.828560 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:07.828541    2563 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Sep 07 11:47:07.828633 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:07.828599    2563 status_manager.go:244] "Starting to sync pod status with apiserver"
Sep 07 11:47:07.828633 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:07.828629    2563 kubelet.go:2448] "Starting kubelet main sync loop"
Sep 07 11:47:07.828727 ip-10-0-135-173 kubenswrapper[2563]: E0907 11:47:07.828710    2563 kubelet.go:2472] "Skipping pod synchronization" err="[container runtime status check may not have completed yet, PLEG is not healthy: pleg has yet to be successful]"
Sep 07 11:47:07.830016 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:07.830002    2563 manager.go:337] "Starting Device Plugin manager"
Sep 07 11:47:07.830106 ip-10-0-135-173 kubenswrapper[2563]: E0907 11:47:07.830052    2563 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Sep 07 11:47:07.830106 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:07.830063    2563 server.go:87] "Starting device plugin registration server"
Sep 07 11:47:07.830419 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:07.830407    2563 eviction_manager.go:189] "Eviction manager: starting control loop"
Sep 07 11:47:07.830489 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:07.830422    2563 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Sep 07 11:47:07.830607 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:07.830595    2563 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Sep 07 11:47:07.830674 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:07.830663    2563 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Sep 07 11:47:07.830724 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:07.830675    2563 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Sep 07 11:47:07.831565 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:07.831128    2563 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 07 11:47:07.831565 ip-10-0-135-173 kubenswrapper[2563]: E0907 11:47:07.831129    2563 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Sep 07 11:47:07.831565 ip-10-0-135-173 kubenswrapper[2563]: E0907 11:47:07.831211    2563 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-135-173.ec2.internal\" not found"
Sep 07 11:47:07.929671 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:07.929647    2563 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-173.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-135-173.ec2.internal"]
Sep 07 11:47:07.929750 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:07.929708    2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 07 11:47:07.930874 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:07.930862    2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 07 11:47:07.931181 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:07.931167    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-173.ec2.internal" event="NodeHasSufficientMemory"
Sep 07 11:47:07.931248 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:07.931193    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-173.ec2.internal" event="NodeHasNoDiskPressure"
Sep 07 11:47:07.931248 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:07.931206    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-173.ec2.internal" event="NodeHasSufficientPID"
Sep 07 11:47:07.931523 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:07.931505    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-173.ec2.internal" event="NodeHasSufficientMemory"
Sep 07 11:47:07.931565 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:07.931542    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-173.ec2.internal" event="NodeHasNoDiskPressure"
Sep 07 11:47:07.931565 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:07.931558    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-173.ec2.internal" event="NodeHasSufficientPID"
Sep 07 11:47:07.931633 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:07.931577    2563 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-135-173.ec2.internal"
Sep 07 11:47:07.933461 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:07.933442    2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 07 11:47:07.933567 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:07.933552    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-173.ec2.internal"
Sep 07 11:47:07.933613 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:07.933588    2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 07 11:47:07.934103 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:07.934085    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-173.ec2.internal" event="NodeHasSufficientMemory"
Sep 07 11:47:07.934198 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:07.934116    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-173.ec2.internal" event="NodeHasNoDiskPressure"
Sep 07 11:47:07.934198 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:07.934132    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-173.ec2.internal" event="NodeHasSufficientPID"
Sep 07 11:47:07.934198 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:07.934091    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-173.ec2.internal" event="NodeHasSufficientMemory"
Sep 07 11:47:07.934198 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:07.934190    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-173.ec2.internal" event="NodeHasNoDiskPressure"
Sep 07 11:47:07.934198 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:07.934200    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-173.ec2.internal" event="NodeHasSufficientPID"
Sep 07 11:47:07.936337 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:07.936323    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-173.ec2.internal"
Sep 07 11:47:07.936402 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:07.936346    2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 07 11:47:07.936969 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:07.936953    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-173.ec2.internal" event="NodeHasSufficientMemory"
Sep 07 11:47:07.937032 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:07.936983    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-173.ec2.internal" event="NodeHasNoDiskPressure"
Sep 07 11:47:07.937032 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:07.936993    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-173.ec2.internal" event="NodeHasSufficientPID"
Sep 07 11:47:07.939851 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:07.939836    2563 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-135-173.ec2.internal"
Sep 07 11:47:07.939908 ip-10-0-135-173 kubenswrapper[2563]: E0907 11:47:07.939856    2563 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-135-173.ec2.internal\": node \"ip-10-0-135-173.ec2.internal\" not found"
Sep 07 11:47:07.956172 ip-10-0-135-173 kubenswrapper[2563]: E0907 11:47:07.956152    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-173.ec2.internal\" not found"
Sep 07 11:47:07.963824 ip-10-0-135-173 kubenswrapper[2563]: E0907 11:47:07.963804    2563 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-135-173.ec2.internal\" not found" node="ip-10-0-135-173.ec2.internal"
Sep 07 11:47:07.968161 ip-10-0-135-173 kubenswrapper[2563]: E0907 11:47:07.968146    2563 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-135-173.ec2.internal\" not found" node="ip-10-0-135-173.ec2.internal"
Sep 07 11:47:08.033100 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:08.033084    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/0baf97a2450fb410ba0091864614894a-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-135-173.ec2.internal\" (UID: \"0baf97a2450fb410ba0091864614894a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-173.ec2.internal"
Sep 07 11:47:08.033213 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:08.033107    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0baf97a2450fb410ba0091864614894a-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-135-173.ec2.internal\" (UID: \"0baf97a2450fb410ba0091864614894a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-173.ec2.internal"
Sep 07 11:47:08.056302 ip-10-0-135-173 kubenswrapper[2563]: E0907 11:47:08.056244    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-173.ec2.internal\" not found"
Sep 07 11:47:08.133517 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:08.133501    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/7d94e6a6c726d03776a8387d983ae9ef-config\") pod \"kube-apiserver-proxy-ip-10-0-135-173.ec2.internal\" (UID: \"7d94e6a6c726d03776a8387d983ae9ef\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-135-173.ec2.internal"
Sep 07 11:47:08.133592 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:08.133540    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/0baf97a2450fb410ba0091864614894a-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-135-173.ec2.internal\" (UID: \"0baf97a2450fb410ba0091864614894a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-173.ec2.internal"
Sep 07 11:47:08.133592 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:08.133557    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0baf97a2450fb410ba0091864614894a-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-135-173.ec2.internal\" (UID: \"0baf97a2450fb410ba0091864614894a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-173.ec2.internal"
Sep 07 11:47:08.133592 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:08.133582    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0baf97a2450fb410ba0091864614894a-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-135-173.ec2.internal\" (UID: \"0baf97a2450fb410ba0091864614894a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-173.ec2.internal"
Sep 07 11:47:08.133683 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:08.133623    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/0baf97a2450fb410ba0091864614894a-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-135-173.ec2.internal\" (UID: \"0baf97a2450fb410ba0091864614894a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-173.ec2.internal"
Sep 07 11:47:08.156583 ip-10-0-135-173 kubenswrapper[2563]: E0907 11:47:08.156567    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-173.ec2.internal\" not found"
Sep 07 11:47:08.233857 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:08.233841    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/7d94e6a6c726d03776a8387d983ae9ef-config\") pod \"kube-apiserver-proxy-ip-10-0-135-173.ec2.internal\" (UID: \"7d94e6a6c726d03776a8387d983ae9ef\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-135-173.ec2.internal"
Sep 07 11:47:08.233904 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:08.233873    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/7d94e6a6c726d03776a8387d983ae9ef-config\") pod \"kube-apiserver-proxy-ip-10-0-135-173.ec2.internal\" (UID: \"7d94e6a6c726d03776a8387d983ae9ef\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-135-173.ec2.internal"
Sep 07 11:47:08.256975 ip-10-0-135-173 kubenswrapper[2563]: E0907 11:47:08.256960    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-173.ec2.internal\" not found"
Sep 07 11:47:08.267113 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:08.267098    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-173.ec2.internal"
Sep 07 11:47:08.270860 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:08.270834    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-173.ec2.internal"
Sep 07 11:47:08.357835 ip-10-0-135-173 kubenswrapper[2563]: E0907 11:47:08.357774    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-173.ec2.internal\" not found"
Sep 07 11:47:09.657159 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:09.657137    2563 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 07 11:47:09.737164 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:09.737143    2563 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-173.ec2.internal"
Sep 07 11:47:09.748606 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:09.748587    2563 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 07 11:47:09.750377 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:09.750365    2563 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-173.ec2.internal"
Sep 07 11:47:09.756313 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:09.756297    2563 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 07 11:47:09.772291 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:09.772276    2563 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Sep 07 11:47:09.772390 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:09.772374    2563 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 07 11:47:09.772461 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:09.772429    2563 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 07 11:47:09.772461 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:09.772430    2563 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 07 11:47:09.819623 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:09.819572    2563 apiserver.go:52] "Watching apiserver"
Sep 07 11:47:09.830640 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:09.830622    2563 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Sep 07 11:47:09.830989 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:09.830961    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-multus/multus-g2v7c","openshift-multus/network-metrics-daemon-xf2gj","openshift-network-operator/iptables-alerter-j9blv","kube-system/global-pull-secret-syncer-nzzj9","kube-system/konnectivity-agent-dl69z","openshift-dns/node-resolver-hxstl","openshift-image-registry/node-ca-wrd4z","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-173.ec2.internal","openshift-network-diagnostics/network-check-target-js7hj","openshift-ovn-kubernetes/ovnkube-node-swqfh","kube-system/kube-apiserver-proxy-ip-10-0-135-173.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rmjjm","openshift-cluster-node-tuning-operator/tuned-lwkzv","openshift-multus/multus-additional-cni-plugins-mv7nw"]
Sep 07 11:47:09.833345 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:09.833332    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-g2v7c"
Sep 07 11:47:09.835488 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:09.835295    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Sep 07 11:47:09.835488 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:09.835307    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Sep 07 11:47:09.835642 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:09.835613    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-trqvc\""
Sep 07 11:47:09.835779 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:09.835762    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Sep 07 11:47:09.836058 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:09.836041    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xf2gj"
Sep 07 11:47:09.836145 ip-10-0-135-173 kubenswrapper[2563]: E0907 11:47:09.836123    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xf2gj" podUID="3acaae8d-5930-49c6-9b9e-d286ac6a75d5"
Sep 07 11:47:09.836196 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:09.836174    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Sep 07 11:47:09.836465 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:09.836449    2563 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Sep 07 11:47:09.838059 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:09.838001    2563 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-09-06 11:42:08 +0000 UTC" deadline="2028-04-03 10:37:25.128675988 +0000 UTC"
Sep 07 11:47:09.838059 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:09.838047    2563 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13774h50m15.290633154s"
Sep 07 11:47:09.838173 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:09.838130    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-j9blv"
Sep 07 11:47:09.839761 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:09.839741    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Sep 07 11:47:09.839850 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:09.839827    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Sep 07 11:47:09.839922 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:09.839901    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Sep 07 11:47:09.840152 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:09.840133    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-xmjvk\""
Sep 07 11:47:09.840356 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:09.840328    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nzzj9"
Sep 07 11:47:09.840461 ip-10-0-135-173 kubenswrapper[2563]: E0907 11:47:09.840388    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nzzj9" podUID="e92172b6-ddda-4b65-81b3-987b307e272e"
Sep 07 11:47:09.841395 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:09.841378    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/64362f4d-7d1d-4faa-a02e-4a7dec54cffc-host-run-netns\") pod \"multus-g2v7c\" (UID: \"64362f4d-7d1d-4faa-a02e-4a7dec54cffc\") " pod="openshift-multus/multus-g2v7c"
Sep 07 11:47:09.841488 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:09.841402    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/64362f4d-7d1d-4faa-a02e-4a7dec54cffc-host-var-lib-cni-bin\") pod \"multus-g2v7c\" (UID: \"64362f4d-7d1d-4faa-a02e-4a7dec54cffc\") " pod="openshift-multus/multus-g2v7c"
Sep 07 11:47:09.841488 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:09.841435    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lddp8\" (UniqueName: \"kubernetes.io/projected/64362f4d-7d1d-4faa-a02e-4a7dec54cffc-kube-api-access-lddp8\") pod \"multus-g2v7c\" (UID: \"64362f4d-7d1d-4faa-a02e-4a7dec54cffc\") " pod="openshift-multus/multus-g2v7c"
Sep 07 11:47:09.841588 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:09.841482    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-68v2l\" (UniqueName: \"kubernetes.io/projected/3acaae8d-5930-49c6-9b9e-d286ac6a75d5-kube-api-access-68v2l\") pod \"network-metrics-daemon-xf2gj\" (UID: \"3acaae8d-5930-49c6-9b9e-d286ac6a75d5\") " pod="openshift-multus/network-metrics-daemon-xf2gj"
Sep 07 11:47:09.841588 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:09.841521    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/8b7fc53e-4815-4833-a4dc-ad05cd3834c4-host-slash\") pod \"iptables-alerter-j9blv\" (UID: \"8b7fc53e-4815-4833-a4dc-ad05cd3834c4\") " pod="openshift-network-operator/iptables-alerter-j9blv"
Sep 07 11:47:09.841588 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:09.841550    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x2d9r\" (UniqueName: \"kubernetes.io/projected/8b7fc53e-4815-4833-a4dc-ad05cd3834c4-kube-api-access-x2d9r\") pod \"iptables-alerter-j9blv\" (UID: \"8b7fc53e-4815-4833-a4dc-ad05cd3834c4\") " pod="openshift-network-operator/iptables-alerter-j9blv"
Sep 07 11:47:09.841588 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:09.841570    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/64362f4d-7d1d-4faa-a02e-4a7dec54cffc-multus-socket-dir-parent\") pod \"multus-g2v7c\" (UID: \"64362f4d-7d1d-4faa-a02e-4a7dec54cffc\") " pod="openshift-multus/multus-g2v7c"
Sep 07 11:47:09.841588 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:09.841585    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/64362f4d-7d1d-4faa-a02e-4a7dec54cffc-hostroot\") pod \"multus-g2v7c\" (UID: \"64362f4d-7d1d-4faa-a02e-4a7dec54cffc\") " pod="openshift-multus/multus-g2v7c"
Sep 07 11:47:09.841784 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:09.841600    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/64362f4d-7d1d-4faa-a02e-4a7dec54cffc-host-run-multus-certs\") pod \"multus-g2v7c\" (UID: \"64362f4d-7d1d-4faa-a02e-4a7dec54cffc\") " pod="openshift-multus/multus-g2v7c"
Sep 07 11:47:09.841784 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:09.841613    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/64362f4d-7d1d-4faa-a02e-4a7dec54cffc-cnibin\") pod \"multus-g2v7c\" (UID: \"64362f4d-7d1d-4faa-a02e-4a7dec54cffc\") " pod="openshift-multus/multus-g2v7c"
Sep 07 11:47:09.841784 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:09.841627    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/64362f4d-7d1d-4faa-a02e-4a7dec54cffc-host-var-lib-cni-multus\") pod \"multus-g2v7c\" (UID: \"64362f4d-7d1d-4faa-a02e-4a7dec54cffc\") " pod="openshift-multus/multus-g2v7c"
Sep 07 11:47:09.841784 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:09.841640    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/64362f4d-7d1d-4faa-a02e-4a7dec54cffc-host-var-lib-kubelet\") pod \"multus-g2v7c\" (UID: \"64362f4d-7d1d-4faa-a02e-4a7dec54cffc\") " pod="openshift-multus/multus-g2v7c"
Sep 07 11:47:09.841784 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:09.841653    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/64362f4d-7d1d-4faa-a02e-4a7dec54cffc-multus-daemon-config\") pod \"multus-g2v7c\" (UID: \"64362f4d-7d1d-4faa-a02e-4a7dec54cffc\") " pod="openshift-multus/multus-g2v7c"
Sep 07 11:47:09.841784 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:09.841674    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3acaae8d-5930-49c6-9b9e-d286ac6a75d5-metrics-certs\") pod \"network-metrics-daemon-xf2gj\" (UID: \"3acaae8d-5930-49c6-9b9e-d286ac6a75d5\") " pod="openshift-multus/network-metrics-daemon-xf2gj"
Sep 07 11:47:09.841784 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:09.841687    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/8b7fc53e-4815-4833-a4dc-ad05cd3834c4-iptables-alerter-script\") pod \"iptables-alerter-j9blv\" (UID: \"8b7fc53e-4815-4833-a4dc-ad05cd3834c4\") " pod="openshift-network-operator/iptables-alerter-j9blv"
Sep 07 11:47:09.841784 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:09.841701    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/64362f4d-7d1d-4faa-a02e-4a7dec54cffc-multus-cni-dir\") pod \"multus-g2v7c\" (UID: \"64362f4d-7d1d-4faa-a02e-4a7dec54cffc\") " pod="openshift-multus/multus-g2v7c"
Sep 07 11:47:09.841784 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:09.841738    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/64362f4d-7d1d-4faa-a02e-4a7dec54cffc-cni-binary-copy\") pod \"multus-g2v7c\" (UID: \"64362f4d-7d1d-4faa-a02e-4a7dec54cffc\") " pod="openshift-multus/multus-g2v7c"
Sep 07 11:47:09.842076 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:09.841788    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/64362f4d-7d1d-4faa-a02e-4a7dec54cffc-host-run-k8s-cni-cncf-io\") pod \"multus-g2v7c\" (UID: \"64362f4d-7d1d-4faa-a02e-4a7dec54cffc\") " pod="openshift-multus/multus-g2v7c"
Sep 07 11:47:09.842076 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:09.841806    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/64362f4d-7d1d-4faa-a02e-4a7dec54cffc-multus-conf-dir\") pod \"multus-g2v7c\" (UID: \"64362f4d-7d1d-4faa-a02e-4a7dec54cffc\") " pod="openshift-multus/multus-g2v7c"
Sep 07 11:47:09.842076 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:09.841820    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/64362f4d-7d1d-4faa-a02e-4a7dec54cffc-etc-kubernetes\") pod \"multus-g2v7c\" (UID: \"64362f4d-7d1d-4faa-a02e-4a7dec54cffc\") " pod="openshift-multus/multus-g2v7c"
Sep 07 11:47:09.842076 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:09.841859    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/64362f4d-7d1d-4faa-a02e-4a7dec54cffc-system-cni-dir\") pod \"multus-g2v7c\" (UID: \"64362f4d-7d1d-4faa-a02e-4a7dec54cffc\") " pod="openshift-multus/multus-g2v7c"
Sep 07 11:47:09.842443 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:09.842429    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-dl69z"
Sep 07 11:47:09.843771 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:09.843753    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Sep 07 11:47:09.843906 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:09.843893    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Sep 07 11:47:09.844144 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:09.844125    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-bp2z6\""
Sep 07 11:47:09.844231 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:09.844155    2563 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 07 11:47:09.844643 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:09.844611    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-hxstl"
Sep 07 11:47:09.845967 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:09.845952    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Sep 07 11:47:09.846050 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:09.845989    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-lc7bd\""
Sep 07 11:47:09.846050 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:09.845996    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Sep 07 11:47:09.846726 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:09.846711    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-wrd4z"
Sep 07 11:47:09.847972 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:09.847945    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-sq6dx\""
Sep 07 11:47:09.848069 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:09.848010    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Sep 07 11:47:09.848069 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:09.848055    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Sep 07 11:47:09.848146 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:09.848102    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Sep 07 11:47:09.848821 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:09.848808    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-js7hj"
Sep 07 11:47:09.848879 ip-10-0-135-173 kubenswrapper[2563]: E0907 11:47:09.848864    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-js7hj" podUID="8d63cc6a-3920-45b6-9626-45b02324e6ce"
Sep 07 11:47:09.851038 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:09.851022    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-swqfh"
Sep 07 11:47:09.852880 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:09.852851    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Sep 07 11:47:09.853013 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:09.852934    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Sep 07 11:47:09.853013 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:09.852995    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Sep 07 11:47:09.853182 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:09.853089    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Sep 07 11:47:09.853228 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:09.853191    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Sep 07 11:47:09.853276 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:09.853263    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-xc82c\""
Sep 07 11:47:09.853355 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:09.853343    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Sep 07 11:47:09.853455 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:09.853442    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rmjjm"
Sep 07 11:47:09.854930 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:09.854914    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Sep 07 11:47:09.854989 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:09.854922    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Sep 07 11:47:09.855136 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:09.855126    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-lwnfq\""
Sep 07 11:47:09.855254 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:09.855240    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Sep 07 11:47:09.855691 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:09.855680    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-lwkzv"
Sep 07 11:47:09.856942 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:09.856927    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Sep 07 11:47:09.857106 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:09.857091    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-6dphq\""
Sep 07 11:47:09.857153 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:09.857130    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Sep 07 11:47:09.857895 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:09.857882    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-mv7nw"
Sep 07 11:47:09.859248 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:09.859232    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Sep 07 11:47:09.859248 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:09.859256    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-hxmwm\""
Sep 07 11:47:09.859496 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:09.859474    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Sep 07 11:47:09.878958 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:09.878088    2563 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-vmhsr"
Sep 07 11:47:09.880769 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:09.880739    2563 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 07 11:47:09.883224 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:09.883206    2563 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-vmhsr"
Sep 07 11:47:09.938782 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:09.938767    2563 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Sep 07 11:47:09.942154 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:09.942133    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/71bc595d-9844-4086-90fd-28341f888e30-registration-dir\") pod \"aws-ebs-csi-driver-node-rmjjm\" (UID: \"71bc595d-9844-4086-90fd-28341f888e30\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rmjjm"
Sep 07 11:47:09.942154 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:09.942156    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/5f723277-6450-4fd8-8f5d-01d9f9f6814b-hosts-file\") pod \"node-resolver-hxstl\" (UID: \"5f723277-6450-4fd8-8f5d-01d9f9f6814b\") " pod="openshift-dns/node-resolver-hxstl"
Sep 07 11:47:09.942367 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:09.942176    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/64362f4d-7d1d-4faa-a02e-4a7dec54cffc-cnibin\") pod \"multus-g2v7c\" (UID: \"64362f4d-7d1d-4faa-a02e-4a7dec54cffc\") " pod="openshift-multus/multus-g2v7c"
Sep 07 11:47:09.942367 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:09.942196    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/64362f4d-7d1d-4faa-a02e-4a7dec54cffc-multus-daemon-config\") pod \"multus-g2v7c\" (UID: \"64362f4d-7d1d-4faa-a02e-4a7dec54cffc\") " pod="openshift-multus/multus-g2v7c"
Sep 07 11:47:09.942367 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:09.942218    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6knwx\" (UniqueName: \"kubernetes.io/projected/b0f92581-a3a7-444e-b915-dc6cc634fe26-kube-api-access-6knwx\") pod \"ovnkube-node-swqfh\" (UID: \"b0f92581-a3a7-444e-b915-dc6cc634fe26\") " pod="openshift-ovn-kubernetes/ovnkube-node-swqfh"
Sep 07 11:47:09.942367 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:09.942273    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/64362f4d-7d1d-4faa-a02e-4a7dec54cffc-cnibin\") pod \"multus-g2v7c\" (UID: \"64362f4d-7d1d-4faa-a02e-4a7dec54cffc\") " pod="openshift-multus/multus-g2v7c"
Sep 07 11:47:09.942367 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:09.942329    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wht8w\" (UniqueName: \"kubernetes.io/projected/7323d14a-1095-4021-982f-9162ee160fe5-kube-api-access-wht8w\") pod \"tuned-lwkzv\" (UID: \"7323d14a-1095-4021-982f-9162ee160fe5\") " pod="openshift-cluster-node-tuning-operator/tuned-lwkzv"
Sep 07 11:47:09.942646 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:09.942368    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/64362f4d-7d1d-4faa-a02e-4a7dec54cffc-multus-cni-dir\") pod \"multus-g2v7c\" (UID: \"64362f4d-7d1d-4faa-a02e-4a7dec54cffc\") " pod="openshift-multus/multus-g2v7c"
Sep 07 11:47:09.942646 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:09.942395    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/64362f4d-7d1d-4faa-a02e-4a7dec54cffc-cni-binary-copy\") pod \"multus-g2v7c\" (UID: \"64362f4d-7d1d-4faa-a02e-4a7dec54cffc\") " pod="openshift-multus/multus-g2v7c"
Sep 07 11:47:09.942646 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:09.942436    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/64362f4d-7d1d-4faa-a02e-4a7dec54cffc-etc-kubernetes\") pod \"multus-g2v7c\" (UID: \"64362f4d-7d1d-4faa-a02e-4a7dec54cffc\") " pod="openshift-multus/multus-g2v7c"
Sep 07 11:47:09.942646 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:09.942465    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nht46\" (UniqueName: \"kubernetes.io/projected/55db3d09-7464-4f97-af5a-985162756ebb-kube-api-access-nht46\") pod \"multus-additional-cni-plugins-mv7nw\" (UID: \"55db3d09-7464-4f97-af5a-985162756ebb\") " pod="openshift-multus/multus-additional-cni-plugins-mv7nw"
Sep 07 11:47:09.942646 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:09.942479    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/64362f4d-7d1d-4faa-a02e-4a7dec54cffc-multus-cni-dir\") pod \"multus-g2v7c\" (UID: \"64362f4d-7d1d-4faa-a02e-4a7dec54cffc\") " pod="openshift-multus/multus-g2v7c"
Sep 07 11:47:09.942646 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:09.942493    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/b0f92581-a3a7-444e-b915-dc6cc634fe26-host-slash\") pod \"ovnkube-node-swqfh\" (UID: \"b0f92581-a3a7-444e-b915-dc6cc634fe26\") " pod="openshift-ovn-kubernetes/ovnkube-node-swqfh"
Sep 07 11:47:09.942646 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:09.942519    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b0f92581-a3a7-444e-b915-dc6cc634fe26-var-lib-openvswitch\") pod \"ovnkube-node-swqfh\" (UID: \"b0f92581-a3a7-444e-b915-dc6cc634fe26\") " pod="openshift-ovn-kubernetes/ovnkube-node-swqfh"
Sep 07 11:47:09.942646 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:09.942530    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/64362f4d-7d1d-4faa-a02e-4a7dec54cffc-etc-kubernetes\") pod \"multus-g2v7c\" (UID: \"64362f4d-7d1d-4faa-a02e-4a7dec54cffc\") " pod="openshift-multus/multus-g2v7c"
Sep 07 11:47:09.942646 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:09.942542    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/b0f92581-a3a7-444e-b915-dc6cc634fe26-host-cni-bin\") pod \"ovnkube-node-swqfh\" (UID: \"b0f92581-a3a7-444e-b915-dc6cc634fe26\") " pod="openshift-ovn-kubernetes/ovnkube-node-swqfh"
Sep 07 11:47:09.942646 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:09.942565    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/b0f92581-a3a7-444e-b915-dc6cc634fe26-env-overrides\") pod \"ovnkube-node-swqfh\" (UID: \"b0f92581-a3a7-444e-b915-dc6cc634fe26\") " pod="openshift-ovn-kubernetes/ovnkube-node-swqfh"
Sep 07 11:47:09.942646 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:09.942588    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/64362f4d-7d1d-4faa-a02e-4a7dec54cffc-system-cni-dir\") pod \"multus-g2v7c\" (UID: \"64362f4d-7d1d-4faa-a02e-4a7dec54cffc\") " pod="openshift-multus/multus-g2v7c"
Sep 07 11:47:09.942646 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:09.942622    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/55db3d09-7464-4f97-af5a-985162756ebb-cni-binary-copy\") pod \"multus-additional-cni-plugins-mv7nw\" (UID: \"55db3d09-7464-4f97-af5a-985162756ebb\") " pod="openshift-multus/multus-additional-cni-plugins-mv7nw"
Sep 07 11:47:09.942646 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:09.942645    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/71bc595d-9844-4086-90fd-28341f888e30-device-dir\") pod \"aws-ebs-csi-driver-node-rmjjm\" (UID: \"71bc595d-9844-4086-90fd-28341f888e30\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rmjjm"
Sep 07 11:47:09.943168 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:09.942686    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xtl7b\" (UniqueName: \"kubernetes.io/projected/b9aae943-4d29-4c1e-909f-0a1c034299f8-kube-api-access-xtl7b\") pod \"node-ca-wrd4z\" (UID: \"b9aae943-4d29-4c1e-909f-0a1c034299f8\") " pod="openshift-image-registry/node-ca-wrd4z"
Sep 07 11:47:09.943168 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:09.942706    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/7323d14a-1095-4021-982f-9162ee160fe5-host\") pod \"tuned-lwkzv\" (UID: \"7323d14a-1095-4021-982f-9162ee160fe5\") " pod="openshift-cluster-node-tuning-operator/tuned-lwkzv"
Sep 07 11:47:09.943168 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:09.942728    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/7323d14a-1095-4021-982f-9162ee160fe5-etc-tuned\") pod \"tuned-lwkzv\" (UID: \"7323d14a-1095-4021-982f-9162ee160fe5\") " pod="openshift-cluster-node-tuning-operator/tuned-lwkzv"
Sep 07 11:47:09.943168 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:09.942734    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/64362f4d-7d1d-4faa-a02e-4a7dec54cffc-system-cni-dir\") pod \"multus-g2v7c\" (UID: \"64362f4d-7d1d-4faa-a02e-4a7dec54cffc\") " pod="openshift-multus/multus-g2v7c"
Sep 07 11:47:09.943168 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:09.942751    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/64362f4d-7d1d-4faa-a02e-4a7dec54cffc-multus-socket-dir-parent\") pod \"multus-g2v7c\" (UID: \"64362f4d-7d1d-4faa-a02e-4a7dec54cffc\") " pod="openshift-multus/multus-g2v7c"
Sep 07 11:47:09.943168 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:09.942766    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/64362f4d-7d1d-4faa-a02e-4a7dec54cffc-multus-daemon-config\") pod \"multus-g2v7c\" (UID: \"64362f4d-7d1d-4faa-a02e-4a7dec54cffc\") " pod="openshift-multus/multus-g2v7c"
Sep 07 11:47:09.943168 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:09.942777    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b0f92581-a3a7-444e-b915-dc6cc634fe26-run-openvswitch\") pod \"ovnkube-node-swqfh\" (UID: \"b0f92581-a3a7-444e-b915-dc6cc634fe26\") " pod="openshift-ovn-kubernetes/ovnkube-node-swqfh"
Sep 07 11:47:09.943168 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:09.942799    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/64362f4d-7d1d-4faa-a02e-4a7dec54cffc-multus-socket-dir-parent\") pod \"multus-g2v7c\" (UID: \"64362f4d-7d1d-4faa-a02e-4a7dec54cffc\") " pod="openshift-multus/multus-g2v7c"
Sep 07 11:47:09.943168 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:09.942801    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/b0f92581-a3a7-444e-b915-dc6cc634fe26-ovnkube-config\") pod \"ovnkube-node-swqfh\" (UID: \"b0f92581-a3a7-444e-b915-dc6cc634fe26\") " pod="openshift-ovn-kubernetes/ovnkube-node-swqfh"
Sep 07 11:47:09.943168 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:09.942837    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/71bc595d-9844-4086-90fd-28341f888e30-etc-selinux\") pod \"aws-ebs-csi-driver-node-rmjjm\" (UID: \"71bc595d-9844-4086-90fd-28341f888e30\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rmjjm"
Sep 07 11:47:09.943168 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:09.942858    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7323d14a-1095-4021-982f-9162ee160fe5-var-lib-kubelet\") pod \"tuned-lwkzv\" (UID: \"7323d14a-1095-4021-982f-9162ee160fe5\") " pod="openshift-cluster-node-tuning-operator/tuned-lwkzv"
Sep 07 11:47:09.943168 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:09.942873    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/64362f4d-7d1d-4faa-a02e-4a7dec54cffc-host-var-lib-kubelet\") pod \"multus-g2v7c\" (UID: \"64362f4d-7d1d-4faa-a02e-4a7dec54cffc\") " pod="openshift-multus/multus-g2v7c"
Sep 07 11:47:09.943168 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:09.942888    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b0f92581-a3a7-444e-b915-dc6cc634fe26-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-swqfh\" (UID: \"b0f92581-a3a7-444e-b915-dc6cc634fe26\") " pod="openshift-ovn-kubernetes/ovnkube-node-swqfh"
Sep 07 11:47:09.943168 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:09.942908    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/71bc595d-9844-4086-90fd-28341f888e30-kubelet-dir\") pod \"aws-ebs-csi-driver-node-rmjjm\" (UID: \"71bc595d-9844-4086-90fd-28341f888e30\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rmjjm"
Sep 07 11:47:09.943168 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:09.942913    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/64362f4d-7d1d-4faa-a02e-4a7dec54cffc-cni-binary-copy\") pod \"multus-g2v7c\" (UID: \"64362f4d-7d1d-4faa-a02e-4a7dec54cffc\") " pod="openshift-multus/multus-g2v7c"
Sep 07 11:47:09.943168 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:09.942942    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/64362f4d-7d1d-4faa-a02e-4a7dec54cffc-host-var-lib-kubelet\") pod \"multus-g2v7c\" (UID: \"64362f4d-7d1d-4faa-a02e-4a7dec54cffc\") " pod="openshift-multus/multus-g2v7c"
Sep 07 11:47:09.943168 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:09.942973    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b9aae943-4d29-4c1e-909f-0a1c034299f8-host\") pod \"node-ca-wrd4z\" (UID: \"b9aae943-4d29-4c1e-909f-0a1c034299f8\") " pod="openshift-image-registry/node-ca-wrd4z"
Sep 07 11:47:09.943869 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:09.942992    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/8b7fc53e-4815-4833-a4dc-ad05cd3834c4-iptables-alerter-script\") pod \"iptables-alerter-j9blv\" (UID: \"8b7fc53e-4815-4833-a4dc-ad05cd3834c4\") " pod="openshift-network-operator/iptables-alerter-j9blv"
Sep 07 11:47:09.943869 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:09.943033    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b0f92581-a3a7-444e-b915-dc6cc634fe26-etc-openvswitch\") pod \"ovnkube-node-swqfh\" (UID: \"b0f92581-a3a7-444e-b915-dc6cc634fe26\") " pod="openshift-ovn-kubernetes/ovnkube-node-swqfh"
Sep 07 11:47:09.943869 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:09.943073    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/7323d14a-1095-4021-982f-9162ee160fe5-etc-systemd\") pod \"tuned-lwkzv\" (UID: \"7323d14a-1095-4021-982f-9162ee160fe5\") " pod="openshift-cluster-node-tuning-operator/tuned-lwkzv"
Sep 07 11:47:09.943869 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:09.943094    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/7323d14a-1095-4021-982f-9162ee160fe5-lib-modules\") pod \"tuned-lwkzv\" (UID: \"7323d14a-1095-4021-982f-9162ee160fe5\") " pod="openshift-cluster-node-tuning-operator/tuned-lwkzv"
Sep 07 11:47:09.943869 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:09.943111    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/64362f4d-7d1d-4faa-a02e-4a7dec54cffc-host-run-netns\") pod \"multus-g2v7c\" (UID: \"64362f4d-7d1d-4faa-a02e-4a7dec54cffc\") " pod="openshift-multus/multus-g2v7c"
Sep 07 11:47:09.943869 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:09.943129    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/55db3d09-7464-4f97-af5a-985162756ebb-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-mv7nw\" (UID: \"55db3d09-7464-4f97-af5a-985162756ebb\") " pod="openshift-multus/multus-additional-cni-plugins-mv7nw"
Sep 07 11:47:09.943869 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:09.943160    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/e92172b6-ddda-4b65-81b3-987b307e272e-kubelet-config\") pod \"global-pull-secret-syncer-nzzj9\" (UID: \"e92172b6-ddda-4b65-81b3-987b307e272e\") " pod="kube-system/global-pull-secret-syncer-nzzj9"
Sep 07 11:47:09.943869 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:09.943169    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/64362f4d-7d1d-4faa-a02e-4a7dec54cffc-host-run-netns\") pod \"multus-g2v7c\" (UID: \"64362f4d-7d1d-4faa-a02e-4a7dec54cffc\") " pod="openshift-multus/multus-g2v7c"
Sep 07 11:47:09.943869 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:09.943174    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/b0f92581-a3a7-444e-b915-dc6cc634fe26-run-systemd\") pod \"ovnkube-node-swqfh\" (UID: \"b0f92581-a3a7-444e-b915-dc6cc634fe26\") " pod="openshift-ovn-kubernetes/ovnkube-node-swqfh"
Sep 07 11:47:09.943869 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:09.943211    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/b0f92581-a3a7-444e-b915-dc6cc634fe26-ovn-node-metrics-cert\") pod \"ovnkube-node-swqfh\" (UID: \"b0f92581-a3a7-444e-b915-dc6cc634fe26\") " pod="openshift-ovn-kubernetes/ovnkube-node-swqfh"
Sep 07 11:47:09.943869 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:09.943230    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/3e71d2e5-bc97-415d-8e01-cdf554957349-agent-certs\") pod \"konnectivity-agent-dl69z\" (UID: \"3e71d2e5-bc97-415d-8e01-cdf554957349\") " pod="kube-system/konnectivity-agent-dl69z"
Sep 07 11:47:09.943869 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:09.943243    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/7323d14a-1095-4021-982f-9162ee160fe5-etc-sysctl-d\") pod \"tuned-lwkzv\" (UID: \"7323d14a-1095-4021-982f-9162ee160fe5\") " pod="openshift-cluster-node-tuning-operator/tuned-lwkzv"
Sep 07 11:47:09.943869 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:09.943258    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/8b7fc53e-4815-4833-a4dc-ad05cd3834c4-host-slash\") pod \"iptables-alerter-j9blv\" (UID: \"8b7fc53e-4815-4833-a4dc-ad05cd3834c4\") " pod="openshift-network-operator/iptables-alerter-j9blv"
Sep 07 11:47:09.943869 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:09.943300    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/64362f4d-7d1d-4faa-a02e-4a7dec54cffc-host-run-multus-certs\") pod \"multus-g2v7c\" (UID: \"64362f4d-7d1d-4faa-a02e-4a7dec54cffc\") " pod="openshift-multus/multus-g2v7c"
Sep 07 11:47:09.943869 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:09.943327    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/8b7fc53e-4815-4833-a4dc-ad05cd3834c4-host-slash\") pod \"iptables-alerter-j9blv\" (UID: \"8b7fc53e-4815-4833-a4dc-ad05cd3834c4\") " pod="openshift-network-operator/iptables-alerter-j9blv"
Sep 07 11:47:09.943869 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:09.943345    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/8b7fc53e-4815-4833-a4dc-ad05cd3834c4-iptables-alerter-script\") pod \"iptables-alerter-j9blv\" (UID: \"8b7fc53e-4815-4833-a4dc-ad05cd3834c4\") " pod="openshift-network-operator/iptables-alerter-j9blv"
Sep 07 11:47:09.943869 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:09.943371    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/b9aae943-4d29-4c1e-909f-0a1c034299f8-serviceca\") pod \"node-ca-wrd4z\" (UID: \"b9aae943-4d29-4c1e-909f-0a1c034299f8\") " pod="openshift-image-registry/node-ca-wrd4z"
Sep 07 11:47:09.944547 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:09.943384    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/64362f4d-7d1d-4faa-a02e-4a7dec54cffc-host-run-multus-certs\") pod \"multus-g2v7c\" (UID: \"64362f4d-7d1d-4faa-a02e-4a7dec54cffc\") " pod="openshift-multus/multus-g2v7c"
Sep 07 11:47:09.944547 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:09.943428    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/7323d14a-1095-4021-982f-9162ee160fe5-etc-sysconfig\") pod \"tuned-lwkzv\" (UID: \"7323d14a-1095-4021-982f-9162ee160fe5\") " pod="openshift-cluster-node-tuning-operator/tuned-lwkzv"
Sep 07 11:47:09.944547 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:09.943456    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/7323d14a-1095-4021-982f-9162ee160fe5-etc-sysctl-conf\") pod \"tuned-lwkzv\" (UID: \"7323d14a-1095-4021-982f-9162ee160fe5\") " pod="openshift-cluster-node-tuning-operator/tuned-lwkzv"
Sep 07 11:47:09.944547 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:09.943473    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/55db3d09-7464-4f97-af5a-985162756ebb-cnibin\") pod \"multus-additional-cni-plugins-mv7nw\" (UID: \"55db3d09-7464-4f97-af5a-985162756ebb\") " pod="openshift-multus/multus-additional-cni-plugins-mv7nw"
Sep 07 11:47:09.944547 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:09.943487    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/b0f92581-a3a7-444e-b915-dc6cc634fe26-run-ovn\") pod \"ovnkube-node-swqfh\" (UID: \"b0f92581-a3a7-444e-b915-dc6cc634fe26\") " pod="openshift-ovn-kubernetes/ovnkube-node-swqfh"
Sep 07 11:47:09.944547 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:09.943500    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5f723277-6450-4fd8-8f5d-01d9f9f6814b-tmp-dir\") pod \"node-resolver-hxstl\" (UID: \"5f723277-6450-4fd8-8f5d-01d9f9f6814b\") " pod="openshift-dns/node-resolver-hxstl"
Sep 07 11:47:09.944547 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:09.943532    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/64362f4d-7d1d-4faa-a02e-4a7dec54cffc-multus-conf-dir\") pod \"multus-g2v7c\" (UID: \"64362f4d-7d1d-4faa-a02e-4a7dec54cffc\") " pod="openshift-multus/multus-g2v7c"
Sep 07 11:47:09.944547 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:09.943552    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/b0f92581-a3a7-444e-b915-dc6cc634fe26-ovnkube-script-lib\") pod \"ovnkube-node-swqfh\" (UID: \"b0f92581-a3a7-444e-b915-dc6cc634fe26\") " pod="openshift-ovn-kubernetes/ovnkube-node-swqfh"
Sep 07 11:47:09.944547 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:09.943567    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sgbkf\" (UniqueName: \"kubernetes.io/projected/71bc595d-9844-4086-90fd-28341f888e30-kube-api-access-sgbkf\") pod \"aws-ebs-csi-driver-node-rmjjm\" (UID: \"71bc595d-9844-4086-90fd-28341f888e30\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rmjjm"
Sep 07 11:47:09.944547 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:09.943580    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/3e71d2e5-bc97-415d-8e01-cdf554957349-konnectivity-ca\") pod \"konnectivity-agent-dl69z\" (UID: \"3e71d2e5-bc97-415d-8e01-cdf554957349\") " pod="kube-system/konnectivity-agent-dl69z"
Sep 07 11:47:09.944547 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:09.943602    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f9tp8\" (UniqueName: \"kubernetes.io/projected/8d63cc6a-3920-45b6-9626-45b02324e6ce-kube-api-access-f9tp8\") pod \"network-check-target-js7hj\" (UID: \"8d63cc6a-3920-45b6-9626-45b02324e6ce\") " pod="openshift-network-diagnostics/network-check-target-js7hj"
Sep 07 11:47:09.944547 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:09.943625    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/64362f4d-7d1d-4faa-a02e-4a7dec54cffc-multus-conf-dir\") pod \"multus-g2v7c\" (UID: \"64362f4d-7d1d-4faa-a02e-4a7dec54cffc\") " pod="openshift-multus/multus-g2v7c"
Sep 07 11:47:09.944547 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:09.943649    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7323d14a-1095-4021-982f-9162ee160fe5-sys\") pod \"tuned-lwkzv\" (UID: \"7323d14a-1095-4021-982f-9162ee160fe5\") " pod="openshift-cluster-node-tuning-operator/tuned-lwkzv"
Sep 07 11:47:09.944547 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:09.943671    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/64362f4d-7d1d-4faa-a02e-4a7dec54cffc-host-var-lib-cni-bin\") pod \"multus-g2v7c\" (UID: \"64362f4d-7d1d-4faa-a02e-4a7dec54cffc\") " pod="openshift-multus/multus-g2v7c"
Sep 07 11:47:09.944547 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:09.943697    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/64362f4d-7d1d-4faa-a02e-4a7dec54cffc-host-var-lib-cni-bin\") pod \"multus-g2v7c\" (UID: \"64362f4d-7d1d-4faa-a02e-4a7dec54cffc\") " pod="openshift-multus/multus-g2v7c"
Sep 07 11:47:09.944547 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:09.943712    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l46jr\" (UniqueName: \"kubernetes.io/projected/5f723277-6450-4fd8-8f5d-01d9f9f6814b-kube-api-access-l46jr\") pod \"node-resolver-hxstl\" (UID: \"5f723277-6450-4fd8-8f5d-01d9f9f6814b\") " pod="openshift-dns/node-resolver-hxstl"
Sep 07 11:47:09.944547 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:09.943727    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7323d14a-1095-4021-982f-9162ee160fe5-etc-kubernetes\") pod \"tuned-lwkzv\" (UID: \"7323d14a-1095-4021-982f-9162ee160fe5\") " pod="openshift-cluster-node-tuning-operator/tuned-lwkzv"
Sep 07 11:47:09.945038 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:09.943741    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/64362f4d-7d1d-4faa-a02e-4a7dec54cffc-hostroot\") pod \"multus-g2v7c\" (UID: \"64362f4d-7d1d-4faa-a02e-4a7dec54cffc\") " pod="openshift-multus/multus-g2v7c"
Sep 07 11:47:09.945038 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:09.943753    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/b0f92581-a3a7-444e-b915-dc6cc634fe26-host-cni-netd\") pod \"ovnkube-node-swqfh\" (UID: \"b0f92581-a3a7-444e-b915-dc6cc634fe26\") " pod="openshift-ovn-kubernetes/ovnkube-node-swqfh"
Sep 07 11:47:09.945038 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:09.943785    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/71bc595d-9844-4086-90fd-28341f888e30-socket-dir\") pod \"aws-ebs-csi-driver-node-rmjjm\" (UID: \"71bc595d-9844-4086-90fd-28341f888e30\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rmjjm"
Sep 07 11:47:09.945038 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:09.943804    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/64362f4d-7d1d-4faa-a02e-4a7dec54cffc-hostroot\") pod \"multus-g2v7c\" (UID: \"64362f4d-7d1d-4faa-a02e-4a7dec54cffc\") " pod="openshift-multus/multus-g2v7c"
Sep 07 11:47:09.945038 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:09.943814    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/71bc595d-9844-4086-90fd-28341f888e30-sys-fs\") pod \"aws-ebs-csi-driver-node-rmjjm\" (UID: \"71bc595d-9844-4086-90fd-28341f888e30\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rmjjm"
Sep 07 11:47:09.945038 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:09.943834    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/7323d14a-1095-4021-982f-9162ee160fe5-run\") pod \"tuned-lwkzv\" (UID: \"7323d14a-1095-4021-982f-9162ee160fe5\") " pod="openshift-cluster-node-tuning-operator/tuned-lwkzv"
Sep 07 11:47:09.945038 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:09.943849    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/64362f4d-7d1d-4faa-a02e-4a7dec54cffc-host-var-lib-cni-multus\") pod \"multus-g2v7c\" (UID: \"64362f4d-7d1d-4faa-a02e-4a7dec54cffc\") " pod="openshift-multus/multus-g2v7c"
Sep 07 11:47:09.945038 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:09.943867    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/55db3d09-7464-4f97-af5a-985162756ebb-system-cni-dir\") pod \"multus-additional-cni-plugins-mv7nw\" (UID: \"55db3d09-7464-4f97-af5a-985162756ebb\") " pod="openshift-multus/multus-additional-cni-plugins-mv7nw"
Sep 07 11:47:09.945038 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:09.943882    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/55db3d09-7464-4f97-af5a-985162756ebb-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-mv7nw\" (UID: \"55db3d09-7464-4f97-af5a-985162756ebb\") " pod="openshift-multus/multus-additional-cni-plugins-mv7nw"
Sep 07 11:47:09.945038 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:09.943897    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/b0f92581-a3a7-444e-b915-dc6cc634fe26-host-kubelet\") pod \"ovnkube-node-swqfh\" (UID: \"b0f92581-a3a7-444e-b915-dc6cc634fe26\") " pod="openshift-ovn-kubernetes/ovnkube-node-swqfh"
Sep 07 11:47:09.945038 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:09.943911    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/b0f92581-a3a7-444e-b915-dc6cc634fe26-log-socket\") pod \"ovnkube-node-swqfh\" (UID: \"b0f92581-a3a7-444e-b915-dc6cc634fe26\") " pod="openshift-ovn-kubernetes/ovnkube-node-swqfh"
Sep 07 11:47:09.945038 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:09.943924    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7323d14a-1095-4021-982f-9162ee160fe5-tmp\") pod \"tuned-lwkzv\" (UID: \"7323d14a-1095-4021-982f-9162ee160fe5\") " pod="openshift-cluster-node-tuning-operator/tuned-lwkzv"
Sep 07 11:47:09.945038 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:09.943929    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/64362f4d-7d1d-4faa-a02e-4a7dec54cffc-host-var-lib-cni-multus\") pod \"multus-g2v7c\" (UID: \"64362f4d-7d1d-4faa-a02e-4a7dec54cffc\") " pod="openshift-multus/multus-g2v7c"
Sep 07 11:47:09.945038 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:09.943940    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3acaae8d-5930-49c6-9b9e-d286ac6a75d5-metrics-certs\") pod \"network-metrics-daemon-xf2gj\" (UID: \"3acaae8d-5930-49c6-9b9e-d286ac6a75d5\") " pod="openshift-multus/network-metrics-daemon-xf2gj"
Sep 07 11:47:09.945038 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:09.943958    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/64362f4d-7d1d-4faa-a02e-4a7dec54cffc-host-run-k8s-cni-cncf-io\") pod \"multus-g2v7c\" (UID: \"64362f4d-7d1d-4faa-a02e-4a7dec54cffc\") " pod="openshift-multus/multus-g2v7c"
Sep 07 11:47:09.945038 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:09.943973    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lddp8\" (UniqueName: \"kubernetes.io/projected/64362f4d-7d1d-4faa-a02e-4a7dec54cffc-kube-api-access-lddp8\") pod \"multus-g2v7c\" (UID: \"64362f4d-7d1d-4faa-a02e-4a7dec54cffc\") " pod="openshift-multus/multus-g2v7c"
Sep 07 11:47:09.945038 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:09.943986    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/b0f92581-a3a7-444e-b915-dc6cc634fe26-systemd-units\") pod \"ovnkube-node-swqfh\" (UID: \"b0f92581-a3a7-444e-b915-dc6cc634fe26\") " pod="openshift-ovn-kubernetes/ovnkube-node-swqfh"
Sep 07 11:47:09.945519 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:09.944009    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/64362f4d-7d1d-4faa-a02e-4a7dec54cffc-host-run-k8s-cni-cncf-io\") pod \"multus-g2v7c\" (UID: \"64362f4d-7d1d-4faa-a02e-4a7dec54cffc\") " pod="openshift-multus/multus-g2v7c"
Sep 07 11:47:09.945519 ip-10-0-135-173 kubenswrapper[2563]: E0907 11:47:09.944035    2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 07 11:47:09.945519 ip-10-0-135-173 kubenswrapper[2563]: E0907 11:47:09.944079    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3acaae8d-5930-49c6-9b9e-d286ac6a75d5-metrics-certs podName:3acaae8d-5930-49c6-9b9e-d286ac6a75d5 nodeName:}" failed. No retries permitted until 2026-09-07 11:47:10.444064972 +0000 UTC m=+1.962796762 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3acaae8d-5930-49c6-9b9e-d286ac6a75d5-metrics-certs") pod "network-metrics-daemon-xf2gj" (UID: "3acaae8d-5930-49c6-9b9e-d286ac6a75d5") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 07 11:47:09.945519 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:09.944098    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/b0f92581-a3a7-444e-b915-dc6cc634fe26-host-run-netns\") pod \"ovnkube-node-swqfh\" (UID: \"b0f92581-a3a7-444e-b915-dc6cc634fe26\") " pod="openshift-ovn-kubernetes/ovnkube-node-swqfh"
Sep 07 11:47:09.945519 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:09.944115    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/b0f92581-a3a7-444e-b915-dc6cc634fe26-node-log\") pod \"ovnkube-node-swqfh\" (UID: \"b0f92581-a3a7-444e-b915-dc6cc634fe26\") " pod="openshift-ovn-kubernetes/ovnkube-node-swqfh"
Sep 07 11:47:09.945519 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:09.944138    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b0f92581-a3a7-444e-b915-dc6cc634fe26-host-run-ovn-kubernetes\") pod \"ovnkube-node-swqfh\" (UID: \"b0f92581-a3a7-444e-b915-dc6cc634fe26\") " pod="openshift-ovn-kubernetes/ovnkube-node-swqfh"
Sep 07 11:47:09.945519 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:09.944158    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/7323d14a-1095-4021-982f-9162ee160fe5-etc-modprobe-d\") pod \"tuned-lwkzv\" (UID: \"7323d14a-1095-4021-982f-9162ee160fe5\") " pod="openshift-cluster-node-tuning-operator/tuned-lwkzv"
Sep 07 11:47:09.945519 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:09.944175    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-68v2l\" (UniqueName: \"kubernetes.io/projected/3acaae8d-5930-49c6-9b9e-d286ac6a75d5-kube-api-access-68v2l\") pod \"network-metrics-daemon-xf2gj\" (UID: \"3acaae8d-5930-49c6-9b9e-d286ac6a75d5\") " pod="openshift-multus/network-metrics-daemon-xf2gj"
Sep 07 11:47:09.945519 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:09.944193    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x2d9r\" (UniqueName: \"kubernetes.io/projected/8b7fc53e-4815-4833-a4dc-ad05cd3834c4-kube-api-access-x2d9r\") pod \"iptables-alerter-j9blv\" (UID: \"8b7fc53e-4815-4833-a4dc-ad05cd3834c4\") " pod="openshift-network-operator/iptables-alerter-j9blv"
Sep 07 11:47:09.945519 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:09.944209    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/55db3d09-7464-4f97-af5a-985162756ebb-tuning-conf-dir\") pod \"multus-additional-cni-plugins-mv7nw\" (UID: \"55db3d09-7464-4f97-af5a-985162756ebb\") " pod="openshift-multus/multus-additional-cni-plugins-mv7nw"
Sep 07 11:47:09.945519 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:09.944222    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/e92172b6-ddda-4b65-81b3-987b307e272e-dbus\") pod \"global-pull-secret-syncer-nzzj9\" (UID: \"e92172b6-ddda-4b65-81b3-987b307e272e\") " pod="kube-system/global-pull-secret-syncer-nzzj9"
Sep 07 11:47:09.945519 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:09.944237    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e92172b6-ddda-4b65-81b3-987b307e272e-original-pull-secret\") pod \"global-pull-secret-syncer-nzzj9\" (UID: \"e92172b6-ddda-4b65-81b3-987b307e272e\") " pod="kube-system/global-pull-secret-syncer-nzzj9"
Sep 07 11:47:09.949378 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:09.949355    2563 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Sep 07 11:47:09.952653 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:09.952631    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-68v2l\" (UniqueName: \"kubernetes.io/projected/3acaae8d-5930-49c6-9b9e-d286ac6a75d5-kube-api-access-68v2l\") pod \"network-metrics-daemon-xf2gj\" (UID: \"3acaae8d-5930-49c6-9b9e-d286ac6a75d5\") " pod="openshift-multus/network-metrics-daemon-xf2gj"
Sep 07 11:47:09.952732 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:09.952652    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lddp8\" (UniqueName: \"kubernetes.io/projected/64362f4d-7d1d-4faa-a02e-4a7dec54cffc-kube-api-access-lddp8\") pod \"multus-g2v7c\" (UID: \"64362f4d-7d1d-4faa-a02e-4a7dec54cffc\") " pod="openshift-multus/multus-g2v7c"
Sep 07 11:47:09.952732 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:09.952631    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x2d9r\" (UniqueName: \"kubernetes.io/projected/8b7fc53e-4815-4833-a4dc-ad05cd3834c4-kube-api-access-x2d9r\") pod \"iptables-alerter-j9blv\" (UID: \"8b7fc53e-4815-4833-a4dc-ad05cd3834c4\") " pod="openshift-network-operator/iptables-alerter-j9blv"
Sep 07 11:47:10.044964 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:10.044927    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5f723277-6450-4fd8-8f5d-01d9f9f6814b-tmp-dir\") pod \"node-resolver-hxstl\" (UID: \"5f723277-6450-4fd8-8f5d-01d9f9f6814b\") " pod="openshift-dns/node-resolver-hxstl"
Sep 07 11:47:10.044964 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:10.044962    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/b0f92581-a3a7-444e-b915-dc6cc634fe26-ovnkube-script-lib\") pod \"ovnkube-node-swqfh\" (UID: \"b0f92581-a3a7-444e-b915-dc6cc634fe26\") " pod="openshift-ovn-kubernetes/ovnkube-node-swqfh"
Sep 07 11:47:10.045174 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:10.044986    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sgbkf\" (UniqueName: \"kubernetes.io/projected/71bc595d-9844-4086-90fd-28341f888e30-kube-api-access-sgbkf\") pod \"aws-ebs-csi-driver-node-rmjjm\" (UID: \"71bc595d-9844-4086-90fd-28341f888e30\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rmjjm"
Sep 07 11:47:10.045174 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:10.045006    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/3e71d2e5-bc97-415d-8e01-cdf554957349-konnectivity-ca\") pod \"konnectivity-agent-dl69z\" (UID: \"3e71d2e5-bc97-415d-8e01-cdf554957349\") " pod="kube-system/konnectivity-agent-dl69z"
Sep 07 11:47:10.045174 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:10.045030    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f9tp8\" (UniqueName: \"kubernetes.io/projected/8d63cc6a-3920-45b6-9626-45b02324e6ce-kube-api-access-f9tp8\") pod \"network-check-target-js7hj\" (UID: \"8d63cc6a-3920-45b6-9626-45b02324e6ce\") " pod="openshift-network-diagnostics/network-check-target-js7hj"
Sep 07 11:47:10.045174 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:10.045050    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7323d14a-1095-4021-982f-9162ee160fe5-sys\") pod \"tuned-lwkzv\" (UID: \"7323d14a-1095-4021-982f-9162ee160fe5\") " pod="openshift-cluster-node-tuning-operator/tuned-lwkzv"
Sep 07 11:47:10.045174 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:10.045072    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l46jr\" (UniqueName: \"kubernetes.io/projected/5f723277-6450-4fd8-8f5d-01d9f9f6814b-kube-api-access-l46jr\") pod \"node-resolver-hxstl\" (UID: \"5f723277-6450-4fd8-8f5d-01d9f9f6814b\") " pod="openshift-dns/node-resolver-hxstl"
Sep 07 11:47:10.045174 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:10.045095    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7323d14a-1095-4021-982f-9162ee160fe5-etc-kubernetes\") pod \"tuned-lwkzv\" (UID: \"7323d14a-1095-4021-982f-9162ee160fe5\") " pod="openshift-cluster-node-tuning-operator/tuned-lwkzv"
Sep 07 11:47:10.045174 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:10.045116    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/b0f92581-a3a7-444e-b915-dc6cc634fe26-host-cni-netd\") pod \"ovnkube-node-swqfh\" (UID: \"b0f92581-a3a7-444e-b915-dc6cc634fe26\") " pod="openshift-ovn-kubernetes/ovnkube-node-swqfh"
Sep 07 11:47:10.045174 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:10.045138    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/71bc595d-9844-4086-90fd-28341f888e30-socket-dir\") pod \"aws-ebs-csi-driver-node-rmjjm\" (UID: \"71bc595d-9844-4086-90fd-28341f888e30\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rmjjm"
Sep 07 11:47:10.045506 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:10.045211    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7323d14a-1095-4021-982f-9162ee160fe5-sys\") pod \"tuned-lwkzv\" (UID: \"7323d14a-1095-4021-982f-9162ee160fe5\") " pod="openshift-cluster-node-tuning-operator/tuned-lwkzv"
Sep 07 11:47:10.045506 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:10.045220    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5f723277-6450-4fd8-8f5d-01d9f9f6814b-tmp-dir\") pod \"node-resolver-hxstl\" (UID: \"5f723277-6450-4fd8-8f5d-01d9f9f6814b\") " pod="openshift-dns/node-resolver-hxstl"
Sep 07 11:47:10.045506 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:10.045257    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/71bc595d-9844-4086-90fd-28341f888e30-socket-dir\") pod \"aws-ebs-csi-driver-node-rmjjm\" (UID: \"71bc595d-9844-4086-90fd-28341f888e30\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rmjjm"
Sep 07 11:47:10.045506 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:10.045277    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/b0f92581-a3a7-444e-b915-dc6cc634fe26-host-cni-netd\") pod \"ovnkube-node-swqfh\" (UID: \"b0f92581-a3a7-444e-b915-dc6cc634fe26\") " pod="openshift-ovn-kubernetes/ovnkube-node-swqfh"
Sep 07 11:47:10.045506 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:10.045291    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/71bc595d-9844-4086-90fd-28341f888e30-sys-fs\") pod \"aws-ebs-csi-driver-node-rmjjm\" (UID: \"71bc595d-9844-4086-90fd-28341f888e30\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rmjjm"
Sep 07 11:47:10.045506 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:10.045310    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7323d14a-1095-4021-982f-9162ee160fe5-etc-kubernetes\") pod \"tuned-lwkzv\" (UID: \"7323d14a-1095-4021-982f-9162ee160fe5\") " pod="openshift-cluster-node-tuning-operator/tuned-lwkzv"
Sep 07 11:47:10.045506 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:10.045322    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/7323d14a-1095-4021-982f-9162ee160fe5-run\") pod \"tuned-lwkzv\" (UID: \"7323d14a-1095-4021-982f-9162ee160fe5\") " pod="openshift-cluster-node-tuning-operator/tuned-lwkzv"
Sep 07 11:47:10.045506 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:10.045359    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/55db3d09-7464-4f97-af5a-985162756ebb-system-cni-dir\") pod \"multus-additional-cni-plugins-mv7nw\" (UID: \"55db3d09-7464-4f97-af5a-985162756ebb\") " pod="openshift-multus/multus-additional-cni-plugins-mv7nw"
Sep 07 11:47:10.045506 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:10.045371    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/71bc595d-9844-4086-90fd-28341f888e30-sys-fs\") pod \"aws-ebs-csi-driver-node-rmjjm\" (UID: \"71bc595d-9844-4086-90fd-28341f888e30\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rmjjm"
Sep 07 11:47:10.045506 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:10.045384    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/55db3d09-7464-4f97-af5a-985162756ebb-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-mv7nw\" (UID: \"55db3d09-7464-4f97-af5a-985162756ebb\") " pod="openshift-multus/multus-additional-cni-plugins-mv7nw"
Sep 07 11:47:10.045506 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:10.045409    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/55db3d09-7464-4f97-af5a-985162756ebb-system-cni-dir\") pod \"multus-additional-cni-plugins-mv7nw\" (UID: \"55db3d09-7464-4f97-af5a-985162756ebb\") " pod="openshift-multus/multus-additional-cni-plugins-mv7nw"
Sep 07 11:47:10.045506 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:10.045430    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/b0f92581-a3a7-444e-b915-dc6cc634fe26-host-kubelet\") pod \"ovnkube-node-swqfh\" (UID: \"b0f92581-a3a7-444e-b915-dc6cc634fe26\") " pod="openshift-ovn-kubernetes/ovnkube-node-swqfh"
Sep 07 11:47:10.045506 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:10.045451    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/b0f92581-a3a7-444e-b915-dc6cc634fe26-log-socket\") pod \"ovnkube-node-swqfh\" (UID: \"b0f92581-a3a7-444e-b915-dc6cc634fe26\") " pod="openshift-ovn-kubernetes/ovnkube-node-swqfh"
Sep 07 11:47:10.045506 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:10.045466    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7323d14a-1095-4021-982f-9162ee160fe5-tmp\") pod \"tuned-lwkzv\" (UID: \"7323d14a-1095-4021-982f-9162ee160fe5\") " pod="openshift-cluster-node-tuning-operator/tuned-lwkzv"
Sep 07 11:47:10.045506 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:10.045409    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/7323d14a-1095-4021-982f-9162ee160fe5-run\") pod \"tuned-lwkzv\" (UID: \"7323d14a-1095-4021-982f-9162ee160fe5\") " pod="openshift-cluster-node-tuning-operator/tuned-lwkzv"
Sep 07 11:47:10.045506 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:10.045496    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/b0f92581-a3a7-444e-b915-dc6cc634fe26-systemd-units\") pod \"ovnkube-node-swqfh\" (UID: \"b0f92581-a3a7-444e-b915-dc6cc634fe26\") " pod="openshift-ovn-kubernetes/ovnkube-node-swqfh"
Sep 07 11:47:10.046261 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:10.045520    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/b0f92581-a3a7-444e-b915-dc6cc634fe26-host-run-netns\") pod \"ovnkube-node-swqfh\" (UID: \"b0f92581-a3a7-444e-b915-dc6cc634fe26\") " pod="openshift-ovn-kubernetes/ovnkube-node-swqfh"
Sep 07 11:47:10.046261 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:10.045539    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/b0f92581-a3a7-444e-b915-dc6cc634fe26-systemd-units\") pod \"ovnkube-node-swqfh\" (UID: \"b0f92581-a3a7-444e-b915-dc6cc634fe26\") " pod="openshift-ovn-kubernetes/ovnkube-node-swqfh"
Sep 07 11:47:10.046261 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:10.045544    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/b0f92581-a3a7-444e-b915-dc6cc634fe26-node-log\") pod \"ovnkube-node-swqfh\" (UID: \"b0f92581-a3a7-444e-b915-dc6cc634fe26\") " pod="openshift-ovn-kubernetes/ovnkube-node-swqfh"
Sep 07 11:47:10.046261 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:10.045538    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/b0f92581-a3a7-444e-b915-dc6cc634fe26-host-kubelet\") pod \"ovnkube-node-swqfh\" (UID: \"b0f92581-a3a7-444e-b915-dc6cc634fe26\") " pod="openshift-ovn-kubernetes/ovnkube-node-swqfh"
Sep 07 11:47:10.046261 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:10.045557    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/b0f92581-a3a7-444e-b915-dc6cc634fe26-host-run-netns\") pod \"ovnkube-node-swqfh\" (UID: \"b0f92581-a3a7-444e-b915-dc6cc634fe26\") " pod="openshift-ovn-kubernetes/ovnkube-node-swqfh"
Sep 07 11:47:10.046261 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:10.045522    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/b0f92581-a3a7-444e-b915-dc6cc634fe26-log-socket\") pod \"ovnkube-node-swqfh\" (UID: \"b0f92581-a3a7-444e-b915-dc6cc634fe26\") " pod="openshift-ovn-kubernetes/ovnkube-node-swqfh"
Sep 07 11:47:10.046261 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:10.045567    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b0f92581-a3a7-444e-b915-dc6cc634fe26-host-run-ovn-kubernetes\") pod \"ovnkube-node-swqfh\" (UID: \"b0f92581-a3a7-444e-b915-dc6cc634fe26\") " pod="openshift-ovn-kubernetes/ovnkube-node-swqfh"
Sep 07 11:47:10.046261 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:10.045590    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/7323d14a-1095-4021-982f-9162ee160fe5-etc-modprobe-d\") pod \"tuned-lwkzv\" (UID: \"7323d14a-1095-4021-982f-9162ee160fe5\") " pod="openshift-cluster-node-tuning-operator/tuned-lwkzv"
Sep 07 11:47:10.046261 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:10.045613    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b0f92581-a3a7-444e-b915-dc6cc634fe26-host-run-ovn-kubernetes\") pod \"ovnkube-node-swqfh\" (UID: \"b0f92581-a3a7-444e-b915-dc6cc634fe26\") " pod="openshift-ovn-kubernetes/ovnkube-node-swqfh"
Sep 07 11:47:10.046261 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:10.045617    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/55db3d09-7464-4f97-af5a-985162756ebb-tuning-conf-dir\") pod \"multus-additional-cni-plugins-mv7nw\" (UID: \"55db3d09-7464-4f97-af5a-985162756ebb\") " pod="openshift-multus/multus-additional-cni-plugins-mv7nw"
Sep 07 11:47:10.046261 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:10.045651    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/e92172b6-ddda-4b65-81b3-987b307e272e-dbus\") pod \"global-pull-secret-syncer-nzzj9\" (UID: \"e92172b6-ddda-4b65-81b3-987b307e272e\") " pod="kube-system/global-pull-secret-syncer-nzzj9"
Sep 07 11:47:10.046261 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:10.045591    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/b0f92581-a3a7-444e-b915-dc6cc634fe26-node-log\") pod \"ovnkube-node-swqfh\" (UID: \"b0f92581-a3a7-444e-b915-dc6cc634fe26\") " pod="openshift-ovn-kubernetes/ovnkube-node-swqfh"
Sep 07 11:47:10.046261 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:10.045675    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e92172b6-ddda-4b65-81b3-987b307e272e-original-pull-secret\") pod \"global-pull-secret-syncer-nzzj9\" (UID: \"e92172b6-ddda-4b65-81b3-987b307e272e\") " pod="kube-system/global-pull-secret-syncer-nzzj9"
Sep 07 11:47:10.046261 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:10.045694    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/7323d14a-1095-4021-982f-9162ee160fe5-etc-modprobe-d\") pod \"tuned-lwkzv\" (UID: \"7323d14a-1095-4021-982f-9162ee160fe5\") " pod="openshift-cluster-node-tuning-operator/tuned-lwkzv"
Sep 07 11:47:10.046261 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:10.045698    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/71bc595d-9844-4086-90fd-28341f888e30-registration-dir\") pod \"aws-ebs-csi-driver-node-rmjjm\" (UID: \"71bc595d-9844-4086-90fd-28341f888e30\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rmjjm"
Sep 07 11:47:10.046261 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:10.045722    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/5f723277-6450-4fd8-8f5d-01d9f9f6814b-hosts-file\") pod \"node-resolver-hxstl\" (UID: \"5f723277-6450-4fd8-8f5d-01d9f9f6814b\") " pod="openshift-dns/node-resolver-hxstl"
Sep 07 11:47:10.046261 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:10.045735    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/e92172b6-ddda-4b65-81b3-987b307e272e-dbus\") pod \"global-pull-secret-syncer-nzzj9\" (UID: \"e92172b6-ddda-4b65-81b3-987b307e272e\") " pod="kube-system/global-pull-secret-syncer-nzzj9"
Sep 07 11:47:10.047096 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:10.045747    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6knwx\" (UniqueName: \"kubernetes.io/projected/b0f92581-a3a7-444e-b915-dc6cc634fe26-kube-api-access-6knwx\") pod \"ovnkube-node-swqfh\" (UID: \"b0f92581-a3a7-444e-b915-dc6cc634fe26\") " pod="openshift-ovn-kubernetes/ovnkube-node-swqfh"
Sep 07 11:47:10.047096 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:10.045763    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/55db3d09-7464-4f97-af5a-985162756ebb-tuning-conf-dir\") pod \"multus-additional-cni-plugins-mv7nw\" (UID: \"55db3d09-7464-4f97-af5a-985162756ebb\") " pod="openshift-multus/multus-additional-cni-plugins-mv7nw"
Sep 07 11:47:10.047096 ip-10-0-135-173 kubenswrapper[2563]: E0907 11:47:10.045768    2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 07 11:47:10.047096 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:10.045772    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wht8w\" (UniqueName: \"kubernetes.io/projected/7323d14a-1095-4021-982f-9162ee160fe5-kube-api-access-wht8w\") pod \"tuned-lwkzv\" (UID: \"7323d14a-1095-4021-982f-9162ee160fe5\") " pod="openshift-cluster-node-tuning-operator/tuned-lwkzv"
Sep 07 11:47:10.047096 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:10.045789    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/71bc595d-9844-4086-90fd-28341f888e30-registration-dir\") pod \"aws-ebs-csi-driver-node-rmjjm\" (UID: \"71bc595d-9844-4086-90fd-28341f888e30\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rmjjm"
Sep 07 11:47:10.047096 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:10.045799    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nht46\" (UniqueName: \"kubernetes.io/projected/55db3d09-7464-4f97-af5a-985162756ebb-kube-api-access-nht46\") pod \"multus-additional-cni-plugins-mv7nw\" (UID: \"55db3d09-7464-4f97-af5a-985162756ebb\") " pod="openshift-multus/multus-additional-cni-plugins-mv7nw"
Sep 07 11:47:10.047096 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:10.045826    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/b0f92581-a3a7-444e-b915-dc6cc634fe26-host-slash\") pod \"ovnkube-node-swqfh\" (UID: \"b0f92581-a3a7-444e-b915-dc6cc634fe26\") " pod="openshift-ovn-kubernetes/ovnkube-node-swqfh"
Sep 07 11:47:10.047096 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:10.045835    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/5f723277-6450-4fd8-8f5d-01d9f9f6814b-hosts-file\") pod \"node-resolver-hxstl\" (UID: \"5f723277-6450-4fd8-8f5d-01d9f9f6814b\") " pod="openshift-dns/node-resolver-hxstl"
Sep 07 11:47:10.047096 ip-10-0-135-173 kubenswrapper[2563]: E0907 11:47:10.045839    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e92172b6-ddda-4b65-81b3-987b307e272e-original-pull-secret podName:e92172b6-ddda-4b65-81b3-987b307e272e nodeName:}" failed. No retries permitted until 2026-09-07 11:47:10.545821445 +0000 UTC m=+2.064553261 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e92172b6-ddda-4b65-81b3-987b307e272e-original-pull-secret") pod "global-pull-secret-syncer-nzzj9" (UID: "e92172b6-ddda-4b65-81b3-987b307e272e") : object "kube-system"/"original-pull-secret" not registered
Sep 07 11:47:10.047096 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:10.045855    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/b0f92581-a3a7-444e-b915-dc6cc634fe26-host-slash\") pod \"ovnkube-node-swqfh\" (UID: \"b0f92581-a3a7-444e-b915-dc6cc634fe26\") " pod="openshift-ovn-kubernetes/ovnkube-node-swqfh"
Sep 07 11:47:10.047096 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:10.045890    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b0f92581-a3a7-444e-b915-dc6cc634fe26-var-lib-openvswitch\") pod \"ovnkube-node-swqfh\" (UID: \"b0f92581-a3a7-444e-b915-dc6cc634fe26\") " pod="openshift-ovn-kubernetes/ovnkube-node-swqfh"
Sep 07 11:47:10.047096 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:10.045920    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/55db3d09-7464-4f97-af5a-985162756ebb-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-mv7nw\" (UID: \"55db3d09-7464-4f97-af5a-985162756ebb\") " pod="openshift-multus/multus-additional-cni-plugins-mv7nw"
Sep 07 11:47:10.047096 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:10.045918    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/b0f92581-a3a7-444e-b915-dc6cc634fe26-host-cni-bin\") pod \"ovnkube-node-swqfh\" (UID: \"b0f92581-a3a7-444e-b915-dc6cc634fe26\") " pod="openshift-ovn-kubernetes/ovnkube-node-swqfh"
Sep 07 11:47:10.047096 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:10.045954    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/b0f92581-a3a7-444e-b915-dc6cc634fe26-host-cni-bin\") pod \"ovnkube-node-swqfh\" (UID: \"b0f92581-a3a7-444e-b915-dc6cc634fe26\") " pod="openshift-ovn-kubernetes/ovnkube-node-swqfh"
Sep 07 11:47:10.047096 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:10.045959    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b0f92581-a3a7-444e-b915-dc6cc634fe26-var-lib-openvswitch\") pod \"ovnkube-node-swqfh\" (UID: \"b0f92581-a3a7-444e-b915-dc6cc634fe26\") " pod="openshift-ovn-kubernetes/ovnkube-node-swqfh"
Sep 07 11:47:10.047096 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:10.045975    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/b0f92581-a3a7-444e-b915-dc6cc634fe26-env-overrides\") pod \"ovnkube-node-swqfh\" (UID: \"b0f92581-a3a7-444e-b915-dc6cc634fe26\") " pod="openshift-ovn-kubernetes/ovnkube-node-swqfh"
Sep 07 11:47:10.047096 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:10.046006    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/55db3d09-7464-4f97-af5a-985162756ebb-cni-binary-copy\") pod \"multus-additional-cni-plugins-mv7nw\" (UID: \"55db3d09-7464-4f97-af5a-985162756ebb\") " pod="openshift-multus/multus-additional-cni-plugins-mv7nw"
Sep 07 11:47:10.048004 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:10.046034    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/71bc595d-9844-4086-90fd-28341f888e30-device-dir\") pod \"aws-ebs-csi-driver-node-rmjjm\" (UID: \"71bc595d-9844-4086-90fd-28341f888e30\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rmjjm"
Sep 07 11:47:10.048004 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:10.046058    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xtl7b\" (UniqueName: \"kubernetes.io/projected/b9aae943-4d29-4c1e-909f-0a1c034299f8-kube-api-access-xtl7b\") pod \"node-ca-wrd4z\" (UID: \"b9aae943-4d29-4c1e-909f-0a1c034299f8\") " pod="openshift-image-registry/node-ca-wrd4z"
Sep 07 11:47:10.048004 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:10.046083    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/7323d14a-1095-4021-982f-9162ee160fe5-host\") pod \"tuned-lwkzv\" (UID: \"7323d14a-1095-4021-982f-9162ee160fe5\") " pod="openshift-cluster-node-tuning-operator/tuned-lwkzv"
Sep 07 11:47:10.048004 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:10.046087    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/b0f92581-a3a7-444e-b915-dc6cc634fe26-ovnkube-script-lib\") pod \"ovnkube-node-swqfh\" (UID: \"b0f92581-a3a7-444e-b915-dc6cc634fe26\") " pod="openshift-ovn-kubernetes/ovnkube-node-swqfh"
Sep 07 11:47:10.048004 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:10.046106    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/7323d14a-1095-4021-982f-9162ee160fe5-etc-tuned\") pod \"tuned-lwkzv\" (UID: \"7323d14a-1095-4021-982f-9162ee160fe5\") " pod="openshift-cluster-node-tuning-operator/tuned-lwkzv"
Sep 07 11:47:10.048004 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:10.046129    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b0f92581-a3a7-444e-b915-dc6cc634fe26-run-openvswitch\") pod \"ovnkube-node-swqfh\" (UID: \"b0f92581-a3a7-444e-b915-dc6cc634fe26\") " pod="openshift-ovn-kubernetes/ovnkube-node-swqfh"
Sep 07 11:47:10.048004 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:10.046156    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/b0f92581-a3a7-444e-b915-dc6cc634fe26-ovnkube-config\") pod \"ovnkube-node-swqfh\" (UID: \"b0f92581-a3a7-444e-b915-dc6cc634fe26\") " pod="openshift-ovn-kubernetes/ovnkube-node-swqfh"
Sep 07 11:47:10.048004 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:10.046161    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/71bc595d-9844-4086-90fd-28341f888e30-device-dir\") pod \"aws-ebs-csi-driver-node-rmjjm\" (UID: \"71bc595d-9844-4086-90fd-28341f888e30\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rmjjm"
Sep 07 11:47:10.048004 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:10.046163    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/7323d14a-1095-4021-982f-9162ee160fe5-host\") pod \"tuned-lwkzv\" (UID: \"7323d14a-1095-4021-982f-9162ee160fe5\") " pod="openshift-cluster-node-tuning-operator/tuned-lwkzv"
Sep 07 11:47:10.048004 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:10.046179    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/71bc595d-9844-4086-90fd-28341f888e30-etc-selinux\") pod \"aws-ebs-csi-driver-node-rmjjm\" (UID: \"71bc595d-9844-4086-90fd-28341f888e30\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rmjjm"
Sep 07 11:47:10.048004 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:10.046205    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7323d14a-1095-4021-982f-9162ee160fe5-var-lib-kubelet\") pod \"tuned-lwkzv\" (UID: \"7323d14a-1095-4021-982f-9162ee160fe5\") " pod="openshift-cluster-node-tuning-operator/tuned-lwkzv"
Sep 07 11:47:10.048004 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:10.046215    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b0f92581-a3a7-444e-b915-dc6cc634fe26-run-openvswitch\") pod \"ovnkube-node-swqfh\" (UID: \"b0f92581-a3a7-444e-b915-dc6cc634fe26\") " pod="openshift-ovn-kubernetes/ovnkube-node-swqfh"
Sep 07 11:47:10.048004 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:10.046231    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b0f92581-a3a7-444e-b915-dc6cc634fe26-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-swqfh\" (UID: \"b0f92581-a3a7-444e-b915-dc6cc634fe26\") " pod="openshift-ovn-kubernetes/ovnkube-node-swqfh"
Sep 07 11:47:10.048004 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:10.046264    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/71bc595d-9844-4086-90fd-28341f888e30-kubelet-dir\") pod \"aws-ebs-csi-driver-node-rmjjm\" (UID: \"71bc595d-9844-4086-90fd-28341f888e30\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rmjjm"
Sep 07 11:47:10.048004 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:10.046286    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b9aae943-4d29-4c1e-909f-0a1c034299f8-host\") pod \"node-ca-wrd4z\" (UID: \"b9aae943-4d29-4c1e-909f-0a1c034299f8\") " pod="openshift-image-registry/node-ca-wrd4z"
Sep 07 11:47:10.048004 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:10.046308    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b0f92581-a3a7-444e-b915-dc6cc634fe26-etc-openvswitch\") pod \"ovnkube-node-swqfh\" (UID: \"b0f92581-a3a7-444e-b915-dc6cc634fe26\") " pod="openshift-ovn-kubernetes/ovnkube-node-swqfh"
Sep 07 11:47:10.048004 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:10.046334    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/7323d14a-1095-4021-982f-9162ee160fe5-etc-systemd\") pod \"tuned-lwkzv\" (UID: \"7323d14a-1095-4021-982f-9162ee160fe5\") " pod="openshift-cluster-node-tuning-operator/tuned-lwkzv"
Sep 07 11:47:10.048835 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:10.046342    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7323d14a-1095-4021-982f-9162ee160fe5-var-lib-kubelet\") pod \"tuned-lwkzv\" (UID: \"7323d14a-1095-4021-982f-9162ee160fe5\") " pod="openshift-cluster-node-tuning-operator/tuned-lwkzv"
Sep 07 11:47:10.048835 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:10.046357    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/7323d14a-1095-4021-982f-9162ee160fe5-lib-modules\") pod \"tuned-lwkzv\" (UID: \"7323d14a-1095-4021-982f-9162ee160fe5\") " pod="openshift-cluster-node-tuning-operator/tuned-lwkzv"
Sep 07 11:47:10.048835 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:10.046406    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/71bc595d-9844-4086-90fd-28341f888e30-etc-selinux\") pod \"aws-ebs-csi-driver-node-rmjjm\" (UID: \"71bc595d-9844-4086-90fd-28341f888e30\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rmjjm"
Sep 07 11:47:10.048835 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:10.046456    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/55db3d09-7464-4f97-af5a-985162756ebb-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-mv7nw\" (UID: \"55db3d09-7464-4f97-af5a-985162756ebb\") " pod="openshift-multus/multus-additional-cni-plugins-mv7nw"
Sep 07 11:47:10.048835 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:10.046470    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/7323d14a-1095-4021-982f-9162ee160fe5-lib-modules\") pod \"tuned-lwkzv\" (UID: \"7323d14a-1095-4021-982f-9162ee160fe5\") " pod="openshift-cluster-node-tuning-operator/tuned-lwkzv"
Sep 07 11:47:10.048835 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:10.046485    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/e92172b6-ddda-4b65-81b3-987b307e272e-kubelet-config\") pod \"global-pull-secret-syncer-nzzj9\" (UID: \"e92172b6-ddda-4b65-81b3-987b307e272e\") " pod="kube-system/global-pull-secret-syncer-nzzj9"
Sep 07 11:47:10.048835 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:10.046511    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/b0f92581-a3a7-444e-b915-dc6cc634fe26-run-systemd\") pod \"ovnkube-node-swqfh\" (UID: \"b0f92581-a3a7-444e-b915-dc6cc634fe26\") " pod="openshift-ovn-kubernetes/ovnkube-node-swqfh"
Sep 07 11:47:10.048835 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:10.046521    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b0f92581-a3a7-444e-b915-dc6cc634fe26-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-swqfh\" (UID: \"b0f92581-a3a7-444e-b915-dc6cc634fe26\") " pod="openshift-ovn-kubernetes/ovnkube-node-swqfh"
Sep 07 11:47:10.048835 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:10.046521    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/55db3d09-7464-4f97-af5a-985162756ebb-cni-binary-copy\") pod \"multus-additional-cni-plugins-mv7nw\" (UID: \"55db3d09-7464-4f97-af5a-985162756ebb\") " pod="openshift-multus/multus-additional-cni-plugins-mv7nw"
Sep 07 11:47:10.048835 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:10.046536    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/b0f92581-a3a7-444e-b915-dc6cc634fe26-ovn-node-metrics-cert\") pod \"ovnkube-node-swqfh\" (UID: \"b0f92581-a3a7-444e-b915-dc6cc634fe26\") " pod="openshift-ovn-kubernetes/ovnkube-node-swqfh"
Sep 07 11:47:10.048835 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:10.046561    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/3e71d2e5-bc97-415d-8e01-cdf554957349-agent-certs\") pod \"konnectivity-agent-dl69z\" (UID: \"3e71d2e5-bc97-415d-8e01-cdf554957349\") " pod="kube-system/konnectivity-agent-dl69z"
Sep 07 11:47:10.048835 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:10.046565    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b9aae943-4d29-4c1e-909f-0a1c034299f8-host\") pod \"node-ca-wrd4z\" (UID: \"b9aae943-4d29-4c1e-909f-0a1c034299f8\") " pod="openshift-image-registry/node-ca-wrd4z"
Sep 07 11:47:10.048835 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:10.046577    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b0f92581-a3a7-444e-b915-dc6cc634fe26-etc-openvswitch\") pod \"ovnkube-node-swqfh\" (UID: \"b0f92581-a3a7-444e-b915-dc6cc634fe26\") " pod="openshift-ovn-kubernetes/ovnkube-node-swqfh"
Sep 07 11:47:10.048835 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:10.046586    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/7323d14a-1095-4021-982f-9162ee160fe5-etc-sysctl-d\") pod \"tuned-lwkzv\" (UID: \"7323d14a-1095-4021-982f-9162ee160fe5\") " pod="openshift-cluster-node-tuning-operator/tuned-lwkzv"
Sep 07 11:47:10.048835 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:10.046621    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/b9aae943-4d29-4c1e-909f-0a1c034299f8-serviceca\") pod \"node-ca-wrd4z\" (UID: \"b9aae943-4d29-4c1e-909f-0a1c034299f8\") " pod="openshift-image-registry/node-ca-wrd4z"
Sep 07 11:47:10.048835 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:10.046627    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/71bc595d-9844-4086-90fd-28341f888e30-kubelet-dir\") pod \"aws-ebs-csi-driver-node-rmjjm\" (UID: \"71bc595d-9844-4086-90fd-28341f888e30\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rmjjm"
Sep 07 11:47:10.048835 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:10.046624    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/b0f92581-a3a7-444e-b915-dc6cc634fe26-run-systemd\") pod \"ovnkube-node-swqfh\" (UID: \"b0f92581-a3a7-444e-b915-dc6cc634fe26\") " pod="openshift-ovn-kubernetes/ovnkube-node-swqfh"
Sep 07 11:47:10.049378 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:10.046661    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/e92172b6-ddda-4b65-81b3-987b307e272e-kubelet-config\") pod \"global-pull-secret-syncer-nzzj9\" (UID: \"e92172b6-ddda-4b65-81b3-987b307e272e\") " pod="kube-system/global-pull-secret-syncer-nzzj9"
Sep 07 11:47:10.049378 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:10.046668    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/7323d14a-1095-4021-982f-9162ee160fe5-etc-systemd\") pod \"tuned-lwkzv\" (UID: \"7323d14a-1095-4021-982f-9162ee160fe5\") " pod="openshift-cluster-node-tuning-operator/tuned-lwkzv"
Sep 07 11:47:10.049378 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:10.046689    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/b0f92581-a3a7-444e-b915-dc6cc634fe26-ovnkube-config\") pod \"ovnkube-node-swqfh\" (UID: \"b0f92581-a3a7-444e-b915-dc6cc634fe26\") " pod="openshift-ovn-kubernetes/ovnkube-node-swqfh"
Sep 07 11:47:10.049378 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:10.046696    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/7323d14a-1095-4021-982f-9162ee160fe5-etc-sysconfig\") pod \"tuned-lwkzv\" (UID: \"7323d14a-1095-4021-982f-9162ee160fe5\") " pod="openshift-cluster-node-tuning-operator/tuned-lwkzv"
Sep 07 11:47:10.049378 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:10.046722    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/7323d14a-1095-4021-982f-9162ee160fe5-etc-sysctl-conf\") pod \"tuned-lwkzv\" (UID: \"7323d14a-1095-4021-982f-9162ee160fe5\") " pod="openshift-cluster-node-tuning-operator/tuned-lwkzv"
Sep 07 11:47:10.049378 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:10.046735    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/7323d14a-1095-4021-982f-9162ee160fe5-etc-sysctl-d\") pod \"tuned-lwkzv\" (UID: \"7323d14a-1095-4021-982f-9162ee160fe5\") " pod="openshift-cluster-node-tuning-operator/tuned-lwkzv"
Sep 07 11:47:10.049378 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:10.046746    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/55db3d09-7464-4f97-af5a-985162756ebb-cnibin\") pod \"multus-additional-cni-plugins-mv7nw\" (UID: \"55db3d09-7464-4f97-af5a-985162756ebb\") " pod="openshift-multus/multus-additional-cni-plugins-mv7nw"
Sep 07 11:47:10.049378 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:10.046769    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/b0f92581-a3a7-444e-b915-dc6cc634fe26-run-ovn\") pod \"ovnkube-node-swqfh\" (UID: \"b0f92581-a3a7-444e-b915-dc6cc634fe26\") " pod="openshift-ovn-kubernetes/ovnkube-node-swqfh"
Sep 07 11:47:10.049378 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:10.046781    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/7323d14a-1095-4021-982f-9162ee160fe5-etc-sysconfig\") pod \"tuned-lwkzv\" (UID: \"7323d14a-1095-4021-982f-9162ee160fe5\") " pod="openshift-cluster-node-tuning-operator/tuned-lwkzv"
Sep 07 11:47:10.049378 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:10.046833    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/55db3d09-7464-4f97-af5a-985162756ebb-cnibin\") pod \"multus-additional-cni-plugins-mv7nw\" (UID: \"55db3d09-7464-4f97-af5a-985162756ebb\") " pod="openshift-multus/multus-additional-cni-plugins-mv7nw"
Sep 07 11:47:10.049378 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:10.046875    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/b0f92581-a3a7-444e-b915-dc6cc634fe26-run-ovn\") pod \"ovnkube-node-swqfh\" (UID: \"b0f92581-a3a7-444e-b915-dc6cc634fe26\") " pod="openshift-ovn-kubernetes/ovnkube-node-swqfh"
Sep 07 11:47:10.049378 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:10.046893    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/7323d14a-1095-4021-982f-9162ee160fe5-etc-sysctl-conf\") pod \"tuned-lwkzv\" (UID: \"7323d14a-1095-4021-982f-9162ee160fe5\") " pod="openshift-cluster-node-tuning-operator/tuned-lwkzv"
Sep 07 11:47:10.049378 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:10.047051    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/b9aae943-4d29-4c1e-909f-0a1c034299f8-serviceca\") pod \"node-ca-wrd4z\" (UID: \"b9aae943-4d29-4c1e-909f-0a1c034299f8\") " pod="openshift-image-registry/node-ca-wrd4z"
Sep 07 11:47:10.049378 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:10.047189    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/3e71d2e5-bc97-415d-8e01-cdf554957349-konnectivity-ca\") pod \"konnectivity-agent-dl69z\" (UID: \"3e71d2e5-bc97-415d-8e01-cdf554957349\") " pod="kube-system/konnectivity-agent-dl69z"
Sep 07 11:47:10.049378 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:10.047239    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/b0f92581-a3a7-444e-b915-dc6cc634fe26-env-overrides\") pod \"ovnkube-node-swqfh\" (UID: \"b0f92581-a3a7-444e-b915-dc6cc634fe26\") " pod="openshift-ovn-kubernetes/ovnkube-node-swqfh"
Sep 07 11:47:10.049378 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:10.047559    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/55db3d09-7464-4f97-af5a-985162756ebb-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-mv7nw\" (UID: \"55db3d09-7464-4f97-af5a-985162756ebb\") " pod="openshift-multus/multus-additional-cni-plugins-mv7nw"
Sep 07 11:47:10.049378 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:10.048316    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7323d14a-1095-4021-982f-9162ee160fe5-tmp\") pod \"tuned-lwkzv\" (UID: \"7323d14a-1095-4021-982f-9162ee160fe5\") " pod="openshift-cluster-node-tuning-operator/tuned-lwkzv"
Sep 07 11:47:10.049378 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:10.048746    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/7323d14a-1095-4021-982f-9162ee160fe5-etc-tuned\") pod \"tuned-lwkzv\" (UID: \"7323d14a-1095-4021-982f-9162ee160fe5\") " pod="openshift-cluster-node-tuning-operator/tuned-lwkzv"
Sep 07 11:47:10.049890 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:10.048944    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/b0f92581-a3a7-444e-b915-dc6cc634fe26-ovn-node-metrics-cert\") pod \"ovnkube-node-swqfh\" (UID: \"b0f92581-a3a7-444e-b915-dc6cc634fe26\") " pod="openshift-ovn-kubernetes/ovnkube-node-swqfh"
Sep 07 11:47:10.049890 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:10.049158    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/3e71d2e5-bc97-415d-8e01-cdf554957349-agent-certs\") pod \"konnectivity-agent-dl69z\" (UID: \"3e71d2e5-bc97-415d-8e01-cdf554957349\") " pod="kube-system/konnectivity-agent-dl69z"
Sep 07 11:47:10.051768 ip-10-0-135-173 kubenswrapper[2563]: E0907 11:47:10.051749    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 07 11:47:10.051847 ip-10-0-135-173 kubenswrapper[2563]: E0907 11:47:10.051779    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 07 11:47:10.051847 ip-10-0-135-173 kubenswrapper[2563]: E0907 11:47:10.051793    2563 projected.go:196] Error preparing data for projected volume kube-api-access-f9tp8 for pod openshift-network-diagnostics/network-check-target-js7hj: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 07 11:47:10.051933 ip-10-0-135-173 kubenswrapper[2563]: E0907 11:47:10.051870    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8d63cc6a-3920-45b6-9626-45b02324e6ce-kube-api-access-f9tp8 podName:8d63cc6a-3920-45b6-9626-45b02324e6ce nodeName:}" failed. No retries permitted until 2026-09-07 11:47:10.551851033 +0000 UTC m=+2.070582849 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-f9tp8" (UniqueName: "kubernetes.io/projected/8d63cc6a-3920-45b6-9626-45b02324e6ce-kube-api-access-f9tp8") pod "network-check-target-js7hj" (UID: "8d63cc6a-3920-45b6-9626-45b02324e6ce") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 07 11:47:10.053857 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:10.053836    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wht8w\" (UniqueName: \"kubernetes.io/projected/7323d14a-1095-4021-982f-9162ee160fe5-kube-api-access-wht8w\") pod \"tuned-lwkzv\" (UID: \"7323d14a-1095-4021-982f-9162ee160fe5\") " pod="openshift-cluster-node-tuning-operator/tuned-lwkzv"
Sep 07 11:47:10.054333 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:10.054313    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sgbkf\" (UniqueName: \"kubernetes.io/projected/71bc595d-9844-4086-90fd-28341f888e30-kube-api-access-sgbkf\") pod \"aws-ebs-csi-driver-node-rmjjm\" (UID: \"71bc595d-9844-4086-90fd-28341f888e30\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rmjjm"
Sep 07 11:47:10.054696 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:10.054671    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l46jr\" (UniqueName: \"kubernetes.io/projected/5f723277-6450-4fd8-8f5d-01d9f9f6814b-kube-api-access-l46jr\") pod \"node-resolver-hxstl\" (UID: \"5f723277-6450-4fd8-8f5d-01d9f9f6814b\") " pod="openshift-dns/node-resolver-hxstl"
Sep 07 11:47:10.054886 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:10.054862    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6knwx\" (UniqueName: \"kubernetes.io/projected/b0f92581-a3a7-444e-b915-dc6cc634fe26-kube-api-access-6knwx\") pod \"ovnkube-node-swqfh\" (UID: \"b0f92581-a3a7-444e-b915-dc6cc634fe26\") " pod="openshift-ovn-kubernetes/ovnkube-node-swqfh"
Sep 07 11:47:10.055108 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:10.055090    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xtl7b\" (UniqueName: \"kubernetes.io/projected/b9aae943-4d29-4c1e-909f-0a1c034299f8-kube-api-access-xtl7b\") pod \"node-ca-wrd4z\" (UID: \"b9aae943-4d29-4c1e-909f-0a1c034299f8\") " pod="openshift-image-registry/node-ca-wrd4z"
Sep 07 11:47:10.055235 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:10.055217    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nht46\" (UniqueName: \"kubernetes.io/projected/55db3d09-7464-4f97-af5a-985162756ebb-kube-api-access-nht46\") pod \"multus-additional-cni-plugins-mv7nw\" (UID: \"55db3d09-7464-4f97-af5a-985162756ebb\") " pod="openshift-multus/multus-additional-cni-plugins-mv7nw"
Sep 07 11:47:10.104217 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:10.104196    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7d94e6a6c726d03776a8387d983ae9ef.slice/crio-566d9c718c73e49d23dae6988a3dd8602db608f4cdbaf90504891127c535e952 WatchSource:0}: Error finding container 566d9c718c73e49d23dae6988a3dd8602db608f4cdbaf90504891127c535e952: Status 404 returned error can't find the container with id 566d9c718c73e49d23dae6988a3dd8602db608f4cdbaf90504891127c535e952
Sep 07 11:47:10.104370 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:10.104358    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0baf97a2450fb410ba0091864614894a.slice/crio-7f93b6518b080440bf21b62f6ef82f6fff5286ff3ed250b79a25d072c075661f WatchSource:0}: Error finding container 7f93b6518b080440bf21b62f6ef82f6fff5286ff3ed250b79a25d072c075661f: Status 404 returned error can't find the container with id 7f93b6518b080440bf21b62f6ef82f6fff5286ff3ed250b79a25d072c075661f
Sep 07 11:47:10.110265 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:10.110250    2563 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 07 11:47:10.161293 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:10.161272    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-g2v7c"
Sep 07 11:47:10.169390 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:10.169372    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-j9blv"
Sep 07 11:47:10.169912 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:10.169894    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod64362f4d_7d1d_4faa_a02e_4a7dec54cffc.slice/crio-0e4e531369a63ecce0033fd045b5cd356063947c290537fd36cdf4b720775dbf WatchSource:0}: Error finding container 0e4e531369a63ecce0033fd045b5cd356063947c290537fd36cdf4b720775dbf: Status 404 returned error can't find the container with id 0e4e531369a63ecce0033fd045b5cd356063947c290537fd36cdf4b720775dbf
Sep 07 11:47:10.177080 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:10.177063    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8b7fc53e_4815_4833_a4dc_ad05cd3834c4.slice/crio-e8572e9fd7977a58a1da8ba2ad405ed967a1086695addf94e33a1fb171454c7b WatchSource:0}: Error finding container e8572e9fd7977a58a1da8ba2ad405ed967a1086695addf94e33a1fb171454c7b: Status 404 returned error can't find the container with id e8572e9fd7977a58a1da8ba2ad405ed967a1086695addf94e33a1fb171454c7b
Sep 07 11:47:10.188573 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:10.188556    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-dl69z"
Sep 07 11:47:10.193282 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:10.193265    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-hxstl"
Sep 07 11:47:10.196357 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:10.196336    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3e71d2e5_bc97_415d_8e01_cdf554957349.slice/crio-251c63e37c4e7299e76244e6964c0894dd519246eaac0292f9320cc94be65e4e WatchSource:0}: Error finding container 251c63e37c4e7299e76244e6964c0894dd519246eaac0292f9320cc94be65e4e: Status 404 returned error can't find the container with id 251c63e37c4e7299e76244e6964c0894dd519246eaac0292f9320cc94be65e4e
Sep 07 11:47:10.201228 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:10.201209    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5f723277_6450_4fd8_8f5d_01d9f9f6814b.slice/crio-85fa1923181abe2fc8a9f414746baa4a0fd7c3906401190c526074ac5e71a85e WatchSource:0}: Error finding container 85fa1923181abe2fc8a9f414746baa4a0fd7c3906401190c526074ac5e71a85e: Status 404 returned error can't find the container with id 85fa1923181abe2fc8a9f414746baa4a0fd7c3906401190c526074ac5e71a85e
Sep 07 11:47:10.216411 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:10.216394    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-wrd4z"
Sep 07 11:47:10.224665 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:10.224646    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb9aae943_4d29_4c1e_909f_0a1c034299f8.slice/crio-2d4b795bcdc067b41cdfaf4f74030fa806b94365e10e3ec1cc9a17e3c7c32bee WatchSource:0}: Error finding container 2d4b795bcdc067b41cdfaf4f74030fa806b94365e10e3ec1cc9a17e3c7c32bee: Status 404 returned error can't find the container with id 2d4b795bcdc067b41cdfaf4f74030fa806b94365e10e3ec1cc9a17e3c7c32bee
Sep 07 11:47:10.233478 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:10.233461    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-swqfh"
Sep 07 11:47:10.239267 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:10.239252    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rmjjm"
Sep 07 11:47:10.241100 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:10.241082    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb0f92581_a3a7_444e_b915_dc6cc634fe26.slice/crio-77c3d358cff7c12bea07985eb9e1eb17278a1559b5946b230907252624375f80 WatchSource:0}: Error finding container 77c3d358cff7c12bea07985eb9e1eb17278a1559b5946b230907252624375f80: Status 404 returned error can't find the container with id 77c3d358cff7c12bea07985eb9e1eb17278a1559b5946b230907252624375f80
Sep 07 11:47:10.243012 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:10.242997    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-lwkzv"
Sep 07 11:47:10.247721 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:10.247693    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod71bc595d_9844_4086_90fd_28341f888e30.slice/crio-0cf187909b88356f056e62668f9e0eee64a897c5f587e94737dc215dc5a56379 WatchSource:0}: Error finding container 0cf187909b88356f056e62668f9e0eee64a897c5f587e94737dc215dc5a56379: Status 404 returned error can't find the container with id 0cf187909b88356f056e62668f9e0eee64a897c5f587e94737dc215dc5a56379
Sep 07 11:47:10.252005 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:10.251984    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7323d14a_1095_4021_982f_9162ee160fe5.slice/crio-544e8486d817ef09e7caef125c1c024abe5a62ef5f25376decec5b3ce33ab508 WatchSource:0}: Error finding container 544e8486d817ef09e7caef125c1c024abe5a62ef5f25376decec5b3ce33ab508: Status 404 returned error can't find the container with id 544e8486d817ef09e7caef125c1c024abe5a62ef5f25376decec5b3ce33ab508
Sep 07 11:47:10.262098 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:10.262082    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-mv7nw"
Sep 07 11:47:10.270408 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:10.270259    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod55db3d09_7464_4f97_af5a_985162756ebb.slice/crio-5794a7f1360d77677d39868c513d125c35ff34c4692533df670feb3fcb7512de WatchSource:0}: Error finding container 5794a7f1360d77677d39868c513d125c35ff34c4692533df670feb3fcb7512de: Status 404 returned error can't find the container with id 5794a7f1360d77677d39868c513d125c35ff34c4692533df670feb3fcb7512de
Sep 07 11:47:10.271762 ip-10-0-135-173 kubenswrapper[2563]: E0907 11:47:10.271743    2563 kuberuntime_manager.go:1459] "Unhandled Error" err="init container egress-router-binary-copy start failed in pod multus-additional-cni-plugins-mv7nw_openshift-multus(55db3d09-7464-4f97-af5a-985162756ebb): ErrImagePull: pull QPS exceeded" logger="UnhandledError"
Sep 07 11:47:10.271830 ip-10-0-135-173 kubenswrapper[2563]: E0907 11:47:10.271778    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"egress-router-binary-copy\" with ErrImagePull: \"pull QPS exceeded\"" pod="openshift-multus/multus-additional-cni-plugins-mv7nw" podUID="55db3d09-7464-4f97-af5a-985162756ebb"
Sep 07 11:47:10.449278 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:10.449201    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3acaae8d-5930-49c6-9b9e-d286ac6a75d5-metrics-certs\") pod \"network-metrics-daemon-xf2gj\" (UID: \"3acaae8d-5930-49c6-9b9e-d286ac6a75d5\") " pod="openshift-multus/network-metrics-daemon-xf2gj"
Sep 07 11:47:10.449439 ip-10-0-135-173 kubenswrapper[2563]: E0907 11:47:10.449310    2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 07 11:47:10.449439 ip-10-0-135-173 kubenswrapper[2563]: E0907 11:47:10.449364    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3acaae8d-5930-49c6-9b9e-d286ac6a75d5-metrics-certs podName:3acaae8d-5930-49c6-9b9e-d286ac6a75d5 nodeName:}" failed. No retries permitted until 2026-09-07 11:47:11.449349735 +0000 UTC m=+2.968081526 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3acaae8d-5930-49c6-9b9e-d286ac6a75d5-metrics-certs") pod "network-metrics-daemon-xf2gj" (UID: "3acaae8d-5930-49c6-9b9e-d286ac6a75d5") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 07 11:47:10.550062 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:10.549997    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e92172b6-ddda-4b65-81b3-987b307e272e-original-pull-secret\") pod \"global-pull-secret-syncer-nzzj9\" (UID: \"e92172b6-ddda-4b65-81b3-987b307e272e\") " pod="kube-system/global-pull-secret-syncer-nzzj9"
Sep 07 11:47:10.550236 ip-10-0-135-173 kubenswrapper[2563]: E0907 11:47:10.550127    2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 07 11:47:10.550236 ip-10-0-135-173 kubenswrapper[2563]: E0907 11:47:10.550215    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e92172b6-ddda-4b65-81b3-987b307e272e-original-pull-secret podName:e92172b6-ddda-4b65-81b3-987b307e272e nodeName:}" failed. No retries permitted until 2026-09-07 11:47:11.550193725 +0000 UTC m=+3.068925526 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e92172b6-ddda-4b65-81b3-987b307e272e-original-pull-secret") pod "global-pull-secret-syncer-nzzj9" (UID: "e92172b6-ddda-4b65-81b3-987b307e272e") : object "kube-system"/"original-pull-secret" not registered
Sep 07 11:47:10.602968 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:10.602757    2563 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 07 11:47:10.651170 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:10.650520    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f9tp8\" (UniqueName: \"kubernetes.io/projected/8d63cc6a-3920-45b6-9626-45b02324e6ce-kube-api-access-f9tp8\") pod \"network-check-target-js7hj\" (UID: \"8d63cc6a-3920-45b6-9626-45b02324e6ce\") " pod="openshift-network-diagnostics/network-check-target-js7hj"
Sep 07 11:47:10.651170 ip-10-0-135-173 kubenswrapper[2563]: E0907 11:47:10.650747    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 07 11:47:10.651170 ip-10-0-135-173 kubenswrapper[2563]: E0907 11:47:10.650767    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 07 11:47:10.651170 ip-10-0-135-173 kubenswrapper[2563]: E0907 11:47:10.650781    2563 projected.go:196] Error preparing data for projected volume kube-api-access-f9tp8 for pod openshift-network-diagnostics/network-check-target-js7hj: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 07 11:47:10.651170 ip-10-0-135-173 kubenswrapper[2563]: E0907 11:47:10.650839    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8d63cc6a-3920-45b6-9626-45b02324e6ce-kube-api-access-f9tp8 podName:8d63cc6a-3920-45b6-9626-45b02324e6ce nodeName:}" failed. No retries permitted until 2026-09-07 11:47:11.650821872 +0000 UTC m=+3.169553684 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-f9tp8" (UniqueName: "kubernetes.io/projected/8d63cc6a-3920-45b6-9626-45b02324e6ce-kube-api-access-f9tp8") pod "network-check-target-js7hj" (UID: "8d63cc6a-3920-45b6-9626-45b02324e6ce") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 07 11:47:10.885937 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:10.885826    2563 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-06 11:42:09 +0000 UTC" deadline="2028-03-05 09:12:59.9159821 +0000 UTC"
Sep 07 11:47:10.885937 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:10.885885    2563 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13077h25m49.030102064s"
Sep 07 11:47:11.034242 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:11.034212    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-js7hj"
Sep 07 11:47:11.034402 ip-10-0-135-173 kubenswrapper[2563]: E0907 11:47:11.034333    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-js7hj" podUID="8d63cc6a-3920-45b6-9626-45b02324e6ce"
Sep 07 11:47:11.034797 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:11.034776    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xf2gj"
Sep 07 11:47:11.034901 ip-10-0-135-173 kubenswrapper[2563]: E0907 11:47:11.034883    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xf2gj" podUID="3acaae8d-5930-49c6-9b9e-d286ac6a75d5"
Sep 07 11:47:11.046724 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:11.046676    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-lwkzv" event={"ID":"7323d14a-1095-4021-982f-9162ee160fe5","Type":"ContainerStarted","Data":"544e8486d817ef09e7caef125c1c024abe5a62ef5f25376decec5b3ce33ab508"}
Sep 07 11:47:11.055919 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:11.055891    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rmjjm" event={"ID":"71bc595d-9844-4086-90fd-28341f888e30","Type":"ContainerStarted","Data":"0cf187909b88356f056e62668f9e0eee64a897c5f587e94737dc215dc5a56379"}
Sep 07 11:47:11.058090 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:11.058060    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-173.ec2.internal" event={"ID":"7d94e6a6c726d03776a8387d983ae9ef","Type":"ContainerStarted","Data":"566d9c718c73e49d23dae6988a3dd8602db608f4cdbaf90504891127c535e952"}
Sep 07 11:47:11.060360 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:11.060335    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mv7nw" event={"ID":"55db3d09-7464-4f97-af5a-985162756ebb","Type":"ContainerStarted","Data":"5794a7f1360d77677d39868c513d125c35ff34c4692533df670feb3fcb7512de"}
Sep 07 11:47:11.062703 ip-10-0-135-173 kubenswrapper[2563]: E0907 11:47:11.062675    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"egress-router-binary-copy\" with ImagePullBackOff: \"Back-off pulling image \\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:0d2fae617617ccbdef514e3068eb3db7ed6f5869fe748bbfc9d66eb536d3e26f\\\": ErrImagePull: pull QPS exceeded\"" pod="openshift-multus/multus-additional-cni-plugins-mv7nw" podUID="55db3d09-7464-4f97-af5a-985162756ebb"
Sep 07 11:47:11.062796 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:11.062773    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-swqfh" event={"ID":"b0f92581-a3a7-444e-b915-dc6cc634fe26","Type":"ContainerStarted","Data":"77c3d358cff7c12bea07985eb9e1eb17278a1559b5946b230907252624375f80"}
Sep 07 11:47:11.065324 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:11.065301    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-wrd4z" event={"ID":"b9aae943-4d29-4c1e-909f-0a1c034299f8","Type":"ContainerStarted","Data":"2d4b795bcdc067b41cdfaf4f74030fa806b94365e10e3ec1cc9a17e3c7c32bee"}
Sep 07 11:47:11.075863 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:11.075824    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-hxstl" event={"ID":"5f723277-6450-4fd8-8f5d-01d9f9f6814b","Type":"ContainerStarted","Data":"85fa1923181abe2fc8a9f414746baa4a0fd7c3906401190c526074ac5e71a85e"}
Sep 07 11:47:11.080317 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:11.080262    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-dl69z" event={"ID":"3e71d2e5-bc97-415d-8e01-cdf554957349","Type":"ContainerStarted","Data":"251c63e37c4e7299e76244e6964c0894dd519246eaac0292f9320cc94be65e4e"}
Sep 07 11:47:11.091264 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:11.091240    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-j9blv" event={"ID":"8b7fc53e-4815-4833-a4dc-ad05cd3834c4","Type":"ContainerStarted","Data":"e8572e9fd7977a58a1da8ba2ad405ed967a1086695addf94e33a1fb171454c7b"}
Sep 07 11:47:11.105887 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:11.105867    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-g2v7c" event={"ID":"64362f4d-7d1d-4faa-a02e-4a7dec54cffc","Type":"ContainerStarted","Data":"0e4e531369a63ecce0033fd045b5cd356063947c290537fd36cdf4b720775dbf"}
Sep 07 11:47:11.120084 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:11.120061    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-173.ec2.internal" event={"ID":"0baf97a2450fb410ba0091864614894a","Type":"ContainerStarted","Data":"7f93b6518b080440bf21b62f6ef82f6fff5286ff3ed250b79a25d072c075661f"}
Sep 07 11:47:11.305252 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:11.305010    2563 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 07 11:47:11.458393 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:11.458360    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3acaae8d-5930-49c6-9b9e-d286ac6a75d5-metrics-certs\") pod \"network-metrics-daemon-xf2gj\" (UID: \"3acaae8d-5930-49c6-9b9e-d286ac6a75d5\") " pod="openshift-multus/network-metrics-daemon-xf2gj"
Sep 07 11:47:11.458583 ip-10-0-135-173 kubenswrapper[2563]: E0907 11:47:11.458538    2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 07 11:47:11.458653 ip-10-0-135-173 kubenswrapper[2563]: E0907 11:47:11.458596    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3acaae8d-5930-49c6-9b9e-d286ac6a75d5-metrics-certs podName:3acaae8d-5930-49c6-9b9e-d286ac6a75d5 nodeName:}" failed. No retries permitted until 2026-09-07 11:47:13.458578077 +0000 UTC m=+4.977309874 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3acaae8d-5930-49c6-9b9e-d286ac6a75d5-metrics-certs") pod "network-metrics-daemon-xf2gj" (UID: "3acaae8d-5930-49c6-9b9e-d286ac6a75d5") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 07 11:47:11.559702 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:11.559581    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e92172b6-ddda-4b65-81b3-987b307e272e-original-pull-secret\") pod \"global-pull-secret-syncer-nzzj9\" (UID: \"e92172b6-ddda-4b65-81b3-987b307e272e\") " pod="kube-system/global-pull-secret-syncer-nzzj9"
Sep 07 11:47:11.559860 ip-10-0-135-173 kubenswrapper[2563]: E0907 11:47:11.559780    2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 07 11:47:11.559860 ip-10-0-135-173 kubenswrapper[2563]: E0907 11:47:11.559840    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e92172b6-ddda-4b65-81b3-987b307e272e-original-pull-secret podName:e92172b6-ddda-4b65-81b3-987b307e272e nodeName:}" failed. No retries permitted until 2026-09-07 11:47:13.55982318 +0000 UTC m=+5.078554976 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e92172b6-ddda-4b65-81b3-987b307e272e-original-pull-secret") pod "global-pull-secret-syncer-nzzj9" (UID: "e92172b6-ddda-4b65-81b3-987b307e272e") : object "kube-system"/"original-pull-secret" not registered
Sep 07 11:47:11.660429 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:11.660370    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f9tp8\" (UniqueName: \"kubernetes.io/projected/8d63cc6a-3920-45b6-9626-45b02324e6ce-kube-api-access-f9tp8\") pod \"network-check-target-js7hj\" (UID: \"8d63cc6a-3920-45b6-9626-45b02324e6ce\") " pod="openshift-network-diagnostics/network-check-target-js7hj"
Sep 07 11:47:11.660593 ip-10-0-135-173 kubenswrapper[2563]: E0907 11:47:11.660531    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 07 11:47:11.660593 ip-10-0-135-173 kubenswrapper[2563]: E0907 11:47:11.660550    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 07 11:47:11.660593 ip-10-0-135-173 kubenswrapper[2563]: E0907 11:47:11.660563    2563 projected.go:196] Error preparing data for projected volume kube-api-access-f9tp8 for pod openshift-network-diagnostics/network-check-target-js7hj: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 07 11:47:11.660757 ip-10-0-135-173 kubenswrapper[2563]: E0907 11:47:11.660619    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8d63cc6a-3920-45b6-9626-45b02324e6ce-kube-api-access-f9tp8 podName:8d63cc6a-3920-45b6-9626-45b02324e6ce nodeName:}" failed. No retries permitted until 2026-09-07 11:47:13.660600238 +0000 UTC m=+5.179332051 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-f9tp8" (UniqueName: "kubernetes.io/projected/8d63cc6a-3920-45b6-9626-45b02324e6ce-kube-api-access-f9tp8") pod "network-check-target-js7hj" (UID: "8d63cc6a-3920-45b6-9626-45b02324e6ce") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 07 11:47:11.887119 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:11.886976    2563 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-06 11:42:09 +0000 UTC" deadline="2028-06-03 14:52:34.12345573 +0000 UTC"
Sep 07 11:47:11.887119 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:11.887014    2563 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15243h5m22.23644635s"
Sep 07 11:47:12.034701 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:12.034596    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nzzj9"
Sep 07 11:47:12.034876 ip-10-0-135-173 kubenswrapper[2563]: E0907 11:47:12.034746    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nzzj9" podUID="e92172b6-ddda-4b65-81b3-987b307e272e"
Sep 07 11:47:12.230699 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:12.230621    2563 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 07 11:47:13.036438 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:13.035145    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-js7hj"
Sep 07 11:47:13.036438 ip-10-0-135-173 kubenswrapper[2563]: E0907 11:47:13.035268    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-js7hj" podUID="8d63cc6a-3920-45b6-9626-45b02324e6ce"
Sep 07 11:47:13.036438 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:13.035641    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xf2gj"
Sep 07 11:47:13.036438 ip-10-0-135-173 kubenswrapper[2563]: E0907 11:47:13.035753    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xf2gj" podUID="3acaae8d-5930-49c6-9b9e-d286ac6a75d5"
Sep 07 11:47:13.475031 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:13.474945    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3acaae8d-5930-49c6-9b9e-d286ac6a75d5-metrics-certs\") pod \"network-metrics-daemon-xf2gj\" (UID: \"3acaae8d-5930-49c6-9b9e-d286ac6a75d5\") " pod="openshift-multus/network-metrics-daemon-xf2gj"
Sep 07 11:47:13.475199 ip-10-0-135-173 kubenswrapper[2563]: E0907 11:47:13.475108    2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 07 11:47:13.475199 ip-10-0-135-173 kubenswrapper[2563]: E0907 11:47:13.475174    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3acaae8d-5930-49c6-9b9e-d286ac6a75d5-metrics-certs podName:3acaae8d-5930-49c6-9b9e-d286ac6a75d5 nodeName:}" failed. No retries permitted until 2026-09-07 11:47:17.475150021 +0000 UTC m=+8.993881815 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3acaae8d-5930-49c6-9b9e-d286ac6a75d5-metrics-certs") pod "network-metrics-daemon-xf2gj" (UID: "3acaae8d-5930-49c6-9b9e-d286ac6a75d5") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 07 11:47:13.576395 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:13.576354    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e92172b6-ddda-4b65-81b3-987b307e272e-original-pull-secret\") pod \"global-pull-secret-syncer-nzzj9\" (UID: \"e92172b6-ddda-4b65-81b3-987b307e272e\") " pod="kube-system/global-pull-secret-syncer-nzzj9"
Sep 07 11:47:13.576579 ip-10-0-135-173 kubenswrapper[2563]: E0907 11:47:13.576532    2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 07 11:47:13.576658 ip-10-0-135-173 kubenswrapper[2563]: E0907 11:47:13.576604    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e92172b6-ddda-4b65-81b3-987b307e272e-original-pull-secret podName:e92172b6-ddda-4b65-81b3-987b307e272e nodeName:}" failed. No retries permitted until 2026-09-07 11:47:17.576583461 +0000 UTC m=+9.095315256 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e92172b6-ddda-4b65-81b3-987b307e272e-original-pull-secret") pod "global-pull-secret-syncer-nzzj9" (UID: "e92172b6-ddda-4b65-81b3-987b307e272e") : object "kube-system"/"original-pull-secret" not registered
Sep 07 11:47:13.677624 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:13.676937    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f9tp8\" (UniqueName: \"kubernetes.io/projected/8d63cc6a-3920-45b6-9626-45b02324e6ce-kube-api-access-f9tp8\") pod \"network-check-target-js7hj\" (UID: \"8d63cc6a-3920-45b6-9626-45b02324e6ce\") " pod="openshift-network-diagnostics/network-check-target-js7hj"
Sep 07 11:47:13.677624 ip-10-0-135-173 kubenswrapper[2563]: E0907 11:47:13.677161    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 07 11:47:13.677624 ip-10-0-135-173 kubenswrapper[2563]: E0907 11:47:13.677186    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 07 11:47:13.677624 ip-10-0-135-173 kubenswrapper[2563]: E0907 11:47:13.677199    2563 projected.go:196] Error preparing data for projected volume kube-api-access-f9tp8 for pod openshift-network-diagnostics/network-check-target-js7hj: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 07 11:47:13.677624 ip-10-0-135-173 kubenswrapper[2563]: E0907 11:47:13.677259    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8d63cc6a-3920-45b6-9626-45b02324e6ce-kube-api-access-f9tp8 podName:8d63cc6a-3920-45b6-9626-45b02324e6ce nodeName:}" failed. No retries permitted until 2026-09-07 11:47:17.677241282 +0000 UTC m=+9.195973092 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-f9tp8" (UniqueName: "kubernetes.io/projected/8d63cc6a-3920-45b6-9626-45b02324e6ce-kube-api-access-f9tp8") pod "network-check-target-js7hj" (UID: "8d63cc6a-3920-45b6-9626-45b02324e6ce") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 07 11:47:14.034661 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:14.034627    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nzzj9"
Sep 07 11:47:14.034831 ip-10-0-135-173 kubenswrapper[2563]: E0907 11:47:14.034763    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nzzj9" podUID="e92172b6-ddda-4b65-81b3-987b307e272e"
Sep 07 11:47:15.035304 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:15.034696    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-js7hj"
Sep 07 11:47:15.035304 ip-10-0-135-173 kubenswrapper[2563]: E0907 11:47:15.034826    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-js7hj" podUID="8d63cc6a-3920-45b6-9626-45b02324e6ce"
Sep 07 11:47:15.035304 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:15.034696    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xf2gj"
Sep 07 11:47:15.035304 ip-10-0-135-173 kubenswrapper[2563]: E0907 11:47:15.035262    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xf2gj" podUID="3acaae8d-5930-49c6-9b9e-d286ac6a75d5"
Sep 07 11:47:16.034528 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:16.034464    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nzzj9"
Sep 07 11:47:16.034713 ip-10-0-135-173 kubenswrapper[2563]: E0907 11:47:16.034597    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nzzj9" podUID="e92172b6-ddda-4b65-81b3-987b307e272e"
Sep 07 11:47:17.034684 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:17.034652    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-js7hj"
Sep 07 11:47:17.035147 ip-10-0-135-173 kubenswrapper[2563]: E0907 11:47:17.034763    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-js7hj" podUID="8d63cc6a-3920-45b6-9626-45b02324e6ce"
Sep 07 11:47:17.035147 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:17.034658    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xf2gj"
Sep 07 11:47:17.035147 ip-10-0-135-173 kubenswrapper[2563]: E0907 11:47:17.034887    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xf2gj" podUID="3acaae8d-5930-49c6-9b9e-d286ac6a75d5"
Sep 07 11:47:17.509651 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:17.509608    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3acaae8d-5930-49c6-9b9e-d286ac6a75d5-metrics-certs\") pod \"network-metrics-daemon-xf2gj\" (UID: \"3acaae8d-5930-49c6-9b9e-d286ac6a75d5\") " pod="openshift-multus/network-metrics-daemon-xf2gj"
Sep 07 11:47:17.509851 ip-10-0-135-173 kubenswrapper[2563]: E0907 11:47:17.509796    2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 07 11:47:17.509920 ip-10-0-135-173 kubenswrapper[2563]: E0907 11:47:17.509878    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3acaae8d-5930-49c6-9b9e-d286ac6a75d5-metrics-certs podName:3acaae8d-5930-49c6-9b9e-d286ac6a75d5 nodeName:}" failed. No retries permitted until 2026-09-07 11:47:25.509855928 +0000 UTC m=+17.028587728 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3acaae8d-5930-49c6-9b9e-d286ac6a75d5-metrics-certs") pod "network-metrics-daemon-xf2gj" (UID: "3acaae8d-5930-49c6-9b9e-d286ac6a75d5") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 07 11:47:17.611291 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:17.610727    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e92172b6-ddda-4b65-81b3-987b307e272e-original-pull-secret\") pod \"global-pull-secret-syncer-nzzj9\" (UID: \"e92172b6-ddda-4b65-81b3-987b307e272e\") " pod="kube-system/global-pull-secret-syncer-nzzj9"
Sep 07 11:47:17.611291 ip-10-0-135-173 kubenswrapper[2563]: E0907 11:47:17.610876    2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 07 11:47:17.611291 ip-10-0-135-173 kubenswrapper[2563]: E0907 11:47:17.610945    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e92172b6-ddda-4b65-81b3-987b307e272e-original-pull-secret podName:e92172b6-ddda-4b65-81b3-987b307e272e nodeName:}" failed. No retries permitted until 2026-09-07 11:47:25.610925459 +0000 UTC m=+17.129657253 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e92172b6-ddda-4b65-81b3-987b307e272e-original-pull-secret") pod "global-pull-secret-syncer-nzzj9" (UID: "e92172b6-ddda-4b65-81b3-987b307e272e") : object "kube-system"/"original-pull-secret" not registered
Sep 07 11:47:17.711272 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:17.711229    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f9tp8\" (UniqueName: \"kubernetes.io/projected/8d63cc6a-3920-45b6-9626-45b02324e6ce-kube-api-access-f9tp8\") pod \"network-check-target-js7hj\" (UID: \"8d63cc6a-3920-45b6-9626-45b02324e6ce\") " pod="openshift-network-diagnostics/network-check-target-js7hj"
Sep 07 11:47:17.711470 ip-10-0-135-173 kubenswrapper[2563]: E0907 11:47:17.711451    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 07 11:47:17.711552 ip-10-0-135-173 kubenswrapper[2563]: E0907 11:47:17.711477    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 07 11:47:17.711552 ip-10-0-135-173 kubenswrapper[2563]: E0907 11:47:17.711491    2563 projected.go:196] Error preparing data for projected volume kube-api-access-f9tp8 for pod openshift-network-diagnostics/network-check-target-js7hj: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 07 11:47:17.711660 ip-10-0-135-173 kubenswrapper[2563]: E0907 11:47:17.711558    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8d63cc6a-3920-45b6-9626-45b02324e6ce-kube-api-access-f9tp8 podName:8d63cc6a-3920-45b6-9626-45b02324e6ce nodeName:}" failed. No retries permitted until 2026-09-07 11:47:25.711537296 +0000 UTC m=+17.230269091 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-f9tp8" (UniqueName: "kubernetes.io/projected/8d63cc6a-3920-45b6-9626-45b02324e6ce-kube-api-access-f9tp8") pod "network-check-target-js7hj" (UID: "8d63cc6a-3920-45b6-9626-45b02324e6ce") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 07 11:47:18.034562 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:18.034532    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nzzj9"
Sep 07 11:47:18.034753 ip-10-0-135-173 kubenswrapper[2563]: E0907 11:47:18.034660    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nzzj9" podUID="e92172b6-ddda-4b65-81b3-987b307e272e"
Sep 07 11:47:19.036364 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:19.036329    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xf2gj"
Sep 07 11:47:19.036828 ip-10-0-135-173 kubenswrapper[2563]: E0907 11:47:19.036483    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xf2gj" podUID="3acaae8d-5930-49c6-9b9e-d286ac6a75d5"
Sep 07 11:47:19.037740 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:19.037541    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-js7hj"
Sep 07 11:47:19.037740 ip-10-0-135-173 kubenswrapper[2563]: E0907 11:47:19.037646    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-js7hj" podUID="8d63cc6a-3920-45b6-9626-45b02324e6ce"
Sep 07 11:47:20.034827 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:20.034798    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nzzj9"
Sep 07 11:47:20.034998 ip-10-0-135-173 kubenswrapper[2563]: E0907 11:47:20.034910    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nzzj9" podUID="e92172b6-ddda-4b65-81b3-987b307e272e"
Sep 07 11:47:21.037153 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:21.037124    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xf2gj"
Sep 07 11:47:21.037731 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:21.037158    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-js7hj"
Sep 07 11:47:21.037731 ip-10-0-135-173 kubenswrapper[2563]: E0907 11:47:21.037260    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xf2gj" podUID="3acaae8d-5930-49c6-9b9e-d286ac6a75d5"
Sep 07 11:47:21.037731 ip-10-0-135-173 kubenswrapper[2563]: E0907 11:47:21.037402    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-js7hj" podUID="8d63cc6a-3920-45b6-9626-45b02324e6ce"
Sep 07 11:47:22.034968 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:22.034935    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nzzj9"
Sep 07 11:47:22.035126 ip-10-0-135-173 kubenswrapper[2563]: E0907 11:47:22.035049    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nzzj9" podUID="e92172b6-ddda-4b65-81b3-987b307e272e"
Sep 07 11:47:23.034881 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:23.034844    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-js7hj"
Sep 07 11:47:23.035347 ip-10-0-135-173 kubenswrapper[2563]: E0907 11:47:23.034976    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-js7hj" podUID="8d63cc6a-3920-45b6-9626-45b02324e6ce"
Sep 07 11:47:23.035347 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:23.035022    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xf2gj"
Sep 07 11:47:23.035347 ip-10-0-135-173 kubenswrapper[2563]: E0907 11:47:23.035119    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xf2gj" podUID="3acaae8d-5930-49c6-9b9e-d286ac6a75d5"
Sep 07 11:47:24.034679 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:24.034588    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nzzj9"
Sep 07 11:47:24.034875 ip-10-0-135-173 kubenswrapper[2563]: E0907 11:47:24.034841    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nzzj9" podUID="e92172b6-ddda-4b65-81b3-987b307e272e"
Sep 07 11:47:25.034468 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:25.034432    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-js7hj"
Sep 07 11:47:25.034904 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:25.034432    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xf2gj"
Sep 07 11:47:25.034904 ip-10-0-135-173 kubenswrapper[2563]: E0907 11:47:25.034561    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-js7hj" podUID="8d63cc6a-3920-45b6-9626-45b02324e6ce"
Sep 07 11:47:25.034904 ip-10-0-135-173 kubenswrapper[2563]: E0907 11:47:25.034674    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xf2gj" podUID="3acaae8d-5930-49c6-9b9e-d286ac6a75d5"
Sep 07 11:47:25.571020 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:25.570984    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3acaae8d-5930-49c6-9b9e-d286ac6a75d5-metrics-certs\") pod \"network-metrics-daemon-xf2gj\" (UID: \"3acaae8d-5930-49c6-9b9e-d286ac6a75d5\") " pod="openshift-multus/network-metrics-daemon-xf2gj"
Sep 07 11:47:25.571220 ip-10-0-135-173 kubenswrapper[2563]: E0907 11:47:25.571124    2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 07 11:47:25.571220 ip-10-0-135-173 kubenswrapper[2563]: E0907 11:47:25.571185    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3acaae8d-5930-49c6-9b9e-d286ac6a75d5-metrics-certs podName:3acaae8d-5930-49c6-9b9e-d286ac6a75d5 nodeName:}" failed. No retries permitted until 2026-09-07 11:47:41.571168907 +0000 UTC m=+33.089900698 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3acaae8d-5930-49c6-9b9e-d286ac6a75d5-metrics-certs") pod "network-metrics-daemon-xf2gj" (UID: "3acaae8d-5930-49c6-9b9e-d286ac6a75d5") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 07 11:47:25.671450 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:25.671410    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e92172b6-ddda-4b65-81b3-987b307e272e-original-pull-secret\") pod \"global-pull-secret-syncer-nzzj9\" (UID: \"e92172b6-ddda-4b65-81b3-987b307e272e\") " pod="kube-system/global-pull-secret-syncer-nzzj9"
Sep 07 11:47:25.671623 ip-10-0-135-173 kubenswrapper[2563]: E0907 11:47:25.671531    2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 07 11:47:25.671623 ip-10-0-135-173 kubenswrapper[2563]: E0907 11:47:25.671583    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e92172b6-ddda-4b65-81b3-987b307e272e-original-pull-secret podName:e92172b6-ddda-4b65-81b3-987b307e272e nodeName:}" failed. No retries permitted until 2026-09-07 11:47:41.671569034 +0000 UTC m=+33.190300842 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e92172b6-ddda-4b65-81b3-987b307e272e-original-pull-secret") pod "global-pull-secret-syncer-nzzj9" (UID: "e92172b6-ddda-4b65-81b3-987b307e272e") : object "kube-system"/"original-pull-secret" not registered
Sep 07 11:47:25.771972 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:25.771944    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f9tp8\" (UniqueName: \"kubernetes.io/projected/8d63cc6a-3920-45b6-9626-45b02324e6ce-kube-api-access-f9tp8\") pod \"network-check-target-js7hj\" (UID: \"8d63cc6a-3920-45b6-9626-45b02324e6ce\") " pod="openshift-network-diagnostics/network-check-target-js7hj"
Sep 07 11:47:25.772158 ip-10-0-135-173 kubenswrapper[2563]: E0907 11:47:25.772107    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 07 11:47:25.772158 ip-10-0-135-173 kubenswrapper[2563]: E0907 11:47:25.772129    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 07 11:47:25.772158 ip-10-0-135-173 kubenswrapper[2563]: E0907 11:47:25.772140    2563 projected.go:196] Error preparing data for projected volume kube-api-access-f9tp8 for pod openshift-network-diagnostics/network-check-target-js7hj: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 07 11:47:25.772305 ip-10-0-135-173 kubenswrapper[2563]: E0907 11:47:25.772208    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8d63cc6a-3920-45b6-9626-45b02324e6ce-kube-api-access-f9tp8 podName:8d63cc6a-3920-45b6-9626-45b02324e6ce nodeName:}" failed. No retries permitted until 2026-09-07 11:47:41.772187947 +0000 UTC m=+33.290919740 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-f9tp8" (UniqueName: "kubernetes.io/projected/8d63cc6a-3920-45b6-9626-45b02324e6ce-kube-api-access-f9tp8") pod "network-check-target-js7hj" (UID: "8d63cc6a-3920-45b6-9626-45b02324e6ce") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 07 11:47:26.034353 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:26.034320    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nzzj9"
Sep 07 11:47:26.034556 ip-10-0-135-173 kubenswrapper[2563]: E0907 11:47:26.034460    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nzzj9" podUID="e92172b6-ddda-4b65-81b3-987b307e272e"
Sep 07 11:47:27.034908 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:27.034870    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-js7hj"
Sep 07 11:47:27.035355 ip-10-0-135-173 kubenswrapper[2563]: E0907 11:47:27.034994    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-js7hj" podUID="8d63cc6a-3920-45b6-9626-45b02324e6ce"
Sep 07 11:47:27.035355 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:27.035051    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xf2gj"
Sep 07 11:47:27.035355 ip-10-0-135-173 kubenswrapper[2563]: E0907 11:47:27.035149    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xf2gj" podUID="3acaae8d-5930-49c6-9b9e-d286ac6a75d5"
Sep 07 11:47:28.034521 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:28.034495    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nzzj9"
Sep 07 11:47:28.034645 ip-10-0-135-173 kubenswrapper[2563]: E0907 11:47:28.034624    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nzzj9" podUID="e92172b6-ddda-4b65-81b3-987b307e272e"
Sep 07 11:47:28.161357 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:28.161084    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-173.ec2.internal" event={"ID":"7d94e6a6c726d03776a8387d983ae9ef","Type":"ContainerStarted","Data":"bd44be998d8d91b02be162d804231fd2353a33a6568ea5eca7be95cb3f709eb9"}
Sep 07 11:47:29.035799 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:29.035378    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-js7hj"
Sep 07 11:47:29.035949 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:29.035470    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xf2gj"
Sep 07 11:47:29.035949 ip-10-0-135-173 kubenswrapper[2563]: E0907 11:47:29.035887    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-js7hj" podUID="8d63cc6a-3920-45b6-9626-45b02324e6ce"
Sep 07 11:47:29.036032 ip-10-0-135-173 kubenswrapper[2563]: E0907 11:47:29.035994    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xf2gj" podUID="3acaae8d-5930-49c6-9b9e-d286ac6a75d5"
Sep 07 11:47:29.167120 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:29.167083    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-swqfh" event={"ID":"b0f92581-a3a7-444e-b915-dc6cc634fe26","Type":"ContainerStarted","Data":"80b3a2c7fdc01d85e8fe28f99f3e4bdcfc2bb076a0e05e1b752393a86f2a5151"}
Sep 07 11:47:29.167745 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:29.167161    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-swqfh" event={"ID":"b0f92581-a3a7-444e-b915-dc6cc634fe26","Type":"ContainerStarted","Data":"1dcf0516ee3641f3ae1ceff250f9256d53f5bb95e2f0975f70a4b8f27b7b523e"}
Sep 07 11:47:29.167745 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:29.167205    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-swqfh" event={"ID":"b0f92581-a3a7-444e-b915-dc6cc634fe26","Type":"ContainerStarted","Data":"e0427d00ed1cdc23df94c776747f150e23048b7c91418d57f2d4290799154b2f"}
Sep 07 11:47:29.167745 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:29.167246    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-swqfh" event={"ID":"b0f92581-a3a7-444e-b915-dc6cc634fe26","Type":"ContainerStarted","Data":"588e73391773c50beb14bc4a76ec4c27a5b598ca6099f0c6637feb268a9ad0c1"}
Sep 07 11:47:29.167745 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:29.167293    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-swqfh" event={"ID":"b0f92581-a3a7-444e-b915-dc6cc634fe26","Type":"ContainerStarted","Data":"6ebea1f4f201a9623961ea7246764f8f1c009f54faed66dd1d8031f9d349042a"}
Sep 07 11:47:29.167745 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:29.167330    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-swqfh" event={"ID":"b0f92581-a3a7-444e-b915-dc6cc634fe26","Type":"ContainerStarted","Data":"d84a7a2ca5a3d6c936d09812893c2584583cb8e3b83d0e2942857084b29f1ebd"}
Sep 07 11:47:29.168654 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:29.168627    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-wrd4z" event={"ID":"b9aae943-4d29-4c1e-909f-0a1c034299f8","Type":"ContainerStarted","Data":"3a2c2f1a0229d34b523fbd77a30eceba41813579fe46d2036ff0d5953a5c3667"}
Sep 07 11:47:29.170306 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:29.170283    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-hxstl" event={"ID":"5f723277-6450-4fd8-8f5d-01d9f9f6814b","Type":"ContainerStarted","Data":"de366fa448e277e85474a7374cef165e9e700498af89954cbf65df8989baefd5"}
Sep 07 11:47:29.171893 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:29.171866    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-dl69z" event={"ID":"3e71d2e5-bc97-415d-8e01-cdf554957349","Type":"ContainerStarted","Data":"f2d8701e44b6f9a78fcd5e6d7b9bbbdbaa67e3801bd8e0b588ffe4598cca0d68"}
Sep 07 11:47:29.173523 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:29.173499    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-g2v7c" event={"ID":"64362f4d-7d1d-4faa-a02e-4a7dec54cffc","Type":"ContainerStarted","Data":"bc5f34b83af2341797b3d2aafead3b795d94a6b9e4b28c9db54043876db00ab7"}
Sep 07 11:47:29.175240 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:29.175215    2563 generic.go:356] "Generic (PLEG): container finished" podID="0baf97a2450fb410ba0091864614894a" containerID="ba2e8c22906f092079e97d60896ba113ed3ca476c1a8887056291a0a95e86293" exitCode=0
Sep 07 11:47:29.175365 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:29.175295    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-173.ec2.internal" event={"ID":"0baf97a2450fb410ba0091864614894a","Type":"ContainerDied","Data":"ba2e8c22906f092079e97d60896ba113ed3ca476c1a8887056291a0a95e86293"}
Sep 07 11:47:29.176831 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:29.176807    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-lwkzv" event={"ID":"7323d14a-1095-4021-982f-9162ee160fe5","Type":"ContainerStarted","Data":"c7df7a6dd9b5248c7d65b9eaf2c229bd6f0429fe479a3fa48b3984282507354a"}
Sep 07 11:47:29.180073 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:29.180027    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-173.ec2.internal" podStartSLOduration=20.180011736 podStartE2EDuration="20.180011736s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-07 11:47:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 11:47:28.178071222 +0000 UTC m=+19.696803039" watchObservedRunningTime="2026-09-07 11:47:29.180011736 +0000 UTC m=+20.698743552"
Sep 07 11:47:29.180660 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:29.180627    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-wrd4z" podStartSLOduration=2.487237088 podStartE2EDuration="20.180612072s" totalImagesPullingTime="17.693374984s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-07 11:47:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 11:47:29.180227283 +0000 UTC m=+20.698959097" watchObservedRunningTime="2026-09-07 11:47:29.180612072 +0000 UTC m=+20.699343886"
Sep 07 11:47:29.180913 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:29.180892    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rmjjm" event={"ID":"71bc595d-9844-4086-90fd-28341f888e30","Type":"ContainerStarted","Data":"59de8b955727e3dd84c04ea8ab81c0ed0c5ba16fba969381963ea8bf1d08f6a6"}
Sep 07 11:47:29.193817 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:29.193773    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-lwkzv" podStartSLOduration=2.497611688 podStartE2EDuration="20.193761504s" totalImagesPullingTime="17.696149816s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-07 11:47:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 11:47:29.193512482 +0000 UTC m=+20.712244295" watchObservedRunningTime="2026-09-07 11:47:29.193761504 +0000 UTC m=+20.712493316"
Sep 07 11:47:29.229445 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:29.229355    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-dl69z" podStartSLOduration=2.508423433 podStartE2EDuration="20.229339614s" totalImagesPullingTime="17.720916181s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 11:47:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 11:47:29.21719671 +0000 UTC m=+20.735928519" watchObservedRunningTime="2026-09-07 11:47:29.229339614 +0000 UTC m=+20.748071427"
Sep 07 11:47:29.229550 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:29.229480    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-hxstl" podStartSLOduration=2.664659322 podStartE2EDuration="20.229468543s" totalImagesPullingTime="17.564809221s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-07 11:47:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 11:47:29.228682796 +0000 UTC m=+20.747414610" watchObservedRunningTime="2026-09-07 11:47:29.229468543 +0000 UTC m=+20.748200356"
Sep 07 11:47:29.245358 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:29.243568    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-g2v7c" podStartSLOduration=2.425804182 podStartE2EDuration="20.243552169s" totalImagesPullingTime="17.817747987s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-07 11:47:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 11:47:29.242437555 +0000 UTC m=+20.761169366" watchObservedRunningTime="2026-09-07 11:47:29.243552169 +0000 UTC m=+20.762283984"
Sep 07 11:47:30.007910 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:30.007737    2563 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Sep 07 11:47:30.034898 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:30.034876    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nzzj9"
Sep 07 11:47:30.034996 ip-10-0-135-173 kubenswrapper[2563]: E0907 11:47:30.034964    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nzzj9" podUID="e92172b6-ddda-4b65-81b3-987b307e272e"
Sep 07 11:47:30.045173 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:30.045097    2563 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-09-07T11:47:30.007905816Z","UUID":"855e8e1d-0b4b-46fa-9b93-f893dc0f552f","Handler":null,"Name":"","Endpoint":""}
Sep 07 11:47:30.046310 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:30.046294    2563 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Sep 07 11:47:30.046384 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:30.046318    2563 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Sep 07 11:47:30.184318 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:30.184258    2563 generic.go:356] "Generic (PLEG): container finished" podID="55db3d09-7464-4f97-af5a-985162756ebb" containerID="770b13e54a86b6a40711d61f0bbcfaee091b8163a43c5737df507153567c89a3" exitCode=0
Sep 07 11:47:30.184882 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:30.184329    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mv7nw" event={"ID":"55db3d09-7464-4f97-af5a-985162756ebb","Type":"ContainerDied","Data":"770b13e54a86b6a40711d61f0bbcfaee091b8163a43c5737df507153567c89a3"}
Sep 07 11:47:30.185941 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:30.185907    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-j9blv" event={"ID":"8b7fc53e-4815-4833-a4dc-ad05cd3834c4","Type":"ContainerStarted","Data":"c071bef5c11fcec27be3d9cf3cbb26f4ceb00df81d37d39b26fe3f6679ed987e"}
Sep 07 11:47:30.187824 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:30.187804    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-173.ec2.internal" event={"ID":"0baf97a2450fb410ba0091864614894a","Type":"ContainerStarted","Data":"ad2f820830d67dbf910a7a521f31fe3f289250a9b4c468341761fe62914761ab"}
Sep 07 11:47:30.189665 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:30.189643    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rmjjm" event={"ID":"71bc595d-9844-4086-90fd-28341f888e30","Type":"ContainerStarted","Data":"80f5e53553d61338454bdd148460e75a811d9b614fedc58452a9bee3ec1b4578"}
Sep 07 11:47:30.215939 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:30.215903    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-173.ec2.internal" podStartSLOduration=21.215891189 podStartE2EDuration="21.215891189s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-07 11:47:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 11:47:30.215438464 +0000 UTC m=+21.734170276" watchObservedRunningTime="2026-09-07 11:47:30.215891189 +0000 UTC m=+21.734623002"
Sep 07 11:47:30.227044 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:30.227005    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-j9blv" podStartSLOduration=3.458101154 podStartE2EDuration="21.226994181s" totalImagesPullingTime="17.768893027s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-07 11:47:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 11:47:30.226614205 +0000 UTC m=+21.745346012" watchObservedRunningTime="2026-09-07 11:47:30.226994181 +0000 UTC m=+21.745725994"
Sep 07 11:47:30.834631 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:30.834610    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-dl69z"
Sep 07 11:47:30.837631 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:30.835553    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-dl69z"
Sep 07 11:47:31.034138 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:31.034109    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xf2gj"
Sep 07 11:47:31.034300 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:31.034116    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-js7hj"
Sep 07 11:47:31.034300 ip-10-0-135-173 kubenswrapper[2563]: E0907 11:47:31.034208    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xf2gj" podUID="3acaae8d-5930-49c6-9b9e-d286ac6a75d5"
Sep 07 11:47:31.034384 ip-10-0-135-173 kubenswrapper[2563]: E0907 11:47:31.034294    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-js7hj" podUID="8d63cc6a-3920-45b6-9626-45b02324e6ce"
Sep 07 11:47:31.195721 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:31.195674    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-swqfh" event={"ID":"b0f92581-a3a7-444e-b915-dc6cc634fe26","Type":"ContainerStarted","Data":"defa53fe1b951638427d180a29b463e74049a221f3a93c415250272f0266bea4"}
Sep 07 11:47:31.198068 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:31.197995    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rmjjm" event={"ID":"71bc595d-9844-4086-90fd-28341f888e30","Type":"ContainerStarted","Data":"75345ab0c3b2c06dcb98bbb94d616460b91a8a71d97cdad589a45522dcd94fc3"}
Sep 07 11:47:31.198249 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:31.198227    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-dl69z"
Sep 07 11:47:31.198732 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:31.198709    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-dl69z"
Sep 07 11:47:31.212471 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:31.212402    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rmjjm" podStartSLOduration=2.549651971 podStartE2EDuration="22.212389803s" totalImagesPullingTime="19.662737832s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-07 11:47:09 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 11:47:31.211988366 +0000 UTC m=+22.730720179" watchObservedRunningTime="2026-09-07 11:47:31.212389803 +0000 UTC m=+22.731121616"
Sep 07 11:47:32.034179 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:32.034145    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nzzj9"
Sep 07 11:47:32.034327 ip-10-0-135-173 kubenswrapper[2563]: E0907 11:47:32.034257    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nzzj9" podUID="e92172b6-ddda-4b65-81b3-987b307e272e"
Sep 07 11:47:33.034560 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:33.034321    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xf2gj"
Sep 07 11:47:33.034560 ip-10-0-135-173 kubenswrapper[2563]: E0907 11:47:33.034506    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xf2gj" podUID="3acaae8d-5930-49c6-9b9e-d286ac6a75d5"
Sep 07 11:47:33.035559 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:33.034900    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-js7hj"
Sep 07 11:47:33.035559 ip-10-0-135-173 kubenswrapper[2563]: E0907 11:47:33.034991    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-js7hj" podUID="8d63cc6a-3920-45b6-9626-45b02324e6ce"
Sep 07 11:47:33.207228 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:33.207191    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-swqfh" event={"ID":"b0f92581-a3a7-444e-b915-dc6cc634fe26","Type":"ContainerStarted","Data":"112f096d994b990a6243c2ce41bca2ef78d2dcd7cc3203b56c281598eefb8873"}
Sep 07 11:47:33.207684 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:33.207665    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-swqfh"
Sep 07 11:47:33.207790 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:33.207738    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-swqfh"
Sep 07 11:47:33.207851 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:33.207791    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-swqfh"
Sep 07 11:47:33.224432 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:33.224252    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-swqfh"
Sep 07 11:47:33.224619 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:33.224600    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-swqfh"
Sep 07 11:47:33.247439 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:33.246751    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-swqfh" podStartSLOduration=6.274940783 podStartE2EDuration="24.246733843s" totalImagesPullingTime="17.97179306s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-07 11:47:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 11:47:33.245306515 +0000 UTC m=+24.764038329" watchObservedRunningTime="2026-09-07 11:47:33.246733843 +0000 UTC m=+24.765465657"
Sep 07 11:47:34.034965 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:34.034931    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nzzj9"
Sep 07 11:47:34.035527 ip-10-0-135-173 kubenswrapper[2563]: E0907 11:47:34.035061    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nzzj9" podUID="e92172b6-ddda-4b65-81b3-987b307e272e"
Sep 07 11:47:34.864435 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:34.864389    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-js7hj"]
Sep 07 11:47:34.864644 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:34.864544    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-js7hj"
Sep 07 11:47:34.864716 ip-10-0-135-173 kubenswrapper[2563]: E0907 11:47:34.864655    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-js7hj" podUID="8d63cc6a-3920-45b6-9626-45b02324e6ce"
Sep 07 11:47:34.867034 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:34.867006    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-xf2gj"]
Sep 07 11:47:34.867155 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:34.867117    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xf2gj"
Sep 07 11:47:34.867213 ip-10-0-135-173 kubenswrapper[2563]: E0907 11:47:34.867196    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xf2gj" podUID="3acaae8d-5930-49c6-9b9e-d286ac6a75d5"
Sep 07 11:47:34.867517 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:34.867491    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-nzzj9"]
Sep 07 11:47:34.867626 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:34.867575    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nzzj9"
Sep 07 11:47:34.867673 ip-10-0-135-173 kubenswrapper[2563]: E0907 11:47:34.867651    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nzzj9" podUID="e92172b6-ddda-4b65-81b3-987b307e272e"
Sep 07 11:47:35.213879 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:35.213820    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mv7nw" event={"ID":"55db3d09-7464-4f97-af5a-985162756ebb","Type":"ContainerStarted","Data":"866d9238acacdf497307f09ed5063ad9bed6345a8ed2a09769860f1e3508fb68"}
Sep 07 11:47:36.034840 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:36.034805    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nzzj9"
Sep 07 11:47:36.035064 ip-10-0-135-173 kubenswrapper[2563]: E0907 11:47:36.034931    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nzzj9" podUID="e92172b6-ddda-4b65-81b3-987b307e272e"
Sep 07 11:47:36.035064 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:36.034981    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xf2gj"
Sep 07 11:47:36.035179 ip-10-0-135-173 kubenswrapper[2563]: E0907 11:47:36.035088    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xf2gj" podUID="3acaae8d-5930-49c6-9b9e-d286ac6a75d5"
Sep 07 11:47:36.218323 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:36.218289    2563 generic.go:356] "Generic (PLEG): container finished" podID="55db3d09-7464-4f97-af5a-985162756ebb" containerID="866d9238acacdf497307f09ed5063ad9bed6345a8ed2a09769860f1e3508fb68" exitCode=0
Sep 07 11:47:36.218323 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:36.218324    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mv7nw" event={"ID":"55db3d09-7464-4f97-af5a-985162756ebb","Type":"ContainerDied","Data":"866d9238acacdf497307f09ed5063ad9bed6345a8ed2a09769860f1e3508fb68"}
Sep 07 11:47:37.034399 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:37.034375    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-js7hj"
Sep 07 11:47:37.034558 ip-10-0-135-173 kubenswrapper[2563]: E0907 11:47:37.034485    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-js7hj" podUID="8d63cc6a-3920-45b6-9626-45b02324e6ce"
Sep 07 11:47:37.222355 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:37.222280    2563 generic.go:356] "Generic (PLEG): container finished" podID="55db3d09-7464-4f97-af5a-985162756ebb" containerID="b5181710f5c92c50c0569a54e83e10ece33a5ea12a985e4f4e584e9794e6b328" exitCode=0
Sep 07 11:47:37.222355 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:37.222323    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mv7nw" event={"ID":"55db3d09-7464-4f97-af5a-985162756ebb","Type":"ContainerDied","Data":"b5181710f5c92c50c0569a54e83e10ece33a5ea12a985e4f4e584e9794e6b328"}
Sep 07 11:47:38.034117 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:38.033958    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xf2gj"
Sep 07 11:47:38.034207 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:38.034001    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nzzj9"
Sep 07 11:47:38.034272 ip-10-0-135-173 kubenswrapper[2563]: E0907 11:47:38.034233    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xf2gj" podUID="3acaae8d-5930-49c6-9b9e-d286ac6a75d5"
Sep 07 11:47:38.034330 ip-10-0-135-173 kubenswrapper[2563]: E0907 11:47:38.034310    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nzzj9" podUID="e92172b6-ddda-4b65-81b3-987b307e272e"
Sep 07 11:47:38.227001 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:38.226977    2563 generic.go:356] "Generic (PLEG): container finished" podID="55db3d09-7464-4f97-af5a-985162756ebb" containerID="665199b6997581a40d44611143567a271fd5ac157e52a37ad2c5adaafcb6619e" exitCode=0
Sep 07 11:47:38.227309 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:38.227013    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mv7nw" event={"ID":"55db3d09-7464-4f97-af5a-985162756ebb","Type":"ContainerDied","Data":"665199b6997581a40d44611143567a271fd5ac157e52a37ad2c5adaafcb6619e"}
Sep 07 11:47:39.035093 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:39.035063    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-js7hj"
Sep 07 11:47:39.035304 ip-10-0-135-173 kubenswrapper[2563]: E0907 11:47:39.035159    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-js7hj" podUID="8d63cc6a-3920-45b6-9626-45b02324e6ce"
Sep 07 11:47:39.355734 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:39.355696    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 11:47:39.436306 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:39.436269    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 11:47:40.034373 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:40.034334    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nzzj9"
Sep 07 11:47:40.034373 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:40.034373    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xf2gj"
Sep 07 11:47:40.034615 ip-10-0-135-173 kubenswrapper[2563]: E0907 11:47:40.034483    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nzzj9" podUID="e92172b6-ddda-4b65-81b3-987b307e272e"
Sep 07 11:47:40.034777 ip-10-0-135-173 kubenswrapper[2563]: E0907 11:47:40.034748    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xf2gj" podUID="3acaae8d-5930-49c6-9b9e-d286ac6a75d5"
Sep 07 11:47:41.034914 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:41.034839    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-js7hj"
Sep 07 11:47:41.035333 ip-10-0-135-173 kubenswrapper[2563]: E0907 11:47:41.034950    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-js7hj" podUID="8d63cc6a-3920-45b6-9626-45b02324e6ce"
Sep 07 11:47:41.295126 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:41.295057    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-173.ec2.internal" event="NodeReady"
Sep 07 11:47:41.295283 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:41.295178    2563 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Sep 07 11:47:41.335863 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:41.335827    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-q9tkc"]
Sep 07 11:47:41.338638 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:41.338611    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-nv8dv"]
Sep 07 11:47:41.338783 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:41.338745    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-q9tkc"
Sep 07 11:47:41.340400 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:41.340371    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Sep 07 11:47:41.340539 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:41.340442    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-kd29r\""
Sep 07 11:47:41.340539 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:41.340467    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-nv8dv"
Sep 07 11:47:41.340651 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:41.340562    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Sep 07 11:47:41.342103 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:41.342075    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Sep 07 11:47:41.342199 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:41.342189    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-7tkzl\""
Sep 07 11:47:41.342777 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:41.342756    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Sep 07 11:47:41.342862 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:41.342835    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Sep 07 11:47:41.348764 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:41.348731    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-nv8dv"]
Sep 07 11:47:41.351692 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:41.351671    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-q9tkc"]
Sep 07 11:47:41.458341 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:41.458311    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 11:47:41.480795 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:41.480495    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/46cab3bd-0f24-46e2-910c-b6e9a5f6ae5e-metrics-tls\") pod \"dns-default-q9tkc\" (UID: \"46cab3bd-0f24-46e2-910c-b6e9a5f6ae5e\") " pod="openshift-dns/dns-default-q9tkc"
Sep 07 11:47:41.480795 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:41.480571    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/46cab3bd-0f24-46e2-910c-b6e9a5f6ae5e-tmp-dir\") pod \"dns-default-q9tkc\" (UID: \"46cab3bd-0f24-46e2-910c-b6e9a5f6ae5e\") " pod="openshift-dns/dns-default-q9tkc"
Sep 07 11:47:41.480795 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:41.480658    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4h72n\" (UniqueName: \"kubernetes.io/projected/46cab3bd-0f24-46e2-910c-b6e9a5f6ae5e-kube-api-access-4h72n\") pod \"dns-default-q9tkc\" (UID: \"46cab3bd-0f24-46e2-910c-b6e9a5f6ae5e\") " pod="openshift-dns/dns-default-q9tkc"
Sep 07 11:47:41.480795 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:41.480692    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/46cab3bd-0f24-46e2-910c-b6e9a5f6ae5e-config-volume\") pod \"dns-default-q9tkc\" (UID: \"46cab3bd-0f24-46e2-910c-b6e9a5f6ae5e\") " pod="openshift-dns/dns-default-q9tkc"
Sep 07 11:47:41.480795 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:41.480715    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d1606d20-6381-462c-9cbe-b65edc598e2f-cert\") pod \"ingress-canary-nv8dv\" (UID: \"d1606d20-6381-462c-9cbe-b65edc598e2f\") " pod="openshift-ingress-canary/ingress-canary-nv8dv"
Sep 07 11:47:41.480795 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:41.480745    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dz4b5\" (UniqueName: \"kubernetes.io/projected/d1606d20-6381-462c-9cbe-b65edc598e2f-kube-api-access-dz4b5\") pod \"ingress-canary-nv8dv\" (UID: \"d1606d20-6381-462c-9cbe-b65edc598e2f\") " pod="openshift-ingress-canary/ingress-canary-nv8dv"
Sep 07 11:47:41.512228 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:41.512190    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 11:47:41.581907 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:41.581842    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/46cab3bd-0f24-46e2-910c-b6e9a5f6ae5e-tmp-dir\") pod \"dns-default-q9tkc\" (UID: \"46cab3bd-0f24-46e2-910c-b6e9a5f6ae5e\") " pod="openshift-dns/dns-default-q9tkc"
Sep 07 11:47:41.581907 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:41.581882    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4h72n\" (UniqueName: \"kubernetes.io/projected/46cab3bd-0f24-46e2-910c-b6e9a5f6ae5e-kube-api-access-4h72n\") pod \"dns-default-q9tkc\" (UID: \"46cab3bd-0f24-46e2-910c-b6e9a5f6ae5e\") " pod="openshift-dns/dns-default-q9tkc"
Sep 07 11:47:41.582114 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:41.581912    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/46cab3bd-0f24-46e2-910c-b6e9a5f6ae5e-config-volume\") pod \"dns-default-q9tkc\" (UID: \"46cab3bd-0f24-46e2-910c-b6e9a5f6ae5e\") " pod="openshift-dns/dns-default-q9tkc"
Sep 07 11:47:41.582114 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:41.581999    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d1606d20-6381-462c-9cbe-b65edc598e2f-cert\") pod \"ingress-canary-nv8dv\" (UID: \"d1606d20-6381-462c-9cbe-b65edc598e2f\") " pod="openshift-ingress-canary/ingress-canary-nv8dv"
Sep 07 11:47:41.582114 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:41.582040    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dz4b5\" (UniqueName: \"kubernetes.io/projected/d1606d20-6381-462c-9cbe-b65edc598e2f-kube-api-access-dz4b5\") pod \"ingress-canary-nv8dv\" (UID: \"d1606d20-6381-462c-9cbe-b65edc598e2f\") " pod="openshift-ingress-canary/ingress-canary-nv8dv"
Sep 07 11:47:41.582114 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:41.582106    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3acaae8d-5930-49c6-9b9e-d286ac6a75d5-metrics-certs\") pod \"network-metrics-daemon-xf2gj\" (UID: \"3acaae8d-5930-49c6-9b9e-d286ac6a75d5\") " pod="openshift-multus/network-metrics-daemon-xf2gj"
Sep 07 11:47:41.582308 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:41.582133    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/46cab3bd-0f24-46e2-910c-b6e9a5f6ae5e-metrics-tls\") pod \"dns-default-q9tkc\" (UID: \"46cab3bd-0f24-46e2-910c-b6e9a5f6ae5e\") " pod="openshift-dns/dns-default-q9tkc"
Sep 07 11:47:41.582308 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:41.582221    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/46cab3bd-0f24-46e2-910c-b6e9a5f6ae5e-tmp-dir\") pod \"dns-default-q9tkc\" (UID: \"46cab3bd-0f24-46e2-910c-b6e9a5f6ae5e\") " pod="openshift-dns/dns-default-q9tkc"
Sep 07 11:47:41.582308 ip-10-0-135-173 kubenswrapper[2563]: E0907 11:47:41.582254    2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 07 11:47:41.582308 ip-10-0-135-173 kubenswrapper[2563]: E0907 11:47:41.582280    2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 07 11:47:41.582491 ip-10-0-135-173 kubenswrapper[2563]: E0907 11:47:41.582318    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3acaae8d-5930-49c6-9b9e-d286ac6a75d5-metrics-certs podName:3acaae8d-5930-49c6-9b9e-d286ac6a75d5 nodeName:}" failed. No retries permitted until 2026-09-07 11:48:13.582299812 +0000 UTC m=+65.101031622 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3acaae8d-5930-49c6-9b9e-d286ac6a75d5-metrics-certs") pod "network-metrics-daemon-xf2gj" (UID: "3acaae8d-5930-49c6-9b9e-d286ac6a75d5") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 07 11:47:41.582491 ip-10-0-135-173 kubenswrapper[2563]: E0907 11:47:41.582376    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/46cab3bd-0f24-46e2-910c-b6e9a5f6ae5e-metrics-tls podName:46cab3bd-0f24-46e2-910c-b6e9a5f6ae5e nodeName:}" failed. No retries permitted until 2026-09-07 11:47:42.082358279 +0000 UTC m=+33.601090070 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/46cab3bd-0f24-46e2-910c-b6e9a5f6ae5e-metrics-tls") pod "dns-default-q9tkc" (UID: "46cab3bd-0f24-46e2-910c-b6e9a5f6ae5e") : secret "dns-default-metrics-tls" not found
Sep 07 11:47:41.582820 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:41.582798    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/46cab3bd-0f24-46e2-910c-b6e9a5f6ae5e-config-volume\") pod \"dns-default-q9tkc\" (UID: \"46cab3bd-0f24-46e2-910c-b6e9a5f6ae5e\") " pod="openshift-dns/dns-default-q9tkc"
Sep 07 11:47:41.586884 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:41.586837    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d1606d20-6381-462c-9cbe-b65edc598e2f-cert\") pod \"ingress-canary-nv8dv\" (UID: \"d1606d20-6381-462c-9cbe-b65edc598e2f\") " pod="openshift-ingress-canary/ingress-canary-nv8dv"
Sep 07 11:47:41.589685 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:41.589642    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4h72n\" (UniqueName: \"kubernetes.io/projected/46cab3bd-0f24-46e2-910c-b6e9a5f6ae5e-kube-api-access-4h72n\") pod \"dns-default-q9tkc\" (UID: \"46cab3bd-0f24-46e2-910c-b6e9a5f6ae5e\") " pod="openshift-dns/dns-default-q9tkc"
Sep 07 11:47:41.589799 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:41.589776    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dz4b5\" (UniqueName: \"kubernetes.io/projected/d1606d20-6381-462c-9cbe-b65edc598e2f-kube-api-access-dz4b5\") pod \"ingress-canary-nv8dv\" (UID: \"d1606d20-6381-462c-9cbe-b65edc598e2f\") " pod="openshift-ingress-canary/ingress-canary-nv8dv"
Sep 07 11:47:41.658565 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:41.658534    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-nv8dv"
Sep 07 11:47:41.683370 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:41.683343    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e92172b6-ddda-4b65-81b3-987b307e272e-original-pull-secret\") pod \"global-pull-secret-syncer-nzzj9\" (UID: \"e92172b6-ddda-4b65-81b3-987b307e272e\") " pod="kube-system/global-pull-secret-syncer-nzzj9"
Sep 07 11:47:41.683480 ip-10-0-135-173 kubenswrapper[2563]: E0907 11:47:41.683460    2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 07 11:47:41.683542 ip-10-0-135-173 kubenswrapper[2563]: E0907 11:47:41.683533    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e92172b6-ddda-4b65-81b3-987b307e272e-original-pull-secret podName:e92172b6-ddda-4b65-81b3-987b307e272e nodeName:}" failed. No retries permitted until 2026-09-07 11:48:13.683516457 +0000 UTC m=+65.202248250 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e92172b6-ddda-4b65-81b3-987b307e272e-original-pull-secret") pod "global-pull-secret-syncer-nzzj9" (UID: "e92172b6-ddda-4b65-81b3-987b307e272e") : object "kube-system"/"original-pull-secret" not registered
Sep 07 11:47:41.784105 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:41.784065    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f9tp8\" (UniqueName: \"kubernetes.io/projected/8d63cc6a-3920-45b6-9626-45b02324e6ce-kube-api-access-f9tp8\") pod \"network-check-target-js7hj\" (UID: \"8d63cc6a-3920-45b6-9626-45b02324e6ce\") " pod="openshift-network-diagnostics/network-check-target-js7hj"
Sep 07 11:47:41.784263 ip-10-0-135-173 kubenswrapper[2563]: E0907 11:47:41.784248    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 07 11:47:41.784323 ip-10-0-135-173 kubenswrapper[2563]: E0907 11:47:41.784276    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 07 11:47:41.784323 ip-10-0-135-173 kubenswrapper[2563]: E0907 11:47:41.784290    2563 projected.go:196] Error preparing data for projected volume kube-api-access-f9tp8 for pod openshift-network-diagnostics/network-check-target-js7hj: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 07 11:47:41.784395 ip-10-0-135-173 kubenswrapper[2563]: E0907 11:47:41.784370    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8d63cc6a-3920-45b6-9626-45b02324e6ce-kube-api-access-f9tp8 podName:8d63cc6a-3920-45b6-9626-45b02324e6ce nodeName:}" failed. No retries permitted until 2026-09-07 11:48:13.784347901 +0000 UTC m=+65.303079692 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-f9tp8" (UniqueName: "kubernetes.io/projected/8d63cc6a-3920-45b6-9626-45b02324e6ce-kube-api-access-f9tp8") pod "network-check-target-js7hj" (UID: "8d63cc6a-3920-45b6-9626-45b02324e6ce") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 07 11:47:42.035094 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:42.034923    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nzzj9"
Sep 07 11:47:42.035524 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:42.034975    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xf2gj"
Sep 07 11:47:42.037012 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:42.036986    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 07 11:47:42.037153 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:42.037015    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-gtblx\""
Sep 07 11:47:42.037153 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:42.037039    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 07 11:47:42.085825 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:42.085792    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/46cab3bd-0f24-46e2-910c-b6e9a5f6ae5e-metrics-tls\") pod \"dns-default-q9tkc\" (UID: \"46cab3bd-0f24-46e2-910c-b6e9a5f6ae5e\") " pod="openshift-dns/dns-default-q9tkc"
Sep 07 11:47:42.085981 ip-10-0-135-173 kubenswrapper[2563]: E0907 11:47:42.085958    2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 07 11:47:42.086062 ip-10-0-135-173 kubenswrapper[2563]: E0907 11:47:42.086049    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/46cab3bd-0f24-46e2-910c-b6e9a5f6ae5e-metrics-tls podName:46cab3bd-0f24-46e2-910c-b6e9a5f6ae5e nodeName:}" failed. No retries permitted until 2026-09-07 11:47:43.086025184 +0000 UTC m=+34.604756995 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/46cab3bd-0f24-46e2-910c-b6e9a5f6ae5e-metrics-tls") pod "dns-default-q9tkc" (UID: "46cab3bd-0f24-46e2-910c-b6e9a5f6ae5e") : secret "dns-default-metrics-tls" not found
Sep 07 11:47:42.240911 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:42.240883    2563 generic.go:356] "Generic (PLEG): container finished" podID="55db3d09-7464-4f97-af5a-985162756ebb" containerID="e5da83527fbc153f1f085a881b71399ec0e0d9f7e12f0014185165499a3e4c73" exitCode=0
Sep 07 11:47:42.241024 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:42.240941    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mv7nw" event={"ID":"55db3d09-7464-4f97-af5a-985162756ebb","Type":"ContainerDied","Data":"e5da83527fbc153f1f085a881b71399ec0e0d9f7e12f0014185165499a3e4c73"}
Sep 07 11:47:42.272358 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:42.272284    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-nv8dv"]
Sep 07 11:47:42.277965 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:47:42.277937    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd1606d20_6381_462c_9cbe_b65edc598e2f.slice/crio-3448ecaa39d27c9abbdd430aa8999f78dd08296c678e911cfb1046e5f3f4519e WatchSource:0}: Error finding container 3448ecaa39d27c9abbdd430aa8999f78dd08296c678e911cfb1046e5f3f4519e: Status 404 returned error can't find the container with id 3448ecaa39d27c9abbdd430aa8999f78dd08296c678e911cfb1046e5f3f4519e
Sep 07 11:47:43.034398 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:43.034356    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-js7hj"
Sep 07 11:47:43.036566 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:43.036292    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 07 11:47:43.036566 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:43.036444    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 07 11:47:43.036566 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:43.036459    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-2v7c7\""
Sep 07 11:47:43.093830 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:43.093802    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/46cab3bd-0f24-46e2-910c-b6e9a5f6ae5e-metrics-tls\") pod \"dns-default-q9tkc\" (UID: \"46cab3bd-0f24-46e2-910c-b6e9a5f6ae5e\") " pod="openshift-dns/dns-default-q9tkc"
Sep 07 11:47:43.093936 ip-10-0-135-173 kubenswrapper[2563]: E0907 11:47:43.093909    2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 07 11:47:43.093985 ip-10-0-135-173 kubenswrapper[2563]: E0907 11:47:43.093964    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/46cab3bd-0f24-46e2-910c-b6e9a5f6ae5e-metrics-tls podName:46cab3bd-0f24-46e2-910c-b6e9a5f6ae5e nodeName:}" failed. No retries permitted until 2026-09-07 11:47:45.093949179 +0000 UTC m=+36.612680971 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/46cab3bd-0f24-46e2-910c-b6e9a5f6ae5e-metrics-tls") pod "dns-default-q9tkc" (UID: "46cab3bd-0f24-46e2-910c-b6e9a5f6ae5e") : secret "dns-default-metrics-tls" not found
Sep 07 11:47:43.244672 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:43.244629    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-nv8dv" event={"ID":"d1606d20-6381-462c-9cbe-b65edc598e2f","Type":"ContainerStarted","Data":"3448ecaa39d27c9abbdd430aa8999f78dd08296c678e911cfb1046e5f3f4519e"}
Sep 07 11:47:43.248280 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:43.248252    2563 generic.go:356] "Generic (PLEG): container finished" podID="55db3d09-7464-4f97-af5a-985162756ebb" containerID="3750c8bdfea0678b0852ed49ea6b80b398e6ddf1ade399ff8cdb76e2a63075b6" exitCode=0
Sep 07 11:47:43.248486 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:43.248306    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mv7nw" event={"ID":"55db3d09-7464-4f97-af5a-985162756ebb","Type":"ContainerDied","Data":"3750c8bdfea0678b0852ed49ea6b80b398e6ddf1ade399ff8cdb76e2a63075b6"}
Sep 07 11:47:43.693337 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:43.693242    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 11:47:43.794476 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:43.794443    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 11:47:44.254890 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:44.254851    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mv7nw" event={"ID":"55db3d09-7464-4f97-af5a-985162756ebb","Type":"ContainerStarted","Data":"fe634e99221e971dae75cb94057f9b0f88ee23b3f669885cc85a205a9fec3226"}
Sep 07 11:47:44.273714 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:44.273500    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-mv7nw" podStartSLOduration=3.5014384 podStartE2EDuration="35.273480889s" totalImagesPullingTime="31.772042489s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-07 11:47:09 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=1 observedRunningTime="2026-09-07 11:47:44.273151947 +0000 UTC m=+35.791883787" watchObservedRunningTime="2026-09-07 11:47:44.273480889 +0000 UTC m=+35.792212703"
Sep 07 11:47:44.402242 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:44.402206    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 11:47:44.492791 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:44.492760    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 11:47:45.110498 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:45.110303    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/46cab3bd-0f24-46e2-910c-b6e9a5f6ae5e-metrics-tls\") pod \"dns-default-q9tkc\" (UID: \"46cab3bd-0f24-46e2-910c-b6e9a5f6ae5e\") " pod="openshift-dns/dns-default-q9tkc"
Sep 07 11:47:45.110625 ip-10-0-135-173 kubenswrapper[2563]: E0907 11:47:45.110462    2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 07 11:47:45.110625 ip-10-0-135-173 kubenswrapper[2563]: E0907 11:47:45.110564    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/46cab3bd-0f24-46e2-910c-b6e9a5f6ae5e-metrics-tls podName:46cab3bd-0f24-46e2-910c-b6e9a5f6ae5e nodeName:}" failed. No retries permitted until 2026-09-07 11:47:49.110549924 +0000 UTC m=+40.629281729 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/46cab3bd-0f24-46e2-910c-b6e9a5f6ae5e-metrics-tls") pod "dns-default-q9tkc" (UID: "46cab3bd-0f24-46e2-910c-b6e9a5f6ae5e") : secret "dns-default-metrics-tls" not found
Sep 07 11:47:45.258063 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:45.257987    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-nv8dv" event={"ID":"d1606d20-6381-462c-9cbe-b65edc598e2f","Type":"ContainerStarted","Data":"bd606e326520747c847f18b895d78dddca2238aaa827b49f5e6ae8b78843ff45"}
Sep 07 11:47:45.271889 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:45.271844    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-nv8dv" podStartSLOduration=1.957095571 podStartE2EDuration="4.271829927s" totalImagesPullingTime="2.314734356s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 11:47:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 11:47:45.270954376 +0000 UTC m=+36.789686189" watchObservedRunningTime="2026-09-07 11:47:45.271829927 +0000 UTC m=+36.790561743"
Sep 07 11:47:46.097800 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:46.097769    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 11:47:47.286991 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:47.286953    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 11:47:47.339901 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:47.339867    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 11:47:47.441364 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:47.441337    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 11:47:48.785658 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:48.785621    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 11:47:49.138564 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:49.138484    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/46cab3bd-0f24-46e2-910c-b6e9a5f6ae5e-metrics-tls\") pod \"dns-default-q9tkc\" (UID: \"46cab3bd-0f24-46e2-910c-b6e9a5f6ae5e\") " pod="openshift-dns/dns-default-q9tkc"
Sep 07 11:47:49.138697 ip-10-0-135-173 kubenswrapper[2563]: E0907 11:47:49.138595    2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 07 11:47:49.138697 ip-10-0-135-173 kubenswrapper[2563]: E0907 11:47:49.138642    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/46cab3bd-0f24-46e2-910c-b6e9a5f6ae5e-metrics-tls podName:46cab3bd-0f24-46e2-910c-b6e9a5f6ae5e nodeName:}" failed. No retries permitted until 2026-09-07 11:47:57.138629514 +0000 UTC m=+48.657361305 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/46cab3bd-0f24-46e2-910c-b6e9a5f6ae5e-metrics-tls") pod "dns-default-q9tkc" (UID: "46cab3bd-0f24-46e2-910c-b6e9a5f6ae5e") : secret "dns-default-metrics-tls" not found
Sep 07 11:47:49.340023 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:49.339988    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 11:47:49.394897 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:49.394830    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 11:47:51.890182 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:51.890152    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 11:47:51.939945 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:51.939911    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 11:47:57.194714 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:47:57.194678    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/46cab3bd-0f24-46e2-910c-b6e9a5f6ae5e-metrics-tls\") pod \"dns-default-q9tkc\" (UID: \"46cab3bd-0f24-46e2-910c-b6e9a5f6ae5e\") " pod="openshift-dns/dns-default-q9tkc"
Sep 07 11:47:57.195090 ip-10-0-135-173 kubenswrapper[2563]: E0907 11:47:57.194793    2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 07 11:47:57.195090 ip-10-0-135-173 kubenswrapper[2563]: E0907 11:47:57.194845    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/46cab3bd-0f24-46e2-910c-b6e9a5f6ae5e-metrics-tls podName:46cab3bd-0f24-46e2-910c-b6e9a5f6ae5e nodeName:}" failed. No retries permitted until 2026-09-07 11:48:13.194832574 +0000 UTC m=+64.713564365 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/46cab3bd-0f24-46e2-910c-b6e9a5f6ae5e-metrics-tls") pod "dns-default-q9tkc" (UID: "46cab3bd-0f24-46e2-910c-b6e9a5f6ae5e") : secret "dns-default-metrics-tls" not found
Sep 07 11:48:00.416733 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:48:00.416698    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 11:48:01.114783 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:48:01.114745    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 11:48:03.426604 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:48:03.426563    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 11:48:03.478925 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:48:03.478898    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 11:48:04.043975 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:48:04.043936    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 11:48:04.145064 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:48:04.145027    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 11:48:04.671887 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:48:04.671850    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 11:48:04.772975 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:48:04.772945    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 11:48:05.231230 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:48:05.231204    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-swqfh"
Sep 07 11:48:05.375537 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:48:05.375508    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 11:48:05.434698 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:48:05.434668    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 11:48:13.200141 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:48:13.200108    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/46cab3bd-0f24-46e2-910c-b6e9a5f6ae5e-metrics-tls\") pod \"dns-default-q9tkc\" (UID: \"46cab3bd-0f24-46e2-910c-b6e9a5f6ae5e\") " pod="openshift-dns/dns-default-q9tkc"
Sep 07 11:48:13.200626 ip-10-0-135-173 kubenswrapper[2563]: E0907 11:48:13.200264    2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 07 11:48:13.200626 ip-10-0-135-173 kubenswrapper[2563]: E0907 11:48:13.200338    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/46cab3bd-0f24-46e2-910c-b6e9a5f6ae5e-metrics-tls podName:46cab3bd-0f24-46e2-910c-b6e9a5f6ae5e nodeName:}" failed. No retries permitted until 2026-09-07 11:48:45.200321681 +0000 UTC m=+96.719053472 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/46cab3bd-0f24-46e2-910c-b6e9a5f6ae5e-metrics-tls") pod "dns-default-q9tkc" (UID: "46cab3bd-0f24-46e2-910c-b6e9a5f6ae5e") : secret "dns-default-metrics-tls" not found
Sep 07 11:48:13.601675 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:48:13.601593    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3acaae8d-5930-49c6-9b9e-d286ac6a75d5-metrics-certs\") pod \"network-metrics-daemon-xf2gj\" (UID: \"3acaae8d-5930-49c6-9b9e-d286ac6a75d5\") " pod="openshift-multus/network-metrics-daemon-xf2gj"
Sep 07 11:48:13.603348 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:48:13.603328    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 07 11:48:13.612662 ip-10-0-135-173 kubenswrapper[2563]: E0907 11:48:13.612645    2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Sep 07 11:48:13.612735 ip-10-0-135-173 kubenswrapper[2563]: E0907 11:48:13.612696    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3acaae8d-5930-49c6-9b9e-d286ac6a75d5-metrics-certs podName:3acaae8d-5930-49c6-9b9e-d286ac6a75d5 nodeName:}" failed. No retries permitted until 2026-09-07 11:49:17.61268152 +0000 UTC m=+129.131413310 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3acaae8d-5930-49c6-9b9e-d286ac6a75d5-metrics-certs") pod "network-metrics-daemon-xf2gj" (UID: "3acaae8d-5930-49c6-9b9e-d286ac6a75d5") : secret "metrics-daemon-secret" not found
Sep 07 11:48:13.702130 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:48:13.702097    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e92172b6-ddda-4b65-81b3-987b307e272e-original-pull-secret\") pod \"global-pull-secret-syncer-nzzj9\" (UID: \"e92172b6-ddda-4b65-81b3-987b307e272e\") " pod="kube-system/global-pull-secret-syncer-nzzj9"
Sep 07 11:48:13.703858 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:48:13.703841    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 07 11:48:13.716253 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:48:13.716230    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e92172b6-ddda-4b65-81b3-987b307e272e-original-pull-secret\") pod \"global-pull-secret-syncer-nzzj9\" (UID: \"e92172b6-ddda-4b65-81b3-987b307e272e\") " pod="kube-system/global-pull-secret-syncer-nzzj9"
Sep 07 11:48:13.803292 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:48:13.803267    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f9tp8\" (UniqueName: \"kubernetes.io/projected/8d63cc6a-3920-45b6-9626-45b02324e6ce-kube-api-access-f9tp8\") pod \"network-check-target-js7hj\" (UID: \"8d63cc6a-3920-45b6-9626-45b02324e6ce\") " pod="openshift-network-diagnostics/network-check-target-js7hj"
Sep 07 11:48:13.804921 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:48:13.804904    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 07 11:48:13.815608 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:48:13.815586    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 07 11:48:13.827052 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:48:13.827032    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f9tp8\" (UniqueName: \"kubernetes.io/projected/8d63cc6a-3920-45b6-9626-45b02324e6ce-kube-api-access-f9tp8\") pod \"network-check-target-js7hj\" (UID: \"8d63cc6a-3920-45b6-9626-45b02324e6ce\") " pod="openshift-network-diagnostics/network-check-target-js7hj"
Sep 07 11:48:13.846953 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:48:13.846933    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nzzj9"
Sep 07 11:48:13.947232 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:48:13.947204    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-2v7c7\""
Sep 07 11:48:13.955541 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:48:13.955525    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-js7hj"
Sep 07 11:48:14.111464 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:48:14.111383    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-nzzj9"]
Sep 07 11:48:14.116518 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:48:14.116491    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode92172b6_ddda_4b65_81b3_987b307e272e.slice/crio-4966e2e2b4657e5927252aa5c3a36e8ef3c7cbb12976da7e9f86aaece91909b3 WatchSource:0}: Error finding container 4966e2e2b4657e5927252aa5c3a36e8ef3c7cbb12976da7e9f86aaece91909b3: Status 404 returned error can't find the container with id 4966e2e2b4657e5927252aa5c3a36e8ef3c7cbb12976da7e9f86aaece91909b3
Sep 07 11:48:14.215887 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:48:14.215860    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-js7hj"]
Sep 07 11:48:14.219005 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:48:14.218982    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8d63cc6a_3920_45b6_9626_45b02324e6ce.slice/crio-24756c413047b8628aeaa547f2c97befc25010cbb787aac70e6464bb460453ae WatchSource:0}: Error finding container 24756c413047b8628aeaa547f2c97befc25010cbb787aac70e6464bb460453ae: Status 404 returned error can't find the container with id 24756c413047b8628aeaa547f2c97befc25010cbb787aac70e6464bb460453ae
Sep 07 11:48:14.321057 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:48:14.321027    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-js7hj" event={"ID":"8d63cc6a-3920-45b6-9626-45b02324e6ce","Type":"ContainerStarted","Data":"24756c413047b8628aeaa547f2c97befc25010cbb787aac70e6464bb460453ae"}
Sep 07 11:48:14.322154 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:48:14.322136    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-nzzj9" event={"ID":"e92172b6-ddda-4b65-81b3-987b307e272e","Type":"ContainerStarted","Data":"4966e2e2b4657e5927252aa5c3a36e8ef3c7cbb12976da7e9f86aaece91909b3"}
Sep 07 11:48:19.336837 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:48:19.336800    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-js7hj" event={"ID":"8d63cc6a-3920-45b6-9626-45b02324e6ce","Type":"ContainerStarted","Data":"03b0a98e6c1724bc2da9d04ab3f4d157d239ddb745b04dda184626b128b07680"}
Sep 07 11:48:19.337317 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:48:19.336889    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-js7hj"
Sep 07 11:48:19.338197 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:48:19.338176    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-nzzj9" event={"ID":"e92172b6-ddda-4b65-81b3-987b307e272e","Type":"ContainerStarted","Data":"8bab2d3fb514c9ec914c0e36366c33be9a7ae86c4af9093b1d437f20585cb726"}
Sep 07 11:48:19.350346 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:48:19.350283    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-js7hj" podStartSLOduration=65.860750896 podStartE2EDuration="1m10.350271572s" totalImagesPullingTime="4.489520676s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 11:47:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 11:48:19.350042164 +0000 UTC m=+70.868773977" watchObservedRunningTime="2026-09-07 11:48:19.350271572 +0000 UTC m=+70.869003401"
Sep 07 11:48:19.362104 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:48:19.362059    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-nzzj9" podStartSLOduration=65.764432559 podStartE2EDuration="1m10.362046409s" totalImagesPullingTime="4.59761385s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-07 11:47:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 11:48:19.361465941 +0000 UTC m=+70.880197752" watchObservedRunningTime="2026-09-07 11:48:19.362046409 +0000 UTC m=+70.880778222"
Sep 07 11:48:26.371063 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:48:26.371022    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 11:48:26.475516 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:48:26.475483    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 11:48:37.805277 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:48:37.805168    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-ccc4f9c94-vkr67"]
Sep 07 11:48:37.808004 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:48:37.807975    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-ccc4f9c94-vkr67"
Sep 07 11:48:37.809826 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:48:37.809802    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Sep 07 11:48:37.809953 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:48:37.809880    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Sep 07 11:48:37.810043 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:48:37.810028    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Sep 07 11:48:37.810111 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:48:37.810031    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-w6958\""
Sep 07 11:48:37.810111 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:48:37.810051    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\""
Sep 07 11:48:37.817294 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:48:37.817274    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-ccc4f9c94-vkr67"]
Sep 07 11:48:37.962015 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:48:37.961987    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/e39e0070-19a6-4159-be9e-6beff874f190-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-ccc4f9c94-vkr67\" (UID: \"e39e0070-19a6-4159-be9e-6beff874f190\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-ccc4f9c94-vkr67"
Sep 07 11:48:37.962160 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:48:37.962018    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kjcbl\" (UniqueName: \"kubernetes.io/projected/e39e0070-19a6-4159-be9e-6beff874f190-kube-api-access-kjcbl\") pod \"managed-serviceaccount-addon-agent-ccc4f9c94-vkr67\" (UID: \"e39e0070-19a6-4159-be9e-6beff874f190\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-ccc4f9c94-vkr67"
Sep 07 11:48:38.062632 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:48:38.062560    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/e39e0070-19a6-4159-be9e-6beff874f190-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-ccc4f9c94-vkr67\" (UID: \"e39e0070-19a6-4159-be9e-6beff874f190\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-ccc4f9c94-vkr67"
Sep 07 11:48:38.062632 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:48:38.062602    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kjcbl\" (UniqueName: \"kubernetes.io/projected/e39e0070-19a6-4159-be9e-6beff874f190-kube-api-access-kjcbl\") pod \"managed-serviceaccount-addon-agent-ccc4f9c94-vkr67\" (UID: \"e39e0070-19a6-4159-be9e-6beff874f190\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-ccc4f9c94-vkr67"
Sep 07 11:48:38.064946 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:48:38.064929    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/e39e0070-19a6-4159-be9e-6beff874f190-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-ccc4f9c94-vkr67\" (UID: \"e39e0070-19a6-4159-be9e-6beff874f190\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-ccc4f9c94-vkr67"
Sep 07 11:48:38.070525 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:48:38.070506    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kjcbl\" (UniqueName: \"kubernetes.io/projected/e39e0070-19a6-4159-be9e-6beff874f190-kube-api-access-kjcbl\") pod \"managed-serviceaccount-addon-agent-ccc4f9c94-vkr67\" (UID: \"e39e0070-19a6-4159-be9e-6beff874f190\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-ccc4f9c94-vkr67"
Sep 07 11:48:38.122000 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:48:38.121981    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-ccc4f9c94-vkr67"
Sep 07 11:48:38.385003 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:48:38.384932    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-ccc4f9c94-vkr67"]
Sep 07 11:48:38.388737 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:48:38.388706    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode39e0070_19a6_4159_be9e_6beff874f190.slice/crio-bb8fb60eae1528049abd94c70585e14ea878b809e9fe50e8cf8257fbde22adeb WatchSource:0}: Error finding container bb8fb60eae1528049abd94c70585e14ea878b809e9fe50e8cf8257fbde22adeb: Status 404 returned error can't find the container with id bb8fb60eae1528049abd94c70585e14ea878b809e9fe50e8cf8257fbde22adeb
Sep 07 11:48:39.388005 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:48:39.387966    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-ccc4f9c94-vkr67" event={"ID":"e39e0070-19a6-4159-be9e-6beff874f190","Type":"ContainerStarted","Data":"bb8fb60eae1528049abd94c70585e14ea878b809e9fe50e8cf8257fbde22adeb"}
Sep 07 11:48:41.394361 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:48:41.394324    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-ccc4f9c94-vkr67" event={"ID":"e39e0070-19a6-4159-be9e-6beff874f190","Type":"ContainerStarted","Data":"0b8fcf5aa9575737f83d0dacfd35853bc11d5b205a2047b32de2c11cd412d1b3"}
Sep 07 11:48:41.407054 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:48:41.407010    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-ccc4f9c94-vkr67" podStartSLOduration=2.049016053 podStartE2EDuration="4.406996418s" totalImagesPullingTime="2.357980365s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 11:48:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 11:48:41.406180927 +0000 UTC m=+92.924912752" watchObservedRunningTime="2026-09-07 11:48:41.406996418 +0000 UTC m=+92.925728231"
Sep 07 11:48:45.205714 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:48:45.205677    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/46cab3bd-0f24-46e2-910c-b6e9a5f6ae5e-metrics-tls\") pod \"dns-default-q9tkc\" (UID: \"46cab3bd-0f24-46e2-910c-b6e9a5f6ae5e\") " pod="openshift-dns/dns-default-q9tkc"
Sep 07 11:48:45.206113 ip-10-0-135-173 kubenswrapper[2563]: E0907 11:48:45.205788    2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 07 11:48:45.206113 ip-10-0-135-173 kubenswrapper[2563]: E0907 11:48:45.205850    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/46cab3bd-0f24-46e2-910c-b6e9a5f6ae5e-metrics-tls podName:46cab3bd-0f24-46e2-910c-b6e9a5f6ae5e nodeName:}" failed. No retries permitted until 2026-09-07 11:49:49.205836135 +0000 UTC m=+160.724567925 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/46cab3bd-0f24-46e2-910c-b6e9a5f6ae5e-metrics-tls") pod "dns-default-q9tkc" (UID: "46cab3bd-0f24-46e2-910c-b6e9a5f6ae5e") : secret "dns-default-metrics-tls" not found
Sep 07 11:48:50.343129 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:48:50.343096    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-js7hj"
Sep 07 11:49:17.626513 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:49:17.626474    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3acaae8d-5930-49c6-9b9e-d286ac6a75d5-metrics-certs\") pod \"network-metrics-daemon-xf2gj\" (UID: \"3acaae8d-5930-49c6-9b9e-d286ac6a75d5\") " pod="openshift-multus/network-metrics-daemon-xf2gj"
Sep 07 11:49:17.626900 ip-10-0-135-173 kubenswrapper[2563]: E0907 11:49:17.626610    2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Sep 07 11:49:17.626900 ip-10-0-135-173 kubenswrapper[2563]: E0907 11:49:17.626669    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3acaae8d-5930-49c6-9b9e-d286ac6a75d5-metrics-certs podName:3acaae8d-5930-49c6-9b9e-d286ac6a75d5 nodeName:}" failed. No retries permitted until 2026-09-07 11:51:19.62665552 +0000 UTC m=+251.145387316 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3acaae8d-5930-49c6-9b9e-d286ac6a75d5-metrics-certs") pod "network-metrics-daemon-xf2gj" (UID: "3acaae8d-5930-49c6-9b9e-d286ac6a75d5") : secret "metrics-daemon-secret" not found
Sep 07 11:49:44.352540 ip-10-0-135-173 kubenswrapper[2563]: E0907 11:49:44.352482    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-q9tkc" podUID="46cab3bd-0f24-46e2-910c-b6e9a5f6ae5e"
Sep 07 11:49:44.540007 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:49:44.539981    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-q9tkc"
Sep 07 11:49:45.053139 ip-10-0-135-173 kubenswrapper[2563]: E0907 11:49:45.053104    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-xf2gj" podUID="3acaae8d-5930-49c6-9b9e-d286ac6a75d5"
Sep 07 11:49:49.220748 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:49:49.220700    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/46cab3bd-0f24-46e2-910c-b6e9a5f6ae5e-metrics-tls\") pod \"dns-default-q9tkc\" (UID: \"46cab3bd-0f24-46e2-910c-b6e9a5f6ae5e\") " pod="openshift-dns/dns-default-q9tkc"
Sep 07 11:49:49.221147 ip-10-0-135-173 kubenswrapper[2563]: E0907 11:49:49.220821    2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 07 11:49:49.221147 ip-10-0-135-173 kubenswrapper[2563]: E0907 11:49:49.220873    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/46cab3bd-0f24-46e2-910c-b6e9a5f6ae5e-metrics-tls podName:46cab3bd-0f24-46e2-910c-b6e9a5f6ae5e nodeName:}" failed. No retries permitted until 2026-09-07 11:51:51.220859511 +0000 UTC m=+282.739591303 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/46cab3bd-0f24-46e2-910c-b6e9a5f6ae5e-metrics-tls") pod "dns-default-q9tkc" (UID: "46cab3bd-0f24-46e2-910c-b6e9a5f6ae5e") : secret "dns-default-metrics-tls" not found
Sep 07 11:49:59.035673 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:49:59.035641    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xf2gj"
Sep 07 11:50:41.673696 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:50:41.673659    2563 generic.go:356] "Generic (PLEG): container finished" podID="e39e0070-19a6-4159-be9e-6beff874f190" containerID="0b8fcf5aa9575737f83d0dacfd35853bc11d5b205a2047b32de2c11cd412d1b3" exitCode=255
Sep 07 11:50:41.674108 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:50:41.673731    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-ccc4f9c94-vkr67" event={"ID":"e39e0070-19a6-4159-be9e-6beff874f190","Type":"ContainerDied","Data":"0b8fcf5aa9575737f83d0dacfd35853bc11d5b205a2047b32de2c11cd412d1b3"}
Sep 07 11:50:41.679406 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:50:41.679388    2563 scope.go:117] "RemoveContainer" containerID="0b8fcf5aa9575737f83d0dacfd35853bc11d5b205a2047b32de2c11cd412d1b3"
Sep 07 11:50:42.677701 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:50:42.677667    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-ccc4f9c94-vkr67" event={"ID":"e39e0070-19a6-4159-be9e-6beff874f190","Type":"ContainerStarted","Data":"34a45747ca2cd321a560a5a42862cf9ec9854cf6d1cf1d647e62e739b2179d96"}
Sep 07 11:50:59.505143 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:50:59.505108    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 11:51:19.630472 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:51:19.630408    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3acaae8d-5930-49c6-9b9e-d286ac6a75d5-metrics-certs\") pod \"network-metrics-daemon-xf2gj\" (UID: \"3acaae8d-5930-49c6-9b9e-d286ac6a75d5\") " pod="openshift-multus/network-metrics-daemon-xf2gj"
Sep 07 11:51:19.630953 ip-10-0-135-173 kubenswrapper[2563]: E0907 11:51:19.630553    2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Sep 07 11:51:19.630953 ip-10-0-135-173 kubenswrapper[2563]: E0907 11:51:19.630619    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3acaae8d-5930-49c6-9b9e-d286ac6a75d5-metrics-certs podName:3acaae8d-5930-49c6-9b9e-d286ac6a75d5 nodeName:}" failed. No retries permitted until 2026-09-07 11:53:21.630603523 +0000 UTC m=+373.149335327 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3acaae8d-5930-49c6-9b9e-d286ac6a75d5-metrics-certs") pod "network-metrics-daemon-xf2gj" (UID: "3acaae8d-5930-49c6-9b9e-d286ac6a75d5") : secret "metrics-daemon-secret" not found
Sep 07 11:51:25.197199 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:51:25.197162    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-85b454874b-4nfbc"]
Sep 07 11:51:25.201362 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:51:25.201336    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-676764f5b5-dsf9f"]
Sep 07 11:51:25.201496 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:51:25.201464    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-85b454874b-4nfbc"
Sep 07 11:51:25.203755 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:51:25.203702    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\""
Sep 07 11:51:25.203895 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:51:25.203763    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\""
Sep 07 11:51:25.203895 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:51:25.203861    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-6zp8c\""
Sep 07 11:51:25.204076 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:51:25.204059    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-676764f5b5-dsf9f"
Sep 07 11:51:25.204647 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:51:25.204628    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\""
Sep 07 11:51:25.204723 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:51:25.204671    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\""
Sep 07 11:51:25.205661 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:51:25.205637    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Sep 07 11:51:25.205661 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:51:25.205656    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Sep 07 11:51:25.205946 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:51:25.205929    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Sep 07 11:51:25.206806 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:51:25.206789    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-6v9dx\""
Sep 07 11:51:25.213472 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:51:25.213449    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\""
Sep 07 11:51:25.214254 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:51:25.214236    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Sep 07 11:51:25.221453 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:51:25.221432    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-85b454874b-4nfbc"]
Sep 07 11:51:25.222337 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:51:25.222317    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-676764f5b5-dsf9f"]
Sep 07 11:51:25.269537 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:51:25.269515    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/9ae06e91-9199-4863-a2ab-aa8397e8cbbb-bound-sa-token\") pod \"image-registry-676764f5b5-dsf9f\" (UID: \"9ae06e91-9199-4863-a2ab-aa8397e8cbbb\") " pod="openshift-image-registry/image-registry-676764f5b5-dsf9f"
Sep 07 11:51:25.269630 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:51:25.269544    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/b34ca85d-ac51-4724-a508-1026106b3f3a-config\") pod \"console-operator-85b454874b-4nfbc\" (UID: \"b34ca85d-ac51-4724-a508-1026106b3f3a\") " pod="openshift-console-operator/console-operator-85b454874b-4nfbc"
Sep 07 11:51:25.269630 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:51:25.269560    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/9ae06e91-9199-4863-a2ab-aa8397e8cbbb-trusted-ca\") pod \"image-registry-676764f5b5-dsf9f\" (UID: \"9ae06e91-9199-4863-a2ab-aa8397e8cbbb\") " pod="openshift-image-registry/image-registry-676764f5b5-dsf9f"
Sep 07 11:51:25.269630 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:51:25.269583    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/9ae06e91-9199-4863-a2ab-aa8397e8cbbb-installation-pull-secrets\") pod \"image-registry-676764f5b5-dsf9f\" (UID: \"9ae06e91-9199-4863-a2ab-aa8397e8cbbb\") " pod="openshift-image-registry/image-registry-676764f5b5-dsf9f"
Sep 07 11:51:25.269630 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:51:25.269621    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/b34ca85d-ac51-4724-a508-1026106b3f3a-trusted-ca\") pod \"console-operator-85b454874b-4nfbc\" (UID: \"b34ca85d-ac51-4724-a508-1026106b3f3a\") " pod="openshift-console-operator/console-operator-85b454874b-4nfbc"
Sep 07 11:51:25.269807 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:51:25.269649    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rmnf6\" (UniqueName: \"kubernetes.io/projected/b34ca85d-ac51-4724-a508-1026106b3f3a-kube-api-access-rmnf6\") pod \"console-operator-85b454874b-4nfbc\" (UID: \"b34ca85d-ac51-4724-a508-1026106b3f3a\") " pod="openshift-console-operator/console-operator-85b454874b-4nfbc"
Sep 07 11:51:25.269807 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:51:25.269667    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/9ae06e91-9199-4863-a2ab-aa8397e8cbbb-registry-certificates\") pod \"image-registry-676764f5b5-dsf9f\" (UID: \"9ae06e91-9199-4863-a2ab-aa8397e8cbbb\") " pod="openshift-image-registry/image-registry-676764f5b5-dsf9f"
Sep 07 11:51:25.269807 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:51:25.269703    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/9ae06e91-9199-4863-a2ab-aa8397e8cbbb-image-registry-private-configuration\") pod \"image-registry-676764f5b5-dsf9f\" (UID: \"9ae06e91-9199-4863-a2ab-aa8397e8cbbb\") " pod="openshift-image-registry/image-registry-676764f5b5-dsf9f"
Sep 07 11:51:25.269807 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:51:25.269720    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/9ae06e91-9199-4863-a2ab-aa8397e8cbbb-ca-trust-extracted\") pod \"image-registry-676764f5b5-dsf9f\" (UID: \"9ae06e91-9199-4863-a2ab-aa8397e8cbbb\") " pod="openshift-image-registry/image-registry-676764f5b5-dsf9f"
Sep 07 11:51:25.269807 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:51:25.269734    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-twzgv\" (UniqueName: \"kubernetes.io/projected/9ae06e91-9199-4863-a2ab-aa8397e8cbbb-kube-api-access-twzgv\") pod \"image-registry-676764f5b5-dsf9f\" (UID: \"9ae06e91-9199-4863-a2ab-aa8397e8cbbb\") " pod="openshift-image-registry/image-registry-676764f5b5-dsf9f"
Sep 07 11:51:25.269807 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:51:25.269754    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9ae06e91-9199-4863-a2ab-aa8397e8cbbb-registry-tls\") pod \"image-registry-676764f5b5-dsf9f\" (UID: \"9ae06e91-9199-4863-a2ab-aa8397e8cbbb\") " pod="openshift-image-registry/image-registry-676764f5b5-dsf9f"
Sep 07 11:51:25.270033 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:51:25.269855    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/b34ca85d-ac51-4724-a508-1026106b3f3a-serving-cert\") pod \"console-operator-85b454874b-4nfbc\" (UID: \"b34ca85d-ac51-4724-a508-1026106b3f3a\") " pod="openshift-console-operator/console-operator-85b454874b-4nfbc"
Sep 07 11:51:25.297973 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:51:25.297929    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-vhbb6"]
Sep 07 11:51:25.300669 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:51:25.300652    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-76bcd6ccf5-zls5m"]
Sep 07 11:51:25.300781 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:51:25.300765    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-vhbb6"
Sep 07 11:51:25.302659 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:51:25.302643    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\""
Sep 07 11:51:25.302767 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:51:25.302749    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\""
Sep 07 11:51:25.303095 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:51:25.303080    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-2qqlc\""
Sep 07 11:51:25.303800 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:51:25.303784    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-685456655f-48cfz"]
Sep 07 11:51:25.303926 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:51:25.303911    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-76bcd6ccf5-zls5m"
Sep 07 11:51:25.305375 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:51:25.305358    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-b2r5f\""
Sep 07 11:51:25.305472 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:51:25.305412    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\""
Sep 07 11:51:25.305565 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:51:25.305532    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Sep 07 11:51:25.305685 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:51:25.305667    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\""
Sep 07 11:51:25.305818 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:51:25.305705    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Sep 07 11:51:25.306901 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:51:25.306882    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-685456655f-48cfz"
Sep 07 11:51:25.308408 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:51:25.308392    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-xbsn7\""
Sep 07 11:51:25.310570 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:51:25.310527    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\""
Sep 07 11:51:25.312402 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:51:25.312385    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-vhbb6"]
Sep 07 11:51:25.313437 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:51:25.313385    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-685456655f-48cfz"]
Sep 07 11:51:25.314294 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:51:25.314274    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-76bcd6ccf5-zls5m"]
Sep 07 11:51:25.370377 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:51:25.370356    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/b34ca85d-ac51-4724-a508-1026106b3f3a-serving-cert\") pod \"console-operator-85b454874b-4nfbc\" (UID: \"b34ca85d-ac51-4724-a508-1026106b3f3a\") " pod="openshift-console-operator/console-operator-85b454874b-4nfbc"
Sep 07 11:51:25.370490 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:51:25.370382    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/9ae06e91-9199-4863-a2ab-aa8397e8cbbb-bound-sa-token\") pod \"image-registry-676764f5b5-dsf9f\" (UID: \"9ae06e91-9199-4863-a2ab-aa8397e8cbbb\") " pod="openshift-image-registry/image-registry-676764f5b5-dsf9f"
Sep 07 11:51:25.370490 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:51:25.370408    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/b34ca85d-ac51-4724-a508-1026106b3f3a-config\") pod \"console-operator-85b454874b-4nfbc\" (UID: \"b34ca85d-ac51-4724-a508-1026106b3f3a\") " pod="openshift-console-operator/console-operator-85b454874b-4nfbc"
Sep 07 11:51:25.370490 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:51:25.370448    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/9ae06e91-9199-4863-a2ab-aa8397e8cbbb-trusted-ca\") pod \"image-registry-676764f5b5-dsf9f\" (UID: \"9ae06e91-9199-4863-a2ab-aa8397e8cbbb\") " pod="openshift-image-registry/image-registry-676764f5b5-dsf9f"
Sep 07 11:51:25.370635 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:51:25.370615    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/9ae06e91-9199-4863-a2ab-aa8397e8cbbb-installation-pull-secrets\") pod \"image-registry-676764f5b5-dsf9f\" (UID: \"9ae06e91-9199-4863-a2ab-aa8397e8cbbb\") " pod="openshift-image-registry/image-registry-676764f5b5-dsf9f"
Sep 07 11:51:25.370680 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:51:25.370651    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7xdbg\" (UniqueName: \"kubernetes.io/projected/b6023824-522a-4bed-b277-7394abe09bbc-kube-api-access-7xdbg\") pod \"insights-operator-76bcd6ccf5-zls5m\" (UID: \"b6023824-522a-4bed-b277-7394abe09bbc\") " pod="openshift-insights/insights-operator-76bcd6ccf5-zls5m"
Sep 07 11:51:25.370723 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:51:25.370682    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/b34ca85d-ac51-4724-a508-1026106b3f3a-trusted-ca\") pod \"console-operator-85b454874b-4nfbc\" (UID: \"b34ca85d-ac51-4724-a508-1026106b3f3a\") " pod="openshift-console-operator/console-operator-85b454874b-4nfbc"
Sep 07 11:51:25.370772 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:51:25.370723    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rmnf6\" (UniqueName: \"kubernetes.io/projected/b34ca85d-ac51-4724-a508-1026106b3f3a-kube-api-access-rmnf6\") pod \"console-operator-85b454874b-4nfbc\" (UID: \"b34ca85d-ac51-4724-a508-1026106b3f3a\") " pod="openshift-console-operator/console-operator-85b454874b-4nfbc"
Sep 07 11:51:25.370772 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:51:25.370764    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/9ae06e91-9199-4863-a2ab-aa8397e8cbbb-registry-certificates\") pod \"image-registry-676764f5b5-dsf9f\" (UID: \"9ae06e91-9199-4863-a2ab-aa8397e8cbbb\") " pod="openshift-image-registry/image-registry-676764f5b5-dsf9f"
Sep 07 11:51:25.370880 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:51:25.370791    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/b6023824-522a-4bed-b277-7394abe09bbc-snapshots\") pod \"insights-operator-76bcd6ccf5-zls5m\" (UID: \"b6023824-522a-4bed-b277-7394abe09bbc\") " pod="openshift-insights/insights-operator-76bcd6ccf5-zls5m"
Sep 07 11:51:25.371595 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:51:25.371223    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/b34ca85d-ac51-4724-a508-1026106b3f3a-config\") pod \"console-operator-85b454874b-4nfbc\" (UID: \"b34ca85d-ac51-4724-a508-1026106b3f3a\") " pod="openshift-console-operator/console-operator-85b454874b-4nfbc"
Sep 07 11:51:25.371740 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:51:25.371643    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/9ae06e91-9199-4863-a2ab-aa8397e8cbbb-registry-certificates\") pod \"image-registry-676764f5b5-dsf9f\" (UID: \"9ae06e91-9199-4863-a2ab-aa8397e8cbbb\") " pod="openshift-image-registry/image-registry-676764f5b5-dsf9f"
Sep 07 11:51:25.371977 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:51:25.371952    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/b34ca85d-ac51-4724-a508-1026106b3f3a-trusted-ca\") pod \"console-operator-85b454874b-4nfbc\" (UID: \"b34ca85d-ac51-4724-a508-1026106b3f3a\") " pod="openshift-console-operator/console-operator-85b454874b-4nfbc"
Sep 07 11:51:25.372967 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:51:25.372943    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-729js\" (UniqueName: \"kubernetes.io/projected/3b027785-afe9-41e7-b5b9-e9eb56f4d276-kube-api-access-729js\") pod \"volume-data-source-validator-695c895bd6-vhbb6\" (UID: \"3b027785-afe9-41e7-b5b9-e9eb56f4d276\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-vhbb6"
Sep 07 11:51:25.373074 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:51:25.373056    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/9ae06e91-9199-4863-a2ab-aa8397e8cbbb-image-registry-private-configuration\") pod \"image-registry-676764f5b5-dsf9f\" (UID: \"9ae06e91-9199-4863-a2ab-aa8397e8cbbb\") " pod="openshift-image-registry/image-registry-676764f5b5-dsf9f"
Sep 07 11:51:25.375898 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:51:25.375575    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/9ae06e91-9199-4863-a2ab-aa8397e8cbbb-image-registry-private-configuration\") pod \"image-registry-676764f5b5-dsf9f\" (UID: \"9ae06e91-9199-4863-a2ab-aa8397e8cbbb\") " pod="openshift-image-registry/image-registry-676764f5b5-dsf9f"
Sep 07 11:51:25.375898 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:51:25.375632    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/9ae06e91-9199-4863-a2ab-aa8397e8cbbb-ca-trust-extracted\") pod \"image-registry-676764f5b5-dsf9f\" (UID: \"9ae06e91-9199-4863-a2ab-aa8397e8cbbb\") " pod="openshift-image-registry/image-registry-676764f5b5-dsf9f"
Sep 07 11:51:25.376054 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:51:25.375891    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-twzgv\" (UniqueName: \"kubernetes.io/projected/9ae06e91-9199-4863-a2ab-aa8397e8cbbb-kube-api-access-twzgv\") pod \"image-registry-676764f5b5-dsf9f\" (UID: \"9ae06e91-9199-4863-a2ab-aa8397e8cbbb\") " pod="openshift-image-registry/image-registry-676764f5b5-dsf9f"
Sep 07 11:51:25.376054 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:51:25.376039    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/9ae06e91-9199-4863-a2ab-aa8397e8cbbb-ca-trust-extracted\") pod \"image-registry-676764f5b5-dsf9f\" (UID: \"9ae06e91-9199-4863-a2ab-aa8397e8cbbb\") " pod="openshift-image-registry/image-registry-676764f5b5-dsf9f"
Sep 07 11:51:25.376178 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:51:25.376095    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b6023824-522a-4bed-b277-7394abe09bbc-trusted-ca-bundle\") pod \"insights-operator-76bcd6ccf5-zls5m\" (UID: \"b6023824-522a-4bed-b277-7394abe09bbc\") " pod="openshift-insights/insights-operator-76bcd6ccf5-zls5m"
Sep 07 11:51:25.376178 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:51:25.376144    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9ae06e91-9199-4863-a2ab-aa8397e8cbbb-registry-tls\") pod \"image-registry-676764f5b5-dsf9f\" (UID: \"9ae06e91-9199-4863-a2ab-aa8397e8cbbb\") " pod="openshift-image-registry/image-registry-676764f5b5-dsf9f"
Sep 07 11:51:25.376282 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:51:25.376224    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b6023824-522a-4bed-b277-7394abe09bbc-service-ca-bundle\") pod \"insights-operator-76bcd6ccf5-zls5m\" (UID: \"b6023824-522a-4bed-b277-7394abe09bbc\") " pod="openshift-insights/insights-operator-76bcd6ccf5-zls5m"
Sep 07 11:51:25.376335 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:51:25.376277    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mllzq\" (UniqueName: \"kubernetes.io/projected/e532492e-7da6-46a3-a234-c0d63595cc99-kube-api-access-mllzq\") pod \"network-check-source-685456655f-48cfz\" (UID: \"e532492e-7da6-46a3-a234-c0d63595cc99\") " pod="openshift-network-diagnostics/network-check-source-685456655f-48cfz"
Sep 07 11:51:25.376335 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:51:25.376325    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b6023824-522a-4bed-b277-7394abe09bbc-tmp\") pod \"insights-operator-76bcd6ccf5-zls5m\" (UID: \"b6023824-522a-4bed-b277-7394abe09bbc\") " pod="openshift-insights/insights-operator-76bcd6ccf5-zls5m"
Sep 07 11:51:25.376457 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:51:25.376358    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/9ae06e91-9199-4863-a2ab-aa8397e8cbbb-trusted-ca\") pod \"image-registry-676764f5b5-dsf9f\" (UID: \"9ae06e91-9199-4863-a2ab-aa8397e8cbbb\") " pod="openshift-image-registry/image-registry-676764f5b5-dsf9f"
Sep 07 11:51:25.376457 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:51:25.376390    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/b6023824-522a-4bed-b277-7394abe09bbc-serving-cert\") pod \"insights-operator-76bcd6ccf5-zls5m\" (UID: \"b6023824-522a-4bed-b277-7394abe09bbc\") " pod="openshift-insights/insights-operator-76bcd6ccf5-zls5m"
Sep 07 11:51:25.376457 ip-10-0-135-173 kubenswrapper[2563]: E0907 11:51:25.376252    2563 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 07 11:51:25.376457 ip-10-0-135-173 kubenswrapper[2563]: E0907 11:51:25.376456    2563 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-676764f5b5-dsf9f: secret "image-registry-tls" not found
Sep 07 11:51:25.376685 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:51:25.376486    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/9ae06e91-9199-4863-a2ab-aa8397e8cbbb-installation-pull-secrets\") pod \"image-registry-676764f5b5-dsf9f\" (UID: \"9ae06e91-9199-4863-a2ab-aa8397e8cbbb\") " pod="openshift-image-registry/image-registry-676764f5b5-dsf9f"
Sep 07 11:51:25.376685 ip-10-0-135-173 kubenswrapper[2563]: E0907 11:51:25.376527    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9ae06e91-9199-4863-a2ab-aa8397e8cbbb-registry-tls podName:9ae06e91-9199-4863-a2ab-aa8397e8cbbb nodeName:}" failed. No retries permitted until 2026-09-07 11:51:25.876505098 +0000 UTC m=+257.395236901 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/9ae06e91-9199-4863-a2ab-aa8397e8cbbb-registry-tls") pod "image-registry-676764f5b5-dsf9f" (UID: "9ae06e91-9199-4863-a2ab-aa8397e8cbbb") : secret "image-registry-tls" not found
Sep 07 11:51:25.376685 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:51:25.376592    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/b34ca85d-ac51-4724-a508-1026106b3f3a-serving-cert\") pod \"console-operator-85b454874b-4nfbc\" (UID: \"b34ca85d-ac51-4724-a508-1026106b3f3a\") " pod="openshift-console-operator/console-operator-85b454874b-4nfbc"
Sep 07 11:51:25.378406 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:51:25.378385    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/9ae06e91-9199-4863-a2ab-aa8397e8cbbb-bound-sa-token\") pod \"image-registry-676764f5b5-dsf9f\" (UID: \"9ae06e91-9199-4863-a2ab-aa8397e8cbbb\") " pod="openshift-image-registry/image-registry-676764f5b5-dsf9f"
Sep 07 11:51:25.379014 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:51:25.378994    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rmnf6\" (UniqueName: \"kubernetes.io/projected/b34ca85d-ac51-4724-a508-1026106b3f3a-kube-api-access-rmnf6\") pod \"console-operator-85b454874b-4nfbc\" (UID: \"b34ca85d-ac51-4724-a508-1026106b3f3a\") " pod="openshift-console-operator/console-operator-85b454874b-4nfbc"
Sep 07 11:51:25.381747 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:51:25.381729    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-twzgv\" (UniqueName: \"kubernetes.io/projected/9ae06e91-9199-4863-a2ab-aa8397e8cbbb-kube-api-access-twzgv\") pod \"image-registry-676764f5b5-dsf9f\" (UID: \"9ae06e91-9199-4863-a2ab-aa8397e8cbbb\") " pod="openshift-image-registry/image-registry-676764f5b5-dsf9f"
Sep 07 11:51:25.423886 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:51:25.423863    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 11:51:25.475772 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:51:25.475701    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 11:51:25.477719 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:51:25.477702    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b6023824-522a-4bed-b277-7394abe09bbc-service-ca-bundle\") pod \"insights-operator-76bcd6ccf5-zls5m\" (UID: \"b6023824-522a-4bed-b277-7394abe09bbc\") " pod="openshift-insights/insights-operator-76bcd6ccf5-zls5m"
Sep 07 11:51:25.477776 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:51:25.477727    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mllzq\" (UniqueName: \"kubernetes.io/projected/e532492e-7da6-46a3-a234-c0d63595cc99-kube-api-access-mllzq\") pod \"network-check-source-685456655f-48cfz\" (UID: \"e532492e-7da6-46a3-a234-c0d63595cc99\") " pod="openshift-network-diagnostics/network-check-source-685456655f-48cfz"
Sep 07 11:51:25.477776 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:51:25.477750    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b6023824-522a-4bed-b277-7394abe09bbc-tmp\") pod \"insights-operator-76bcd6ccf5-zls5m\" (UID: \"b6023824-522a-4bed-b277-7394abe09bbc\") " pod="openshift-insights/insights-operator-76bcd6ccf5-zls5m"
Sep 07 11:51:25.477876 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:51:25.477777    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/b6023824-522a-4bed-b277-7394abe09bbc-serving-cert\") pod \"insights-operator-76bcd6ccf5-zls5m\" (UID: \"b6023824-522a-4bed-b277-7394abe09bbc\") " pod="openshift-insights/insights-operator-76bcd6ccf5-zls5m"
Sep 07 11:51:25.477988 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:51:25.477966    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7xdbg\" (UniqueName: \"kubernetes.io/projected/b6023824-522a-4bed-b277-7394abe09bbc-kube-api-access-7xdbg\") pod \"insights-operator-76bcd6ccf5-zls5m\" (UID: \"b6023824-522a-4bed-b277-7394abe09bbc\") " pod="openshift-insights/insights-operator-76bcd6ccf5-zls5m"
Sep 07 11:51:25.478052 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:51:25.478034    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/b6023824-522a-4bed-b277-7394abe09bbc-snapshots\") pod \"insights-operator-76bcd6ccf5-zls5m\" (UID: \"b6023824-522a-4bed-b277-7394abe09bbc\") " pod="openshift-insights/insights-operator-76bcd6ccf5-zls5m"
Sep 07 11:51:25.478101 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:51:25.478074    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-729js\" (UniqueName: \"kubernetes.io/projected/3b027785-afe9-41e7-b5b9-e9eb56f4d276-kube-api-access-729js\") pod \"volume-data-source-validator-695c895bd6-vhbb6\" (UID: \"3b027785-afe9-41e7-b5b9-e9eb56f4d276\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-vhbb6"
Sep 07 11:51:25.478158 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:51:25.478106    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b6023824-522a-4bed-b277-7394abe09bbc-trusted-ca-bundle\") pod \"insights-operator-76bcd6ccf5-zls5m\" (UID: \"b6023824-522a-4bed-b277-7394abe09bbc\") " pod="openshift-insights/insights-operator-76bcd6ccf5-zls5m"
Sep 07 11:51:25.478208 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:51:25.478165    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b6023824-522a-4bed-b277-7394abe09bbc-tmp\") pod \"insights-operator-76bcd6ccf5-zls5m\" (UID: \"b6023824-522a-4bed-b277-7394abe09bbc\") " pod="openshift-insights/insights-operator-76bcd6ccf5-zls5m"
Sep 07 11:51:25.478295 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:51:25.478279    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b6023824-522a-4bed-b277-7394abe09bbc-service-ca-bundle\") pod \"insights-operator-76bcd6ccf5-zls5m\" (UID: \"b6023824-522a-4bed-b277-7394abe09bbc\") " pod="openshift-insights/insights-operator-76bcd6ccf5-zls5m"
Sep 07 11:51:25.478605 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:51:25.478585    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/b6023824-522a-4bed-b277-7394abe09bbc-snapshots\") pod \"insights-operator-76bcd6ccf5-zls5m\" (UID: \"b6023824-522a-4bed-b277-7394abe09bbc\") " pod="openshift-insights/insights-operator-76bcd6ccf5-zls5m"
Sep 07 11:51:25.479095 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:51:25.479074    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b6023824-522a-4bed-b277-7394abe09bbc-trusted-ca-bundle\") pod \"insights-operator-76bcd6ccf5-zls5m\" (UID: \"b6023824-522a-4bed-b277-7394abe09bbc\") " pod="openshift-insights/insights-operator-76bcd6ccf5-zls5m"
Sep 07 11:51:25.480086 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:51:25.480064    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/b6023824-522a-4bed-b277-7394abe09bbc-serving-cert\") pod \"insights-operator-76bcd6ccf5-zls5m\" (UID: \"b6023824-522a-4bed-b277-7394abe09bbc\") " pod="openshift-insights/insights-operator-76bcd6ccf5-zls5m"
Sep 07 11:51:25.485091 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:51:25.485064    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mllzq\" (UniqueName: \"kubernetes.io/projected/e532492e-7da6-46a3-a234-c0d63595cc99-kube-api-access-mllzq\") pod \"network-check-source-685456655f-48cfz\" (UID: \"e532492e-7da6-46a3-a234-c0d63595cc99\") " pod="openshift-network-diagnostics/network-check-source-685456655f-48cfz"
Sep 07 11:51:25.485170 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:51:25.485134    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-729js\" (UniqueName: \"kubernetes.io/projected/3b027785-afe9-41e7-b5b9-e9eb56f4d276-kube-api-access-729js\") pod \"volume-data-source-validator-695c895bd6-vhbb6\" (UID: \"3b027785-afe9-41e7-b5b9-e9eb56f4d276\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-vhbb6"
Sep 07 11:51:25.485625 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:51:25.485608    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7xdbg\" (UniqueName: \"kubernetes.io/projected/b6023824-522a-4bed-b277-7394abe09bbc-kube-api-access-7xdbg\") pod \"insights-operator-76bcd6ccf5-zls5m\" (UID: \"b6023824-522a-4bed-b277-7394abe09bbc\") " pod="openshift-insights/insights-operator-76bcd6ccf5-zls5m"
Sep 07 11:51:25.512469 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:51:25.512430    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-85b454874b-4nfbc"
Sep 07 11:51:25.610654 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:51:25.610613    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-vhbb6"
Sep 07 11:51:25.617756 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:51:25.617733    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-76bcd6ccf5-zls5m"
Sep 07 11:51:25.622472 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:51:25.622452    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-685456655f-48cfz"
Sep 07 11:51:25.775911 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:51:25.775846    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-85b454874b-4nfbc"]
Sep 07 11:51:25.778624 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:51:25.778599    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb34ca85d_ac51_4724_a508_1026106b3f3a.slice/crio-3cc73b401408d919dc691d2c24504ae6a0815a68a9648dc07a774fc3d93a1136 WatchSource:0}: Error finding container 3cc73b401408d919dc691d2c24504ae6a0815a68a9648dc07a774fc3d93a1136: Status 404 returned error can't find the container with id 3cc73b401408d919dc691d2c24504ae6a0815a68a9648dc07a774fc3d93a1136
Sep 07 11:51:25.786662 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:51:25.786637    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-85b454874b-4nfbc" event={"ID":"b34ca85d-ac51-4724-a508-1026106b3f3a","Type":"ContainerStarted","Data":"3cc73b401408d919dc691d2c24504ae6a0815a68a9648dc07a774fc3d93a1136"}
Sep 07 11:51:25.875669 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:51:25.875641    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-vhbb6"]
Sep 07 11:51:25.878969 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:51:25.878947    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3b027785_afe9_41e7_b5b9_e9eb56f4d276.slice/crio-8293990bd9048a066c12c3b6188d364eca9996df1671639444207e636fc470b2 WatchSource:0}: Error finding container 8293990bd9048a066c12c3b6188d364eca9996df1671639444207e636fc470b2: Status 404 returned error can't find the container with id 8293990bd9048a066c12c3b6188d364eca9996df1671639444207e636fc470b2
Sep 07 11:51:25.879855 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:51:25.879836    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9ae06e91-9199-4863-a2ab-aa8397e8cbbb-registry-tls\") pod \"image-registry-676764f5b5-dsf9f\" (UID: \"9ae06e91-9199-4863-a2ab-aa8397e8cbbb\") " pod="openshift-image-registry/image-registry-676764f5b5-dsf9f"
Sep 07 11:51:25.879949 ip-10-0-135-173 kubenswrapper[2563]: E0907 11:51:25.879931    2563 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 07 11:51:25.879949 ip-10-0-135-173 kubenswrapper[2563]: E0907 11:51:25.879947    2563 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-676764f5b5-dsf9f: secret "image-registry-tls" not found
Sep 07 11:51:25.880083 ip-10-0-135-173 kubenswrapper[2563]: E0907 11:51:25.880004    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9ae06e91-9199-4863-a2ab-aa8397e8cbbb-registry-tls podName:9ae06e91-9199-4863-a2ab-aa8397e8cbbb nodeName:}" failed. No retries permitted until 2026-09-07 11:51:26.879986112 +0000 UTC m=+258.398717916 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/9ae06e91-9199-4863-a2ab-aa8397e8cbbb-registry-tls") pod "image-registry-676764f5b5-dsf9f" (UID: "9ae06e91-9199-4863-a2ab-aa8397e8cbbb") : secret "image-registry-tls" not found
Sep 07 11:51:25.887348 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:51:25.887325    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-76bcd6ccf5-zls5m"]
Sep 07 11:51:25.890161 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:51:25.890134    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb6023824_522a_4bed_b277_7394abe09bbc.slice/crio-6ed14794714b827dd1389686fbccb3951d5f7091e2242b3a291d69663ac945c3 WatchSource:0}: Error finding container 6ed14794714b827dd1389686fbccb3951d5f7091e2242b3a291d69663ac945c3: Status 404 returned error can't find the container with id 6ed14794714b827dd1389686fbccb3951d5f7091e2242b3a291d69663ac945c3
Sep 07 11:51:25.898206 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:51:25.898187    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-685456655f-48cfz"]
Sep 07 11:51:25.901131 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:51:25.901110    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode532492e_7da6_46a3_a234_c0d63595cc99.slice/crio-64ef1e493b6c4c285731d23e639c9e5de9caa3b5f2057a48a97c0ef2a7ad03fb WatchSource:0}: Error finding container 64ef1e493b6c4c285731d23e639c9e5de9caa3b5f2057a48a97c0ef2a7ad03fb: Status 404 returned error can't find the container with id 64ef1e493b6c4c285731d23e639c9e5de9caa3b5f2057a48a97c0ef2a7ad03fb
Sep 07 11:51:26.793830 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:51:26.793758    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-76bcd6ccf5-zls5m" event={"ID":"b6023824-522a-4bed-b277-7394abe09bbc","Type":"ContainerStarted","Data":"6ed14794714b827dd1389686fbccb3951d5f7091e2242b3a291d69663ac945c3"}
Sep 07 11:51:26.795286 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:51:26.795213    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-vhbb6" event={"ID":"3b027785-afe9-41e7-b5b9-e9eb56f4d276","Type":"ContainerStarted","Data":"8293990bd9048a066c12c3b6188d364eca9996df1671639444207e636fc470b2"}
Sep 07 11:51:26.796960 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:51:26.796937    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-685456655f-48cfz" event={"ID":"e532492e-7da6-46a3-a234-c0d63595cc99","Type":"ContainerStarted","Data":"5072a16ec610e1062c1cfd8c44f70b913b34c0233b155915dba83240c4c8fc87"}
Sep 07 11:51:26.797067 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:51:26.796978    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-685456655f-48cfz" event={"ID":"e532492e-7da6-46a3-a234-c0d63595cc99","Type":"ContainerStarted","Data":"64ef1e493b6c4c285731d23e639c9e5de9caa3b5f2057a48a97c0ef2a7ad03fb"}
Sep 07 11:51:26.810666 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:51:26.810614    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-685456655f-48cfz" podStartSLOduration=1.810595891 podStartE2EDuration="1.810595891s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 11:51:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 11:51:26.809307613 +0000 UTC m=+258.328039426" watchObservedRunningTime="2026-09-07 11:51:26.810595891 +0000 UTC m=+258.329327705"
Sep 07 11:51:26.888130 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:51:26.888092    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9ae06e91-9199-4863-a2ab-aa8397e8cbbb-registry-tls\") pod \"image-registry-676764f5b5-dsf9f\" (UID: \"9ae06e91-9199-4863-a2ab-aa8397e8cbbb\") " pod="openshift-image-registry/image-registry-676764f5b5-dsf9f"
Sep 07 11:51:26.888281 ip-10-0-135-173 kubenswrapper[2563]: E0907 11:51:26.888174    2563 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 07 11:51:26.888281 ip-10-0-135-173 kubenswrapper[2563]: E0907 11:51:26.888197    2563 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-676764f5b5-dsf9f: secret "image-registry-tls" not found
Sep 07 11:51:26.888397 ip-10-0-135-173 kubenswrapper[2563]: E0907 11:51:26.888305    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9ae06e91-9199-4863-a2ab-aa8397e8cbbb-registry-tls podName:9ae06e91-9199-4863-a2ab-aa8397e8cbbb nodeName:}" failed. No retries permitted until 2026-09-07 11:51:28.888284817 +0000 UTC m=+260.407016624 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/9ae06e91-9199-4863-a2ab-aa8397e8cbbb-registry-tls") pod "image-registry-676764f5b5-dsf9f" (UID: "9ae06e91-9199-4863-a2ab-aa8397e8cbbb") : secret "image-registry-tls" not found
Sep 07 11:51:27.802342 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:51:27.802297    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-vhbb6" event={"ID":"3b027785-afe9-41e7-b5b9-e9eb56f4d276","Type":"ContainerStarted","Data":"c4cc81069c31bd92a545502bca0ffaf2f85a62803a9a81e242a25f1b432b0005"}
Sep 07 11:51:27.815860 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:51:27.815819    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-vhbb6" podStartSLOduration=1.708716267 podStartE2EDuration="2.815803826s" totalImagesPullingTime="1.107087559s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 11:51:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 11:51:27.814988893 +0000 UTC m=+259.333720709" watchObservedRunningTime="2026-09-07 11:51:27.815803826 +0000 UTC m=+259.334535638"
Sep 07 11:51:27.884954 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:51:27.884911    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 11:51:28.807322 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:51:28.807279    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-76bcd6ccf5-zls5m" event={"ID":"b6023824-522a-4bed-b277-7394abe09bbc","Type":"ContainerStarted","Data":"66ca367eafbc2c34a13a95755721461617f1f4e756dcd9f3cfb0728b362c31cf"}
Sep 07 11:51:28.813767 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:51:28.813745    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-4nfbc_b34ca85d-ac51-4724-a508-1026106b3f3a/console-operator/0.log"
Sep 07 11:51:28.813888 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:51:28.813783    2563 generic.go:356] "Generic (PLEG): container finished" podID="b34ca85d-ac51-4724-a508-1026106b3f3a" containerID="9da7503fe7096c2b3dcd28f7e2680290a91205250ddec06e5f6bcfe9a5421983" exitCode=255
Sep 07 11:51:28.813888 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:51:28.813815    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-85b454874b-4nfbc" event={"ID":"b34ca85d-ac51-4724-a508-1026106b3f3a","Type":"ContainerDied","Data":"9da7503fe7096c2b3dcd28f7e2680290a91205250ddec06e5f6bcfe9a5421983"}
Sep 07 11:51:28.814095 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:51:28.814081    2563 scope.go:117] "RemoveContainer" containerID="9da7503fe7096c2b3dcd28f7e2680290a91205250ddec06e5f6bcfe9a5421983"
Sep 07 11:51:28.823485 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:51:28.823442    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-76bcd6ccf5-zls5m" podStartSLOduration=1.465190767 podStartE2EDuration="3.82340921s" totalImagesPullingTime="2.358218443s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 11:51:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 11:51:28.821852434 +0000 UTC m=+260.340584246" watchObservedRunningTime="2026-09-07 11:51:28.82340921 +0000 UTC m=+260.342141026"
Sep 07 11:51:28.902437 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:51:28.902396    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9ae06e91-9199-4863-a2ab-aa8397e8cbbb-registry-tls\") pod \"image-registry-676764f5b5-dsf9f\" (UID: \"9ae06e91-9199-4863-a2ab-aa8397e8cbbb\") " pod="openshift-image-registry/image-registry-676764f5b5-dsf9f"
Sep 07 11:51:28.902593 ip-10-0-135-173 kubenswrapper[2563]: E0907 11:51:28.902571    2563 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 07 11:51:28.902652 ip-10-0-135-173 kubenswrapper[2563]: E0907 11:51:28.902595    2563 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-676764f5b5-dsf9f: secret "image-registry-tls" not found
Sep 07 11:51:28.902709 ip-10-0-135-173 kubenswrapper[2563]: E0907 11:51:28.902665    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9ae06e91-9199-4863-a2ab-aa8397e8cbbb-registry-tls podName:9ae06e91-9199-4863-a2ab-aa8397e8cbbb nodeName:}" failed. No retries permitted until 2026-09-07 11:51:32.902645028 +0000 UTC m=+264.421376823 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/9ae06e91-9199-4863-a2ab-aa8397e8cbbb-registry-tls") pod "image-registry-676764f5b5-dsf9f" (UID: "9ae06e91-9199-4863-a2ab-aa8397e8cbbb") : secret "image-registry-tls" not found
Sep 07 11:51:29.434568 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:51:29.434528    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-mchxp"]
Sep 07 11:51:29.437674 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:51:29.437654    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-mchxp"
Sep 07 11:51:29.439595 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:51:29.439570    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Sep 07 11:51:29.439687 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:51:29.439575    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Sep 07 11:51:29.439728 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:51:29.439695    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-vwz2b\""
Sep 07 11:51:29.448975 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:51:29.448954    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-mchxp"]
Sep 07 11:51:29.508620 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:51:29.508593    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f302b236-f400-4839-8d56-edaf5de4bb75-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-mchxp\" (UID: \"f302b236-f400-4839-8d56-edaf5de4bb75\") " pod="openshift-insights/insights-runtime-extractor-mchxp"
Sep 07 11:51:29.508721 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:51:29.508641    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/f302b236-f400-4839-8d56-edaf5de4bb75-data-volume\") pod \"insights-runtime-extractor-mchxp\" (UID: \"f302b236-f400-4839-8d56-edaf5de4bb75\") " pod="openshift-insights/insights-runtime-extractor-mchxp"
Sep 07 11:51:29.508788 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:51:29.508758    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f302b236-f400-4839-8d56-edaf5de4bb75-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-mchxp\" (UID: \"f302b236-f400-4839-8d56-edaf5de4bb75\") " pod="openshift-insights/insights-runtime-extractor-mchxp"
Sep 07 11:51:29.508788 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:51:29.508783    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mk2tv\" (UniqueName: \"kubernetes.io/projected/f302b236-f400-4839-8d56-edaf5de4bb75-kube-api-access-mk2tv\") pod \"insights-runtime-extractor-mchxp\" (UID: \"f302b236-f400-4839-8d56-edaf5de4bb75\") " pod="openshift-insights/insights-runtime-extractor-mchxp"
Sep 07 11:51:29.508871 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:51:29.508809    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/f302b236-f400-4839-8d56-edaf5de4bb75-crio-socket\") pod \"insights-runtime-extractor-mchxp\" (UID: \"f302b236-f400-4839-8d56-edaf5de4bb75\") " pod="openshift-insights/insights-runtime-extractor-mchxp"
Sep 07 11:51:29.609707 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:51:29.609681    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/f302b236-f400-4839-8d56-edaf5de4bb75-data-volume\") pod \"insights-runtime-extractor-mchxp\" (UID: \"f302b236-f400-4839-8d56-edaf5de4bb75\") " pod="openshift-insights/insights-runtime-extractor-mchxp"
Sep 07 11:51:29.609797 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:51:29.609737    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f302b236-f400-4839-8d56-edaf5de4bb75-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-mchxp\" (UID: \"f302b236-f400-4839-8d56-edaf5de4bb75\") " pod="openshift-insights/insights-runtime-extractor-mchxp"
Sep 07 11:51:29.609797 ip-10-0-135-173 kubenswrapper[2563]: E0907 11:51:29.609791    2563 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 07 11:51:29.609855 ip-10-0-135-173 kubenswrapper[2563]: E0907 11:51:29.609840    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/f302b236-f400-4839-8d56-edaf5de4bb75-kube-rbac-proxy-cm podName:f302b236-f400-4839-8d56-edaf5de4bb75 nodeName:}" failed. No retries permitted until 2026-09-07 11:51:30.109827448 +0000 UTC m=+261.628559251 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/f302b236-f400-4839-8d56-edaf5de4bb75-kube-rbac-proxy-cm") pod "insights-runtime-extractor-mchxp" (UID: "f302b236-f400-4839-8d56-edaf5de4bb75") : configmap "kube-rbac-proxy" not found
Sep 07 11:51:29.609904 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:51:29.609851    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mk2tv\" (UniqueName: \"kubernetes.io/projected/f302b236-f400-4839-8d56-edaf5de4bb75-kube-api-access-mk2tv\") pod \"insights-runtime-extractor-mchxp\" (UID: \"f302b236-f400-4839-8d56-edaf5de4bb75\") " pod="openshift-insights/insights-runtime-extractor-mchxp"
Sep 07 11:51:29.609904 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:51:29.609875    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/f302b236-f400-4839-8d56-edaf5de4bb75-crio-socket\") pod \"insights-runtime-extractor-mchxp\" (UID: \"f302b236-f400-4839-8d56-edaf5de4bb75\") " pod="openshift-insights/insights-runtime-extractor-mchxp"
Sep 07 11:51:29.609904 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:51:29.609902    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f302b236-f400-4839-8d56-edaf5de4bb75-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-mchxp\" (UID: \"f302b236-f400-4839-8d56-edaf5de4bb75\") " pod="openshift-insights/insights-runtime-extractor-mchxp"
Sep 07 11:51:29.610008 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:51:29.609960    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/f302b236-f400-4839-8d56-edaf5de4bb75-crio-socket\") pod \"insights-runtime-extractor-mchxp\" (UID: \"f302b236-f400-4839-8d56-edaf5de4bb75\") " pod="openshift-insights/insights-runtime-extractor-mchxp"
Sep 07 11:51:29.610008 ip-10-0-135-173 kubenswrapper[2563]: E0907 11:51:29.609967    2563 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 07 11:51:29.610008 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:51:29.609991    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/f302b236-f400-4839-8d56-edaf5de4bb75-data-volume\") pod \"insights-runtime-extractor-mchxp\" (UID: \"f302b236-f400-4839-8d56-edaf5de4bb75\") " pod="openshift-insights/insights-runtime-extractor-mchxp"
Sep 07 11:51:29.610099 ip-10-0-135-173 kubenswrapper[2563]: E0907 11:51:29.610049    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f302b236-f400-4839-8d56-edaf5de4bb75-insights-runtime-extractor-tls podName:f302b236-f400-4839-8d56-edaf5de4bb75 nodeName:}" failed. No retries permitted until 2026-09-07 11:51:30.110032392 +0000 UTC m=+261.628764195 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/f302b236-f400-4839-8d56-edaf5de4bb75-insights-runtime-extractor-tls") pod "insights-runtime-extractor-mchxp" (UID: "f302b236-f400-4839-8d56-edaf5de4bb75") : secret "insights-runtime-extractor-tls" not found
Sep 07 11:51:29.618088 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:51:29.618059    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mk2tv\" (UniqueName: \"kubernetes.io/projected/f302b236-f400-4839-8d56-edaf5de4bb75-kube-api-access-mk2tv\") pod \"insights-runtime-extractor-mchxp\" (UID: \"f302b236-f400-4839-8d56-edaf5de4bb75\") " pod="openshift-insights/insights-runtime-extractor-mchxp"
Sep 07 11:51:29.686646 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:51:29.686585    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 11:51:29.754022 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:51:29.753991    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 11:51:29.820357 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:51:29.820338    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-4nfbc_b34ca85d-ac51-4724-a508-1026106b3f3a/console-operator/1.log"
Sep 07 11:51:29.820783 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:51:29.820769    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-4nfbc_b34ca85d-ac51-4724-a508-1026106b3f3a/console-operator/0.log"
Sep 07 11:51:29.820825 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:51:29.820801    2563 generic.go:356] "Generic (PLEG): container finished" podID="b34ca85d-ac51-4724-a508-1026106b3f3a" containerID="b8346748a923f5cd04b940accc239af84a8a0c117c969c7c1ca982102a7daf9b" exitCode=255
Sep 07 11:51:29.820884 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:51:29.820872    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-85b454874b-4nfbc" event={"ID":"b34ca85d-ac51-4724-a508-1026106b3f3a","Type":"ContainerDied","Data":"b8346748a923f5cd04b940accc239af84a8a0c117c969c7c1ca982102a7daf9b"}
Sep 07 11:51:29.820929 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:51:29.820920    2563 scope.go:117] "RemoveContainer" containerID="9da7503fe7096c2b3dcd28f7e2680290a91205250ddec06e5f6bcfe9a5421983"
Sep 07 11:51:29.821092 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:51:29.821072    2563 scope.go:117] "RemoveContainer" containerID="b8346748a923f5cd04b940accc239af84a8a0c117c969c7c1ca982102a7daf9b"
Sep 07 11:51:29.821247 ip-10-0-135-173 kubenswrapper[2563]: E0907 11:51:29.821225    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-85b454874b-4nfbc_openshift-console-operator(b34ca85d-ac51-4724-a508-1026106b3f3a)\"" pod="openshift-console-operator/console-operator-85b454874b-4nfbc" podUID="b34ca85d-ac51-4724-a508-1026106b3f3a"
Sep 07 11:51:30.113813 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:51:30.113767    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f302b236-f400-4839-8d56-edaf5de4bb75-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-mchxp\" (UID: \"f302b236-f400-4839-8d56-edaf5de4bb75\") " pod="openshift-insights/insights-runtime-extractor-mchxp"
Sep 07 11:51:30.113962 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:51:30.113829    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f302b236-f400-4839-8d56-edaf5de4bb75-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-mchxp\" (UID: \"f302b236-f400-4839-8d56-edaf5de4bb75\") " pod="openshift-insights/insights-runtime-extractor-mchxp"
Sep 07 11:51:30.113962 ip-10-0-135-173 kubenswrapper[2563]: E0907 11:51:30.113910    2563 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 07 11:51:30.113962 ip-10-0-135-173 kubenswrapper[2563]: E0907 11:51:30.113914    2563 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 07 11:51:30.114076 ip-10-0-135-173 kubenswrapper[2563]: E0907 11:51:30.113975    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/f302b236-f400-4839-8d56-edaf5de4bb75-kube-rbac-proxy-cm podName:f302b236-f400-4839-8d56-edaf5de4bb75 nodeName:}" failed. No retries permitted until 2026-09-07 11:51:31.113954931 +0000 UTC m=+262.632686745 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/f302b236-f400-4839-8d56-edaf5de4bb75-kube-rbac-proxy-cm") pod "insights-runtime-extractor-mchxp" (UID: "f302b236-f400-4839-8d56-edaf5de4bb75") : configmap "kube-rbac-proxy" not found
Sep 07 11:51:30.114076 ip-10-0-135-173 kubenswrapper[2563]: E0907 11:51:30.113996    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f302b236-f400-4839-8d56-edaf5de4bb75-insights-runtime-extractor-tls podName:f302b236-f400-4839-8d56-edaf5de4bb75 nodeName:}" failed. No retries permitted until 2026-09-07 11:51:31.113986626 +0000 UTC m=+262.632718445 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/f302b236-f400-4839-8d56-edaf5de4bb75-insights-runtime-extractor-tls") pod "insights-runtime-extractor-mchxp" (UID: "f302b236-f400-4839-8d56-edaf5de4bb75") : secret "insights-runtime-extractor-tls" not found
Sep 07 11:51:30.825248 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:51:30.825217    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-4nfbc_b34ca85d-ac51-4724-a508-1026106b3f3a/console-operator/1.log"
Sep 07 11:51:30.825663 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:51:30.825540    2563 scope.go:117] "RemoveContainer" containerID="b8346748a923f5cd04b940accc239af84a8a0c117c969c7c1ca982102a7daf9b"
Sep 07 11:51:30.825736 ip-10-0-135-173 kubenswrapper[2563]: E0907 11:51:30.825681    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-85b454874b-4nfbc_openshift-console-operator(b34ca85d-ac51-4724-a508-1026106b3f3a)\"" pod="openshift-console-operator/console-operator-85b454874b-4nfbc" podUID="b34ca85d-ac51-4724-a508-1026106b3f3a"
Sep 07 11:51:31.120467 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:51:31.120346    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f302b236-f400-4839-8d56-edaf5de4bb75-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-mchxp\" (UID: \"f302b236-f400-4839-8d56-edaf5de4bb75\") " pod="openshift-insights/insights-runtime-extractor-mchxp"
Sep 07 11:51:31.120467 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:51:31.120443    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f302b236-f400-4839-8d56-edaf5de4bb75-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-mchxp\" (UID: \"f302b236-f400-4839-8d56-edaf5de4bb75\") " pod="openshift-insights/insights-runtime-extractor-mchxp"
Sep 07 11:51:31.120625 ip-10-0-135-173 kubenswrapper[2563]: E0907 11:51:31.120506    2563 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 07 11:51:31.120625 ip-10-0-135-173 kubenswrapper[2563]: E0907 11:51:31.120583    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f302b236-f400-4839-8d56-edaf5de4bb75-insights-runtime-extractor-tls podName:f302b236-f400-4839-8d56-edaf5de4bb75 nodeName:}" failed. No retries permitted until 2026-09-07 11:51:33.120563045 +0000 UTC m=+264.639294835 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/f302b236-f400-4839-8d56-edaf5de4bb75-insights-runtime-extractor-tls") pod "insights-runtime-extractor-mchxp" (UID: "f302b236-f400-4839-8d56-edaf5de4bb75") : secret "insights-runtime-extractor-tls" not found
Sep 07 11:51:31.120625 ip-10-0-135-173 kubenswrapper[2563]: E0907 11:51:31.120512    2563 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 07 11:51:31.120736 ip-10-0-135-173 kubenswrapper[2563]: E0907 11:51:31.120649    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/f302b236-f400-4839-8d56-edaf5de4bb75-kube-rbac-proxy-cm podName:f302b236-f400-4839-8d56-edaf5de4bb75 nodeName:}" failed. No retries permitted until 2026-09-07 11:51:33.120634252 +0000 UTC m=+264.639366043 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/f302b236-f400-4839-8d56-edaf5de4bb75-kube-rbac-proxy-cm") pod "insights-runtime-extractor-mchxp" (UID: "f302b236-f400-4839-8d56-edaf5de4bb75") : configmap "kube-rbac-proxy" not found
Sep 07 11:51:31.188244 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:51:31.188213    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 11:51:31.322409 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:51:31.322376    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-75d99c484-mkhtk"]
Sep 07 11:51:31.325222 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:51:31.325206    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-75d99c484-mkhtk"
Sep 07 11:51:31.327222 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:51:31.327202    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\""
Sep 07 11:51:31.327476 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:51:31.327462    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\""
Sep 07 11:51:31.327629 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:51:31.327613    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\""
Sep 07 11:51:31.327694 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:51:31.327641    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-p9gwq\""
Sep 07 11:51:31.327749 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:51:31.327689    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\""
Sep 07 11:51:31.331559 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:51:31.331532    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-75d99c484-mkhtk"]
Sep 07 11:51:31.357429 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:51:31.357384    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 11:51:31.423076 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:51:31.423019    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/8cb95bab-17d3-4c08-bbed-e36c508c688d-signing-key\") pod \"service-ca-75d99c484-mkhtk\" (UID: \"8cb95bab-17d3-4c08-bbed-e36c508c688d\") " pod="openshift-service-ca/service-ca-75d99c484-mkhtk"
Sep 07 11:51:31.423076 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:51:31.423048    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2jsqz\" (UniqueName: \"kubernetes.io/projected/8cb95bab-17d3-4c08-bbed-e36c508c688d-kube-api-access-2jsqz\") pod \"service-ca-75d99c484-mkhtk\" (UID: \"8cb95bab-17d3-4c08-bbed-e36c508c688d\") " pod="openshift-service-ca/service-ca-75d99c484-mkhtk"
Sep 07 11:51:31.423202 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:51:31.423122    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/8cb95bab-17d3-4c08-bbed-e36c508c688d-signing-cabundle\") pod \"service-ca-75d99c484-mkhtk\" (UID: \"8cb95bab-17d3-4c08-bbed-e36c508c688d\") " pod="openshift-service-ca/service-ca-75d99c484-mkhtk"
Sep 07 11:51:31.523546 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:51:31.523519    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/8cb95bab-17d3-4c08-bbed-e36c508c688d-signing-cabundle\") pod \"service-ca-75d99c484-mkhtk\" (UID: \"8cb95bab-17d3-4c08-bbed-e36c508c688d\") " pod="openshift-service-ca/service-ca-75d99c484-mkhtk"
Sep 07 11:51:31.523645 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:51:31.523585    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/8cb95bab-17d3-4c08-bbed-e36c508c688d-signing-key\") pod \"service-ca-75d99c484-mkhtk\" (UID: \"8cb95bab-17d3-4c08-bbed-e36c508c688d\") " pod="openshift-service-ca/service-ca-75d99c484-mkhtk"
Sep 07 11:51:31.523645 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:51:31.523620    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2jsqz\" (UniqueName: \"kubernetes.io/projected/8cb95bab-17d3-4c08-bbed-e36c508c688d-kube-api-access-2jsqz\") pod \"service-ca-75d99c484-mkhtk\" (UID: \"8cb95bab-17d3-4c08-bbed-e36c508c688d\") " pod="openshift-service-ca/service-ca-75d99c484-mkhtk"
Sep 07 11:51:31.524149 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:51:31.524124    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/8cb95bab-17d3-4c08-bbed-e36c508c688d-signing-cabundle\") pod \"service-ca-75d99c484-mkhtk\" (UID: \"8cb95bab-17d3-4c08-bbed-e36c508c688d\") " pod="openshift-service-ca/service-ca-75d99c484-mkhtk"
Sep 07 11:51:31.525853 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:51:31.525833    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/8cb95bab-17d3-4c08-bbed-e36c508c688d-signing-key\") pod \"service-ca-75d99c484-mkhtk\" (UID: \"8cb95bab-17d3-4c08-bbed-e36c508c688d\") " pod="openshift-service-ca/service-ca-75d99c484-mkhtk"
Sep 07 11:51:31.530982 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:51:31.530958    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2jsqz\" (UniqueName: \"kubernetes.io/projected/8cb95bab-17d3-4c08-bbed-e36c508c688d-kube-api-access-2jsqz\") pod \"service-ca-75d99c484-mkhtk\" (UID: \"8cb95bab-17d3-4c08-bbed-e36c508c688d\") " pod="openshift-service-ca/service-ca-75d99c484-mkhtk"
Sep 07 11:51:31.635303 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:51:31.635274    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-75d99c484-mkhtk"
Sep 07 11:51:31.900115 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:51:31.900079    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-75d99c484-mkhtk"]
Sep 07 11:51:31.903401 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:51:31.903374    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8cb95bab_17d3_4c08_bbed_e36c508c688d.slice/crio-5e0cf30e2bc9d06f5bc64db35410cfb51365ae920d07bf4dee17e5fd22beb9f4 WatchSource:0}: Error finding container 5e0cf30e2bc9d06f5bc64db35410cfb51365ae920d07bf4dee17e5fd22beb9f4: Status 404 returned error can't find the container with id 5e0cf30e2bc9d06f5bc64db35410cfb51365ae920d07bf4dee17e5fd22beb9f4
Sep 07 11:51:32.004981 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:51:32.004956    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-hxstl_5f723277-6450-4fd8-8f5d-01d9f9f6814b/dns-node-resolver/0.log"
Sep 07 11:51:32.229303 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:51:32.229273    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 11:51:32.833432 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:51:32.833378    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-75d99c484-mkhtk" event={"ID":"8cb95bab-17d3-4c08-bbed-e36c508c688d","Type":"ContainerStarted","Data":"5e0cf30e2bc9d06f5bc64db35410cfb51365ae920d07bf4dee17e5fd22beb9f4"}
Sep 07 11:51:32.933708 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:51:32.933663    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9ae06e91-9199-4863-a2ab-aa8397e8cbbb-registry-tls\") pod \"image-registry-676764f5b5-dsf9f\" (UID: \"9ae06e91-9199-4863-a2ab-aa8397e8cbbb\") " pod="openshift-image-registry/image-registry-676764f5b5-dsf9f"
Sep 07 11:51:32.934161 ip-10-0-135-173 kubenswrapper[2563]: E0907 11:51:32.933836    2563 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 07 11:51:32.934161 ip-10-0-135-173 kubenswrapper[2563]: E0907 11:51:32.933857    2563 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-676764f5b5-dsf9f: secret "image-registry-tls" not found
Sep 07 11:51:32.934161 ip-10-0-135-173 kubenswrapper[2563]: E0907 11:51:32.933928    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9ae06e91-9199-4863-a2ab-aa8397e8cbbb-registry-tls podName:9ae06e91-9199-4863-a2ab-aa8397e8cbbb nodeName:}" failed. No retries permitted until 2026-09-07 11:51:40.933905901 +0000 UTC m=+272.452637707 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/9ae06e91-9199-4863-a2ab-aa8397e8cbbb-registry-tls") pod "image-registry-676764f5b5-dsf9f" (UID: "9ae06e91-9199-4863-a2ab-aa8397e8cbbb") : secret "image-registry-tls" not found
Sep 07 11:51:33.135594 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:51:33.135499    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f302b236-f400-4839-8d56-edaf5de4bb75-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-mchxp\" (UID: \"f302b236-f400-4839-8d56-edaf5de4bb75\") " pod="openshift-insights/insights-runtime-extractor-mchxp"
Sep 07 11:51:33.135754 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:51:33.135621    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f302b236-f400-4839-8d56-edaf5de4bb75-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-mchxp\" (UID: \"f302b236-f400-4839-8d56-edaf5de4bb75\") " pod="openshift-insights/insights-runtime-extractor-mchxp"
Sep 07 11:51:33.135754 ip-10-0-135-173 kubenswrapper[2563]: E0907 11:51:33.135686    2563 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 07 11:51:33.135754 ip-10-0-135-173 kubenswrapper[2563]: E0907 11:51:33.135713    2563 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 07 11:51:33.135894 ip-10-0-135-173 kubenswrapper[2563]: E0907 11:51:33.135768    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f302b236-f400-4839-8d56-edaf5de4bb75-insights-runtime-extractor-tls podName:f302b236-f400-4839-8d56-edaf5de4bb75 nodeName:}" failed. No retries permitted until 2026-09-07 11:51:37.135747265 +0000 UTC m=+268.654479056 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/f302b236-f400-4839-8d56-edaf5de4bb75-insights-runtime-extractor-tls") pod "insights-runtime-extractor-mchxp" (UID: "f302b236-f400-4839-8d56-edaf5de4bb75") : secret "insights-runtime-extractor-tls" not found
Sep 07 11:51:33.135894 ip-10-0-135-173 kubenswrapper[2563]: E0907 11:51:33.135791    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/f302b236-f400-4839-8d56-edaf5de4bb75-kube-rbac-proxy-cm podName:f302b236-f400-4839-8d56-edaf5de4bb75 nodeName:}" failed. No retries permitted until 2026-09-07 11:51:37.135780918 +0000 UTC m=+268.654512711 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/f302b236-f400-4839-8d56-edaf5de4bb75-kube-rbac-proxy-cm") pod "insights-runtime-extractor-mchxp" (UID: "f302b236-f400-4839-8d56-edaf5de4bb75") : configmap "kube-rbac-proxy" not found
Sep 07 11:51:33.404152 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:51:33.404069    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-wrd4z_b9aae943-4d29-4c1e-909f-0a1c034299f8/node-ca/0.log"
Sep 07 11:51:33.802183 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:51:33.802155    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-nv8dv_d1606d20-6381-462c-9cbe-b65edc598e2f/serve-healthcheck-canary/0.log"
Sep 07 11:51:33.838376 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:51:33.838344    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-75d99c484-mkhtk" event={"ID":"8cb95bab-17d3-4c08-bbed-e36c508c688d","Type":"ContainerStarted","Data":"36c4e683abf6b53f11d1829c66ed635e104d00b61e6c929a4bad39a5c58f9b30"}
Sep 07 11:51:33.851767 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:51:33.851727    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-75d99c484-mkhtk" podStartSLOduration=1.202531445 podStartE2EDuration="2.851708343s" totalImagesPullingTime="1.649176898s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 11:51:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 11:51:33.851170833 +0000 UTC m=+265.369902647" watchObservedRunningTime="2026-09-07 11:51:33.851708343 +0000 UTC m=+265.370440157"
Sep 07 11:51:34.503215 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:51:34.503180    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 11:51:34.774947 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:51:34.774865    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 11:51:34.876286 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:51:34.876259    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 11:51:35.132106 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:51:35.132007    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 11:51:35.231620 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:51:35.231592    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 11:51:35.283124 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:51:35.283054    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 11:51:35.346618 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:51:35.346593    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 11:51:35.448270 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:51:35.448237    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 11:51:35.513196 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:51:35.513167    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-85b454874b-4nfbc"
Sep 07 11:51:35.513561 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:51:35.513231    2563 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-85b454874b-4nfbc"
Sep 07 11:51:35.513667 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:51:35.513652    2563 scope.go:117] "RemoveContainer" containerID="b8346748a923f5cd04b940accc239af84a8a0c117c969c7c1ca982102a7daf9b"
Sep 07 11:51:35.513861 ip-10-0-135-173 kubenswrapper[2563]: E0907 11:51:35.513840    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-85b454874b-4nfbc_openshift-console-operator(b34ca85d-ac51-4724-a508-1026106b3f3a)\"" pod="openshift-console-operator/console-operator-85b454874b-4nfbc" podUID="b34ca85d-ac51-4724-a508-1026106b3f3a"
Sep 07 11:51:35.625297 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:51:35.625270    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 11:51:36.635168 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:51:36.635138    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 11:51:37.029769 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:51:37.029730    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 11:51:37.166959 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:51:37.166931    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f302b236-f400-4839-8d56-edaf5de4bb75-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-mchxp\" (UID: \"f302b236-f400-4839-8d56-edaf5de4bb75\") " pod="openshift-insights/insights-runtime-extractor-mchxp"
Sep 07 11:51:37.167127 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:51:37.167002    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f302b236-f400-4839-8d56-edaf5de4bb75-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-mchxp\" (UID: \"f302b236-f400-4839-8d56-edaf5de4bb75\") " pod="openshift-insights/insights-runtime-extractor-mchxp"
Sep 07 11:51:37.167127 ip-10-0-135-173 kubenswrapper[2563]: E0907 11:51:37.167073    2563 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 07 11:51:37.167205 ip-10-0-135-173 kubenswrapper[2563]: E0907 11:51:37.167174    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/f302b236-f400-4839-8d56-edaf5de4bb75-kube-rbac-proxy-cm podName:f302b236-f400-4839-8d56-edaf5de4bb75 nodeName:}" failed. No retries permitted until 2026-09-07 11:51:45.167160927 +0000 UTC m=+276.685892732 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/f302b236-f400-4839-8d56-edaf5de4bb75-kube-rbac-proxy-cm") pod "insights-runtime-extractor-mchxp" (UID: "f302b236-f400-4839-8d56-edaf5de4bb75") : configmap "kube-rbac-proxy" not found
Sep 07 11:51:37.167205 ip-10-0-135-173 kubenswrapper[2563]: E0907 11:51:37.167074    2563 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 07 11:51:37.167286 ip-10-0-135-173 kubenswrapper[2563]: E0907 11:51:37.167207    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f302b236-f400-4839-8d56-edaf5de4bb75-insights-runtime-extractor-tls podName:f302b236-f400-4839-8d56-edaf5de4bb75 nodeName:}" failed. No retries permitted until 2026-09-07 11:51:45.167201083 +0000 UTC m=+276.685932874 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/f302b236-f400-4839-8d56-edaf5de4bb75-insights-runtime-extractor-tls") pod "insights-runtime-extractor-mchxp" (UID: "f302b236-f400-4839-8d56-edaf5de4bb75") : secret "insights-runtime-extractor-tls" not found
Sep 07 11:51:37.230532 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:51:37.230501    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 11:51:37.430599 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:51:37.430494    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 11:51:37.631752 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:51:37.631705    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 11:51:38.408169 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:51:38.408140    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 11:51:38.608880 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:51:38.608849    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 11:51:38.659464 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:51:38.659395    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 11:51:38.827400 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:51:38.827369    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 11:51:39.029034 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:51:39.029004    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 11:51:39.430154 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:51:39.430072    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 11:51:40.038989 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:51:40.038950    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 11:51:40.626223 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:51:40.626187    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 11:51:40.827052 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:51:40.827020    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 11:51:40.984782 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:51:40.984745    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 11:51:40.997752 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:51:40.997724    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9ae06e91-9199-4863-a2ab-aa8397e8cbbb-registry-tls\") pod \"image-registry-676764f5b5-dsf9f\" (UID: \"9ae06e91-9199-4863-a2ab-aa8397e8cbbb\") " pod="openshift-image-registry/image-registry-676764f5b5-dsf9f"
Sep 07 11:51:41.000120 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:51:41.000099    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9ae06e91-9199-4863-a2ab-aa8397e8cbbb-registry-tls\") pod \"image-registry-676764f5b5-dsf9f\" (UID: \"9ae06e91-9199-4863-a2ab-aa8397e8cbbb\") " pod="openshift-image-registry/image-registry-676764f5b5-dsf9f"
Sep 07 11:51:41.085911 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:51:41.085885    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 11:51:41.117891 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:51:41.117862    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-676764f5b5-dsf9f"
Sep 07 11:51:41.390192 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:51:41.390123    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-676764f5b5-dsf9f"]
Sep 07 11:51:41.393117 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:51:41.393090    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9ae06e91_9199_4863_a2ab_aa8397e8cbbb.slice/crio-b8962cbfcdc6ac731ce93cea66541f8ceecac8294a71a703f6cdf9cd99db88c2 WatchSource:0}: Error finding container b8962cbfcdc6ac731ce93cea66541f8ceecac8294a71a703f6cdf9cd99db88c2: Status 404 returned error can't find the container with id b8962cbfcdc6ac731ce93cea66541f8ceecac8294a71a703f6cdf9cd99db88c2
Sep 07 11:51:41.429993 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:51:41.429961    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 11:51:41.867108 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:51:41.867074    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-676764f5b5-dsf9f" event={"ID":"9ae06e91-9199-4863-a2ab-aa8397e8cbbb","Type":"ContainerStarted","Data":"d88e772a3f606d24efae5bc9e4925f3260099ffe19b7e03e6eb039253688b227"}
Sep 07 11:51:41.867500 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:51:41.867132    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-676764f5b5-dsf9f" event={"ID":"9ae06e91-9199-4863-a2ab-aa8397e8cbbb","Type":"ContainerStarted","Data":"b8962cbfcdc6ac731ce93cea66541f8ceecac8294a71a703f6cdf9cd99db88c2"}
Sep 07 11:51:41.867500 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:51:41.867199    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-676764f5b5-dsf9f"
Sep 07 11:51:41.883732 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:51:41.883689    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-676764f5b5-dsf9f" podStartSLOduration=16.883675259 podStartE2EDuration="16.883675259s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 11:51:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 11:51:41.88234432 +0000 UTC m=+273.401076132" watchObservedRunningTime="2026-09-07 11:51:41.883675259 +0000 UTC m=+273.402407072"
Sep 07 11:51:42.029927 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:51:42.029897    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 11:51:42.289356 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:51:42.289325    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 11:51:42.631476 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:51:42.631391    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 11:51:42.823055 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:51:42.823025    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 11:51:43.308033 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:51:43.308003    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 11:51:43.424647 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:51:43.424610    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 11:51:43.934622 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:51:43.934589    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 11:51:44.035271 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:51:44.035230    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 11:51:44.424803 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:51:44.424759    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 11:51:44.829079 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:51:44.829036    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 11:51:45.029992 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:51:45.029961    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 11:51:45.232761 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:51:45.232728    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 11:51:45.235621 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:51:45.235592    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f302b236-f400-4839-8d56-edaf5de4bb75-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-mchxp\" (UID: \"f302b236-f400-4839-8d56-edaf5de4bb75\") " pod="openshift-insights/insights-runtime-extractor-mchxp"
Sep 07 11:51:45.235739 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:51:45.235648    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f302b236-f400-4839-8d56-edaf5de4bb75-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-mchxp\" (UID: \"f302b236-f400-4839-8d56-edaf5de4bb75\") " pod="openshift-insights/insights-runtime-extractor-mchxp"
Sep 07 11:51:45.236096 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:51:45.236080    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f302b236-f400-4839-8d56-edaf5de4bb75-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-mchxp\" (UID: \"f302b236-f400-4839-8d56-edaf5de4bb75\") " pod="openshift-insights/insights-runtime-extractor-mchxp"
Sep 07 11:51:45.237778 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:51:45.237760    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f302b236-f400-4839-8d56-edaf5de4bb75-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-mchxp\" (UID: \"f302b236-f400-4839-8d56-edaf5de4bb75\") " pod="openshift-insights/insights-runtime-extractor-mchxp"
Sep 07 11:51:45.350330 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:51:45.350290    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-mchxp"
Sep 07 11:51:45.617384 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:51:45.617308    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-mchxp"]
Sep 07 11:51:45.620871 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:51:45.620844    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf302b236_f400_4839_8d56_edaf5de4bb75.slice/crio-116efb826191b9141883015dd27716f1fd9c6f43c192940b7c8e043618585505 WatchSource:0}: Error finding container 116efb826191b9141883015dd27716f1fd9c6f43c192940b7c8e043618585505: Status 404 returned error can't find the container with id 116efb826191b9141883015dd27716f1fd9c6f43c192940b7c8e043618585505
Sep 07 11:51:45.880742 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:51:45.880660    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-mchxp" event={"ID":"f302b236-f400-4839-8d56-edaf5de4bb75","Type":"ContainerStarted","Data":"2f1d4ebf6f39f70975cb7c53aab41bb4ec7d1ac8913fc108754abf884ea7f96d"}
Sep 07 11:51:45.880742 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:51:45.880712    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-mchxp" event={"ID":"f302b236-f400-4839-8d56-edaf5de4bb75","Type":"ContainerStarted","Data":"116efb826191b9141883015dd27716f1fd9c6f43c192940b7c8e043618585505"}
Sep 07 11:51:46.885953 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:51:46.885914    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-mchxp" event={"ID":"f302b236-f400-4839-8d56-edaf5de4bb75","Type":"ContainerStarted","Data":"3d0d4a2236248ac32a1cc9cf7b57c25bde7d03524b549220223b835694b52f64"}
Sep 07 11:51:47.541326 ip-10-0-135-173 kubenswrapper[2563]: E0907 11:51:47.541280    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-q9tkc" podUID="46cab3bd-0f24-46e2-910c-b6e9a5f6ae5e"
Sep 07 11:51:47.892149 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:51:47.892087    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-q9tkc"
Sep 07 11:51:47.892528 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:51:47.892080    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-mchxp" event={"ID":"f302b236-f400-4839-8d56-edaf5de4bb75","Type":"ContainerStarted","Data":"7bf5f03c675a94c2e552754f5d101414c0819260a1f3cad27243e6348b2c39db"}
Sep 07 11:51:47.907737 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:51:47.907695    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-mchxp" podStartSLOduration=16.975761726 podStartE2EDuration="18.90768163s" totalImagesPullingTime="1.931919904s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-07 11:51:29 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 11:51:47.906978756 +0000 UTC m=+279.425710573" watchObservedRunningTime="2026-09-07 11:51:47.90768163 +0000 UTC m=+279.426413442"
Sep 07 11:51:48.034760 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:51:48.034736    2563 scope.go:117] "RemoveContainer" containerID="b8346748a923f5cd04b940accc239af84a8a0c117c969c7c1ca982102a7daf9b"
Sep 07 11:51:48.896881 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:51:48.896853    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-4nfbc_b34ca85d-ac51-4724-a508-1026106b3f3a/console-operator/2.log"
Sep 07 11:51:48.897329 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:51:48.897313    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-4nfbc_b34ca85d-ac51-4724-a508-1026106b3f3a/console-operator/1.log"
Sep 07 11:51:48.897389 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:51:48.897348    2563 generic.go:356] "Generic (PLEG): container finished" podID="b34ca85d-ac51-4724-a508-1026106b3f3a" containerID="725f5bd2022473854b9b6803069247eb819f5bf8e09a47410ca95ea96dacab2b" exitCode=255
Sep 07 11:51:48.897468 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:51:48.897446    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-85b454874b-4nfbc" event={"ID":"b34ca85d-ac51-4724-a508-1026106b3f3a","Type":"ContainerDied","Data":"725f5bd2022473854b9b6803069247eb819f5bf8e09a47410ca95ea96dacab2b"}
Sep 07 11:51:48.897523 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:51:48.897510    2563 scope.go:117] "RemoveContainer" containerID="b8346748a923f5cd04b940accc239af84a8a0c117c969c7c1ca982102a7daf9b"
Sep 07 11:51:48.897997 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:51:48.897976    2563 scope.go:117] "RemoveContainer" containerID="725f5bd2022473854b9b6803069247eb819f5bf8e09a47410ca95ea96dacab2b"
Sep 07 11:51:48.898182 ip-10-0-135-173 kubenswrapper[2563]: E0907 11:51:48.898155    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-85b454874b-4nfbc_openshift-console-operator(b34ca85d-ac51-4724-a508-1026106b3f3a)\"" pod="openshift-console-operator/console-operator-85b454874b-4nfbc" podUID="b34ca85d-ac51-4724-a508-1026106b3f3a"
Sep 07 11:51:49.901871 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:51:49.901846    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-4nfbc_b34ca85d-ac51-4724-a508-1026106b3f3a/console-operator/2.log"
Sep 07 11:51:51.280570 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:51:51.280535    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/46cab3bd-0f24-46e2-910c-b6e9a5f6ae5e-metrics-tls\") pod \"dns-default-q9tkc\" (UID: \"46cab3bd-0f24-46e2-910c-b6e9a5f6ae5e\") " pod="openshift-dns/dns-default-q9tkc"
Sep 07 11:51:51.282728 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:51:51.282703    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/46cab3bd-0f24-46e2-910c-b6e9a5f6ae5e-metrics-tls\") pod \"dns-default-q9tkc\" (UID: \"46cab3bd-0f24-46e2-910c-b6e9a5f6ae5e\") " pod="openshift-dns/dns-default-q9tkc"
Sep 07 11:51:51.494689 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:51:51.494662    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-kd29r\""
Sep 07 11:51:51.503902 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:51:51.503883    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-q9tkc"
Sep 07 11:51:51.771994 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:51:51.771963    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-q9tkc"]
Sep 07 11:51:51.774814 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:51:51.774785    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod46cab3bd_0f24_46e2_910c_b6e9a5f6ae5e.slice/crio-51f718cf7f370c73145daf1961f5e42ef982341d302d84bd35ae516139774421 WatchSource:0}: Error finding container 51f718cf7f370c73145daf1961f5e42ef982341d302d84bd35ae516139774421: Status 404 returned error can't find the container with id 51f718cf7f370c73145daf1961f5e42ef982341d302d84bd35ae516139774421
Sep 07 11:51:51.887907 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:51:51.887824    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 11:51:51.909035 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:51:51.909007    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-q9tkc" event={"ID":"46cab3bd-0f24-46e2-910c-b6e9a5f6ae5e","Type":"ContainerStarted","Data":"51f718cf7f370c73145daf1961f5e42ef982341d302d84bd35ae516139774421"}
Sep 07 11:51:51.941393 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:51:51.941365    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 11:51:52.553506 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:51:52.553468    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-676764f5b5-dsf9f"]
Sep 07 11:51:52.588889 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:51:52.588863    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-74ccddf898-jp5pm"]
Sep 07 11:51:52.592649 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:51:52.592614    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-74ccddf898-jp5pm"
Sep 07 11:51:52.605527 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:51:52.605488    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-74ccddf898-jp5pm"]
Sep 07 11:51:52.637687 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:51:52.637636    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 11:51:52.690105 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:51:52.690072    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a5bfeb4e-09b1-4bdb-8692-e1e6d8b5bd73-registry-tls\") pod \"image-registry-74ccddf898-jp5pm\" (UID: \"a5bfeb4e-09b1-4bdb-8692-e1e6d8b5bd73\") " pod="openshift-image-registry/image-registry-74ccddf898-jp5pm"
Sep 07 11:51:52.690265 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:51:52.690112    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/a5bfeb4e-09b1-4bdb-8692-e1e6d8b5bd73-installation-pull-secrets\") pod \"image-registry-74ccddf898-jp5pm\" (UID: \"a5bfeb4e-09b1-4bdb-8692-e1e6d8b5bd73\") " pod="openshift-image-registry/image-registry-74ccddf898-jp5pm"
Sep 07 11:51:52.690265 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:51:52.690145    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a5bfeb4e-09b1-4bdb-8692-e1e6d8b5bd73-bound-sa-token\") pod \"image-registry-74ccddf898-jp5pm\" (UID: \"a5bfeb4e-09b1-4bdb-8692-e1e6d8b5bd73\") " pod="openshift-image-registry/image-registry-74ccddf898-jp5pm"
Sep 07 11:51:52.690265 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:51:52.690227    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sxp4s\" (UniqueName: \"kubernetes.io/projected/a5bfeb4e-09b1-4bdb-8692-e1e6d8b5bd73-kube-api-access-sxp4s\") pod \"image-registry-74ccddf898-jp5pm\" (UID: \"a5bfeb4e-09b1-4bdb-8692-e1e6d8b5bd73\") " pod="openshift-image-registry/image-registry-74ccddf898-jp5pm"
Sep 07 11:51:52.690265 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:51:52.690256    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/a5bfeb4e-09b1-4bdb-8692-e1e6d8b5bd73-image-registry-private-configuration\") pod \"image-registry-74ccddf898-jp5pm\" (UID: \"a5bfeb4e-09b1-4bdb-8692-e1e6d8b5bd73\") " pod="openshift-image-registry/image-registry-74ccddf898-jp5pm"
Sep 07 11:51:52.690497 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:51:52.690290    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/a5bfeb4e-09b1-4bdb-8692-e1e6d8b5bd73-ca-trust-extracted\") pod \"image-registry-74ccddf898-jp5pm\" (UID: \"a5bfeb4e-09b1-4bdb-8692-e1e6d8b5bd73\") " pod="openshift-image-registry/image-registry-74ccddf898-jp5pm"
Sep 07 11:51:52.690497 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:51:52.690317    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/a5bfeb4e-09b1-4bdb-8692-e1e6d8b5bd73-trusted-ca\") pod \"image-registry-74ccddf898-jp5pm\" (UID: \"a5bfeb4e-09b1-4bdb-8692-e1e6d8b5bd73\") " pod="openshift-image-registry/image-registry-74ccddf898-jp5pm"
Sep 07 11:51:52.690497 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:51:52.690343    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/a5bfeb4e-09b1-4bdb-8692-e1e6d8b5bd73-registry-certificates\") pod \"image-registry-74ccddf898-jp5pm\" (UID: \"a5bfeb4e-09b1-4bdb-8692-e1e6d8b5bd73\") " pod="openshift-image-registry/image-registry-74ccddf898-jp5pm"
Sep 07 11:51:52.730304 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:51:52.730275    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 11:51:52.791055 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:51:52.791024    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/a5bfeb4e-09b1-4bdb-8692-e1e6d8b5bd73-registry-certificates\") pod \"image-registry-74ccddf898-jp5pm\" (UID: \"a5bfeb4e-09b1-4bdb-8692-e1e6d8b5bd73\") " pod="openshift-image-registry/image-registry-74ccddf898-jp5pm"
Sep 07 11:51:52.791210 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:51:52.791084    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a5bfeb4e-09b1-4bdb-8692-e1e6d8b5bd73-registry-tls\") pod \"image-registry-74ccddf898-jp5pm\" (UID: \"a5bfeb4e-09b1-4bdb-8692-e1e6d8b5bd73\") " pod="openshift-image-registry/image-registry-74ccddf898-jp5pm"
Sep 07 11:51:52.791210 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:51:52.791102    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/a5bfeb4e-09b1-4bdb-8692-e1e6d8b5bd73-installation-pull-secrets\") pod \"image-registry-74ccddf898-jp5pm\" (UID: \"a5bfeb4e-09b1-4bdb-8692-e1e6d8b5bd73\") " pod="openshift-image-registry/image-registry-74ccddf898-jp5pm"
Sep 07 11:51:52.791210 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:51:52.791120    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a5bfeb4e-09b1-4bdb-8692-e1e6d8b5bd73-bound-sa-token\") pod \"image-registry-74ccddf898-jp5pm\" (UID: \"a5bfeb4e-09b1-4bdb-8692-e1e6d8b5bd73\") " pod="openshift-image-registry/image-registry-74ccddf898-jp5pm"
Sep 07 11:51:52.791210 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:51:52.791149    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sxp4s\" (UniqueName: \"kubernetes.io/projected/a5bfeb4e-09b1-4bdb-8692-e1e6d8b5bd73-kube-api-access-sxp4s\") pod \"image-registry-74ccddf898-jp5pm\" (UID: \"a5bfeb4e-09b1-4bdb-8692-e1e6d8b5bd73\") " pod="openshift-image-registry/image-registry-74ccddf898-jp5pm"
Sep 07 11:51:52.791210 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:51:52.791168    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/a5bfeb4e-09b1-4bdb-8692-e1e6d8b5bd73-image-registry-private-configuration\") pod \"image-registry-74ccddf898-jp5pm\" (UID: \"a5bfeb4e-09b1-4bdb-8692-e1e6d8b5bd73\") " pod="openshift-image-registry/image-registry-74ccddf898-jp5pm"
Sep 07 11:51:52.791210 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:51:52.791199    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/a5bfeb4e-09b1-4bdb-8692-e1e6d8b5bd73-ca-trust-extracted\") pod \"image-registry-74ccddf898-jp5pm\" (UID: \"a5bfeb4e-09b1-4bdb-8692-e1e6d8b5bd73\") " pod="openshift-image-registry/image-registry-74ccddf898-jp5pm"
Sep 07 11:51:52.791539 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:51:52.791225    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/a5bfeb4e-09b1-4bdb-8692-e1e6d8b5bd73-trusted-ca\") pod \"image-registry-74ccddf898-jp5pm\" (UID: \"a5bfeb4e-09b1-4bdb-8692-e1e6d8b5bd73\") " pod="openshift-image-registry/image-registry-74ccddf898-jp5pm"
Sep 07 11:51:52.791859 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:51:52.791829    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/a5bfeb4e-09b1-4bdb-8692-e1e6d8b5bd73-ca-trust-extracted\") pod \"image-registry-74ccddf898-jp5pm\" (UID: \"a5bfeb4e-09b1-4bdb-8692-e1e6d8b5bd73\") " pod="openshift-image-registry/image-registry-74ccddf898-jp5pm"
Sep 07 11:51:52.791982 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:51:52.791949    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/a5bfeb4e-09b1-4bdb-8692-e1e6d8b5bd73-registry-certificates\") pod \"image-registry-74ccddf898-jp5pm\" (UID: \"a5bfeb4e-09b1-4bdb-8692-e1e6d8b5bd73\") " pod="openshift-image-registry/image-registry-74ccddf898-jp5pm"
Sep 07 11:51:52.792097 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:51:52.792078    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/a5bfeb4e-09b1-4bdb-8692-e1e6d8b5bd73-trusted-ca\") pod \"image-registry-74ccddf898-jp5pm\" (UID: \"a5bfeb4e-09b1-4bdb-8692-e1e6d8b5bd73\") " pod="openshift-image-registry/image-registry-74ccddf898-jp5pm"
Sep 07 11:51:52.793842 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:51:52.793813    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/a5bfeb4e-09b1-4bdb-8692-e1e6d8b5bd73-installation-pull-secrets\") pod \"image-registry-74ccddf898-jp5pm\" (UID: \"a5bfeb4e-09b1-4bdb-8692-e1e6d8b5bd73\") " pod="openshift-image-registry/image-registry-74ccddf898-jp5pm"
Sep 07 11:51:52.794004 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:51:52.793972    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a5bfeb4e-09b1-4bdb-8692-e1e6d8b5bd73-registry-tls\") pod \"image-registry-74ccddf898-jp5pm\" (UID: \"a5bfeb4e-09b1-4bdb-8692-e1e6d8b5bd73\") " pod="openshift-image-registry/image-registry-74ccddf898-jp5pm"
Sep 07 11:51:52.794081 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:51:52.794018    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/a5bfeb4e-09b1-4bdb-8692-e1e6d8b5bd73-image-registry-private-configuration\") pod \"image-registry-74ccddf898-jp5pm\" (UID: \"a5bfeb4e-09b1-4bdb-8692-e1e6d8b5bd73\") " pod="openshift-image-registry/image-registry-74ccddf898-jp5pm"
Sep 07 11:51:52.802097 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:51:52.802076    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a5bfeb4e-09b1-4bdb-8692-e1e6d8b5bd73-bound-sa-token\") pod \"image-registry-74ccddf898-jp5pm\" (UID: \"a5bfeb4e-09b1-4bdb-8692-e1e6d8b5bd73\") " pod="openshift-image-registry/image-registry-74ccddf898-jp5pm"
Sep 07 11:51:52.802202 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:51:52.802105    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sxp4s\" (UniqueName: \"kubernetes.io/projected/a5bfeb4e-09b1-4bdb-8692-e1e6d8b5bd73-kube-api-access-sxp4s\") pod \"image-registry-74ccddf898-jp5pm\" (UID: \"a5bfeb4e-09b1-4bdb-8692-e1e6d8b5bd73\") " pod="openshift-image-registry/image-registry-74ccddf898-jp5pm"
Sep 07 11:51:52.903835 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:51:52.903748    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-74ccddf898-jp5pm"
Sep 07 11:51:53.495134 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:51:53.495063    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-74ccddf898-jp5pm"]
Sep 07 11:51:53.498192 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:51:53.498169    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda5bfeb4e_09b1_4bdb_8692_e1e6d8b5bd73.slice/crio-b87a6f58466f9eebb90ccfc99711dbb6a32ce7ed1cd69b150515396060a04374 WatchSource:0}: Error finding container b87a6f58466f9eebb90ccfc99711dbb6a32ce7ed1cd69b150515396060a04374: Status 404 returned error can't find the container with id b87a6f58466f9eebb90ccfc99711dbb6a32ce7ed1cd69b150515396060a04374
Sep 07 11:51:53.917257 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:51:53.917167    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-74ccddf898-jp5pm" event={"ID":"a5bfeb4e-09b1-4bdb-8692-e1e6d8b5bd73","Type":"ContainerStarted","Data":"be98346d35fb1ffc2cca89c8a61c50ba494503e7e01faa1af5ee8d6ad3902a7b"}
Sep 07 11:51:53.917697 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:51:53.917265    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-74ccddf898-jp5pm"
Sep 07 11:51:53.917697 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:51:53.917289    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-74ccddf898-jp5pm" event={"ID":"a5bfeb4e-09b1-4bdb-8692-e1e6d8b5bd73","Type":"ContainerStarted","Data":"b87a6f58466f9eebb90ccfc99711dbb6a32ce7ed1cd69b150515396060a04374"}
Sep 07 11:51:53.918903 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:51:53.918879    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-q9tkc" event={"ID":"46cab3bd-0f24-46e2-910c-b6e9a5f6ae5e","Type":"ContainerStarted","Data":"05cd9f697a1af64e81147bae7d24d9925e2c1c0c18c2441d9c3b92947f653a3c"}
Sep 07 11:51:53.918990 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:51:53.918921    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-q9tkc" event={"ID":"46cab3bd-0f24-46e2-910c-b6e9a5f6ae5e","Type":"ContainerStarted","Data":"b59de847939548a465b56da0f3cade64a37aa1d58284a3d883c90531c1ffc1e1"}
Sep 07 11:51:53.919033 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:51:53.919014    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-q9tkc"
Sep 07 11:51:53.934018 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:51:53.933977    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-74ccddf898-jp5pm" podStartSLOduration=1.933964848 podStartE2EDuration="1.933964848s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 11:51:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 11:51:53.932713137 +0000 UTC m=+285.451444950" watchObservedRunningTime="2026-09-07 11:51:53.933964848 +0000 UTC m=+285.452696660"
Sep 07 11:51:53.946917 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:51:53.946876    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-q9tkc" podStartSLOduration=251.443446723 podStartE2EDuration="4m12.946866736s" totalImagesPullingTime="1.503420013s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 11:47:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 11:51:53.946034404 +0000 UTC m=+285.464766372" watchObservedRunningTime="2026-09-07 11:51:53.946866736 +0000 UTC m=+285.465598549"
Sep 07 11:51:55.513507 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:51:55.513477    2563 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-85b454874b-4nfbc"
Sep 07 11:51:55.513872 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:51:55.513527    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-85b454874b-4nfbc"
Sep 07 11:51:55.513872 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:51:55.513833    2563 scope.go:117] "RemoveContainer" containerID="725f5bd2022473854b9b6803069247eb819f5bf8e09a47410ca95ea96dacab2b"
Sep 07 11:51:55.514027 ip-10-0-135-173 kubenswrapper[2563]: E0907 11:51:55.514006    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-85b454874b-4nfbc_openshift-console-operator(b34ca85d-ac51-4724-a508-1026106b3f3a)\"" pod="openshift-console-operator/console-operator-85b454874b-4nfbc" podUID="b34ca85d-ac51-4724-a508-1026106b3f3a"
Sep 07 11:51:59.325400 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:51:59.325367    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 11:51:59.427053 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:51:59.427017    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 11:51:59.493097 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:51:59.493070    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 11:51:59.969758 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:51:59.969722    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 11:52:02.036848 ip-10-0-135-173 kubenswrapper[2563]: E0907 11:52:02.036795    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-xf2gj" podUID="3acaae8d-5930-49c6-9b9e-d286ac6a75d5"
Sep 07 11:52:02.538511 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:52:02.538469    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 11:52:02.558902 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:52:02.558874    2563 patch_prober.go:28] interesting pod/image-registry-676764f5b5-dsf9f container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Sep 07 11:52:02.559017 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:52:02.558921    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-676764f5b5-dsf9f" podUID="9ae06e91-9199-4863-a2ab-aa8397e8cbbb" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 07 11:52:02.626754 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:52:02.626719    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 11:52:03.168227 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:52:03.168189    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 11:52:03.923813 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:52:03.923784    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-q9tkc"
Sep 07 11:52:05.127767 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:52:05.127735    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 11:52:06.034438 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:52:06.034390    2563 scope.go:117] "RemoveContainer" containerID="725f5bd2022473854b9b6803069247eb819f5bf8e09a47410ca95ea96dacab2b"
Sep 07 11:52:06.034600 ip-10-0-135-173 kubenswrapper[2563]: E0907 11:52:06.034558    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-85b454874b-4nfbc_openshift-console-operator(b34ca85d-ac51-4724-a508-1026106b3f3a)\"" pod="openshift-console-operator/console-operator-85b454874b-4nfbc" podUID="b34ca85d-ac51-4724-a508-1026106b3f3a"
Sep 07 11:52:06.440437 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:52:06.440336    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 11:52:06.491487 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:52:06.491457    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 11:52:06.660431 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:52:06.660398    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 11:52:06.914890 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:52:06.914850    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 11:52:06.921300 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:52:06.921274    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-74cb57df7-wzrfr"]
Sep 07 11:52:06.923831 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:52:06.923806    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-66czn"]
Sep 07 11:52:06.923945 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:52:06.923929    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-74cb57df7-wzrfr"
Sep 07 11:52:06.926362 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:52:06.926343    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-66czn"
Sep 07 11:52:06.926628 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:52:06.926476    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\""
Sep 07 11:52:06.926628 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:52:06.926482    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Sep 07 11:52:06.927261 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:52:06.927095    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\""
Sep 07 11:52:06.927261 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:52:06.927134    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Sep 07 11:52:06.927261 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:52:06.927147    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Sep 07 11:52:06.927261 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:52:06.927200    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-ljdzk\""
Sep 07 11:52:06.927261 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:52:06.927154    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\""
Sep 07 11:52:06.928680 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:52:06.928660    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Sep 07 11:52:06.928863 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:52:06.928843    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-rw24l\""
Sep 07 11:52:06.929102 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:52:06.929076    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Sep 07 11:52:06.929181 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:52:06.929076    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Sep 07 11:52:06.932651 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:52:06.932631    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-74cb57df7-wzrfr"]
Sep 07 11:52:07.003403 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:52:07.003370    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 11:52:07.095548 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:52:07.095516    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/67bac748-66ed-4dcb-8a9c-1814e8074836-node-exporter-textfile\") pod \"node-exporter-66czn\" (UID: \"67bac748-66ed-4dcb-8a9c-1814e8074836\") " pod="openshift-monitoring/node-exporter-66czn"
Sep 07 11:52:07.095724 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:52:07.095563    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6wcm6\" (UniqueName: \"kubernetes.io/projected/67bac748-66ed-4dcb-8a9c-1814e8074836-kube-api-access-6wcm6\") pod \"node-exporter-66czn\" (UID: \"67bac748-66ed-4dcb-8a9c-1814e8074836\") " pod="openshift-monitoring/node-exporter-66czn"
Sep 07 11:52:07.095724 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:52:07.095618    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/67bac748-66ed-4dcb-8a9c-1814e8074836-node-exporter-tls\") pod \"node-exporter-66czn\" (UID: \"67bac748-66ed-4dcb-8a9c-1814e8074836\") " pod="openshift-monitoring/node-exporter-66czn"
Sep 07 11:52:07.095724 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:52:07.095650    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/67bac748-66ed-4dcb-8a9c-1814e8074836-node-exporter-accelerators-collector-config\") pod \"node-exporter-66czn\" (UID: \"67bac748-66ed-4dcb-8a9c-1814e8074836\") " pod="openshift-monitoring/node-exporter-66czn"
Sep 07 11:52:07.095724 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:52:07.095715    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ea883d51-c080-4b94-90e3-c5e26c612f53-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-74cb57df7-wzrfr\" (UID: \"ea883d51-c080-4b94-90e3-c5e26c612f53\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-wzrfr"
Sep 07 11:52:07.095890 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:52:07.095736    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/67bac748-66ed-4dcb-8a9c-1814e8074836-sys\") pod \"node-exporter-66czn\" (UID: \"67bac748-66ed-4dcb-8a9c-1814e8074836\") " pod="openshift-monitoring/node-exporter-66czn"
Sep 07 11:52:07.095890 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:52:07.095780    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/ea883d51-c080-4b94-90e3-c5e26c612f53-kube-state-metrics-tls\") pod \"kube-state-metrics-74cb57df7-wzrfr\" (UID: \"ea883d51-c080-4b94-90e3-c5e26c612f53\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-wzrfr"
Sep 07 11:52:07.095890 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:52:07.095806    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/67bac748-66ed-4dcb-8a9c-1814e8074836-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-66czn\" (UID: \"67bac748-66ed-4dcb-8a9c-1814e8074836\") " pod="openshift-monitoring/node-exporter-66czn"
Sep 07 11:52:07.095890 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:52:07.095828    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/67bac748-66ed-4dcb-8a9c-1814e8074836-root\") pod \"node-exporter-66czn\" (UID: \"67bac748-66ed-4dcb-8a9c-1814e8074836\") " pod="openshift-monitoring/node-exporter-66czn"
Sep 07 11:52:07.095890 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:52:07.095847    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/67bac748-66ed-4dcb-8a9c-1814e8074836-node-exporter-wtmp\") pod \"node-exporter-66czn\" (UID: \"67bac748-66ed-4dcb-8a9c-1814e8074836\") " pod="openshift-monitoring/node-exporter-66czn"
Sep 07 11:52:07.095890 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:52:07.095862    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/67bac748-66ed-4dcb-8a9c-1814e8074836-metrics-client-ca\") pod \"node-exporter-66czn\" (UID: \"67bac748-66ed-4dcb-8a9c-1814e8074836\") " pod="openshift-monitoring/node-exporter-66czn"
Sep 07 11:52:07.095890 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:52:07.095888    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/ea883d51-c080-4b94-90e3-c5e26c612f53-volume-directive-shadow\") pod \"kube-state-metrics-74cb57df7-wzrfr\" (UID: \"ea883d51-c080-4b94-90e3-c5e26c612f53\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-wzrfr"
Sep 07 11:52:07.096153 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:52:07.095918    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ea883d51-c080-4b94-90e3-c5e26c612f53-metrics-client-ca\") pod \"kube-state-metrics-74cb57df7-wzrfr\" (UID: \"ea883d51-c080-4b94-90e3-c5e26c612f53\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-wzrfr"
Sep 07 11:52:07.096153 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:52:07.095953    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/ea883d51-c080-4b94-90e3-c5e26c612f53-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-74cb57df7-wzrfr\" (UID: \"ea883d51-c080-4b94-90e3-c5e26c612f53\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-wzrfr"
Sep 07 11:52:07.096153 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:52:07.096022    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ztmb8\" (UniqueName: \"kubernetes.io/projected/ea883d51-c080-4b94-90e3-c5e26c612f53-kube-api-access-ztmb8\") pod \"kube-state-metrics-74cb57df7-wzrfr\" (UID: \"ea883d51-c080-4b94-90e3-c5e26c612f53\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-wzrfr"
Sep 07 11:52:07.103202 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:52:07.103166    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 11:52:07.196988 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:52:07.196963    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ea883d51-c080-4b94-90e3-c5e26c612f53-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-74cb57df7-wzrfr\" (UID: \"ea883d51-c080-4b94-90e3-c5e26c612f53\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-wzrfr"
Sep 07 11:52:07.197124 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:52:07.196992    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/67bac748-66ed-4dcb-8a9c-1814e8074836-sys\") pod \"node-exporter-66czn\" (UID: \"67bac748-66ed-4dcb-8a9c-1814e8074836\") " pod="openshift-monitoring/node-exporter-66czn"
Sep 07 11:52:07.197124 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:52:07.197012    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/ea883d51-c080-4b94-90e3-c5e26c612f53-kube-state-metrics-tls\") pod \"kube-state-metrics-74cb57df7-wzrfr\" (UID: \"ea883d51-c080-4b94-90e3-c5e26c612f53\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-wzrfr"
Sep 07 11:52:07.197124 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:52:07.197028    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/67bac748-66ed-4dcb-8a9c-1814e8074836-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-66czn\" (UID: \"67bac748-66ed-4dcb-8a9c-1814e8074836\") " pod="openshift-monitoring/node-exporter-66czn"
Sep 07 11:52:07.197124 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:52:07.197044    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/67bac748-66ed-4dcb-8a9c-1814e8074836-root\") pod \"node-exporter-66czn\" (UID: \"67bac748-66ed-4dcb-8a9c-1814e8074836\") " pod="openshift-monitoring/node-exporter-66czn"
Sep 07 11:52:07.197124 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:52:07.197063    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/67bac748-66ed-4dcb-8a9c-1814e8074836-node-exporter-wtmp\") pod \"node-exporter-66czn\" (UID: \"67bac748-66ed-4dcb-8a9c-1814e8074836\") " pod="openshift-monitoring/node-exporter-66czn"
Sep 07 11:52:07.197124 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:52:07.197085    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/67bac748-66ed-4dcb-8a9c-1814e8074836-metrics-client-ca\") pod \"node-exporter-66czn\" (UID: \"67bac748-66ed-4dcb-8a9c-1814e8074836\") " pod="openshift-monitoring/node-exporter-66czn"
Sep 07 11:52:07.197124 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:52:07.197104    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/ea883d51-c080-4b94-90e3-c5e26c612f53-volume-directive-shadow\") pod \"kube-state-metrics-74cb57df7-wzrfr\" (UID: \"ea883d51-c080-4b94-90e3-c5e26c612f53\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-wzrfr"
Sep 07 11:52:07.197505 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:52:07.197109    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/67bac748-66ed-4dcb-8a9c-1814e8074836-root\") pod \"node-exporter-66czn\" (UID: \"67bac748-66ed-4dcb-8a9c-1814e8074836\") " pod="openshift-monitoring/node-exporter-66czn"
Sep 07 11:52:07.197505 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:52:07.197125    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ea883d51-c080-4b94-90e3-c5e26c612f53-metrics-client-ca\") pod \"kube-state-metrics-74cb57df7-wzrfr\" (UID: \"ea883d51-c080-4b94-90e3-c5e26c612f53\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-wzrfr"
Sep 07 11:52:07.197505 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:52:07.197199    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/ea883d51-c080-4b94-90e3-c5e26c612f53-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-74cb57df7-wzrfr\" (UID: \"ea883d51-c080-4b94-90e3-c5e26c612f53\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-wzrfr"
Sep 07 11:52:07.197505 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:52:07.197239    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ztmb8\" (UniqueName: \"kubernetes.io/projected/ea883d51-c080-4b94-90e3-c5e26c612f53-kube-api-access-ztmb8\") pod \"kube-state-metrics-74cb57df7-wzrfr\" (UID: \"ea883d51-c080-4b94-90e3-c5e26c612f53\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-wzrfr"
Sep 07 11:52:07.197505 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:52:07.197288    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/67bac748-66ed-4dcb-8a9c-1814e8074836-node-exporter-textfile\") pod \"node-exporter-66czn\" (UID: \"67bac748-66ed-4dcb-8a9c-1814e8074836\") " pod="openshift-monitoring/node-exporter-66czn"
Sep 07 11:52:07.197505 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:52:07.197323    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6wcm6\" (UniqueName: \"kubernetes.io/projected/67bac748-66ed-4dcb-8a9c-1814e8074836-kube-api-access-6wcm6\") pod \"node-exporter-66czn\" (UID: \"67bac748-66ed-4dcb-8a9c-1814e8074836\") " pod="openshift-monitoring/node-exporter-66czn"
Sep 07 11:52:07.197505 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:52:07.197386    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/67bac748-66ed-4dcb-8a9c-1814e8074836-node-exporter-tls\") pod \"node-exporter-66czn\" (UID: \"67bac748-66ed-4dcb-8a9c-1814e8074836\") " pod="openshift-monitoring/node-exporter-66czn"
Sep 07 11:52:07.197505 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:52:07.197439    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/67bac748-66ed-4dcb-8a9c-1814e8074836-node-exporter-accelerators-collector-config\") pod \"node-exporter-66czn\" (UID: \"67bac748-66ed-4dcb-8a9c-1814e8074836\") " pod="openshift-monitoring/node-exporter-66czn"
Sep 07 11:52:07.197886 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:52:07.197522    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/67bac748-66ed-4dcb-8a9c-1814e8074836-node-exporter-wtmp\") pod \"node-exporter-66czn\" (UID: \"67bac748-66ed-4dcb-8a9c-1814e8074836\") " pod="openshift-monitoring/node-exporter-66czn"
Sep 07 11:52:07.197886 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:52:07.197569    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/67bac748-66ed-4dcb-8a9c-1814e8074836-sys\") pod \"node-exporter-66czn\" (UID: \"67bac748-66ed-4dcb-8a9c-1814e8074836\") " pod="openshift-monitoring/node-exporter-66czn"
Sep 07 11:52:07.197886 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:52:07.197873    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ea883d51-c080-4b94-90e3-c5e26c612f53-metrics-client-ca\") pod \"kube-state-metrics-74cb57df7-wzrfr\" (UID: \"ea883d51-c080-4b94-90e3-c5e26c612f53\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-wzrfr"
Sep 07 11:52:07.198080 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:52:07.198057    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/67bac748-66ed-4dcb-8a9c-1814e8074836-node-exporter-accelerators-collector-config\") pod \"node-exporter-66czn\" (UID: \"67bac748-66ed-4dcb-8a9c-1814e8074836\") " pod="openshift-monitoring/node-exporter-66czn"
Sep 07 11:52:07.198141 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:52:07.198096    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/67bac748-66ed-4dcb-8a9c-1814e8074836-metrics-client-ca\") pod \"node-exporter-66czn\" (UID: \"67bac748-66ed-4dcb-8a9c-1814e8074836\") " pod="openshift-monitoring/node-exporter-66czn"
Sep 07 11:52:07.198292 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:52:07.198263    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/ea883d51-c080-4b94-90e3-c5e26c612f53-volume-directive-shadow\") pod \"kube-state-metrics-74cb57df7-wzrfr\" (UID: \"ea883d51-c080-4b94-90e3-c5e26c612f53\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-wzrfr"
Sep 07 11:52:07.198659 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:52:07.198619    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/67bac748-66ed-4dcb-8a9c-1814e8074836-node-exporter-textfile\") pod \"node-exporter-66czn\" (UID: \"67bac748-66ed-4dcb-8a9c-1814e8074836\") " pod="openshift-monitoring/node-exporter-66czn"
Sep 07 11:52:07.198769 ip-10-0-135-173 kubenswrapper[2563]: E0907 11:52:07.198707    2563 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found
Sep 07 11:52:07.198829 ip-10-0-135-173 kubenswrapper[2563]: E0907 11:52:07.198775    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/67bac748-66ed-4dcb-8a9c-1814e8074836-node-exporter-tls podName:67bac748-66ed-4dcb-8a9c-1814e8074836 nodeName:}" failed. No retries permitted until 2026-09-07 11:52:07.698756493 +0000 UTC m=+299.217488290 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/67bac748-66ed-4dcb-8a9c-1814e8074836-node-exporter-tls") pod "node-exporter-66czn" (UID: "67bac748-66ed-4dcb-8a9c-1814e8074836") : secret "node-exporter-tls" not found
Sep 07 11:52:07.199260 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:52:07.199237    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/ea883d51-c080-4b94-90e3-c5e26c612f53-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-74cb57df7-wzrfr\" (UID: \"ea883d51-c080-4b94-90e3-c5e26c612f53\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-wzrfr"
Sep 07 11:52:07.199955 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:52:07.199930    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ea883d51-c080-4b94-90e3-c5e26c612f53-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-74cb57df7-wzrfr\" (UID: \"ea883d51-c080-4b94-90e3-c5e26c612f53\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-wzrfr"
Sep 07 11:52:07.199955 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:52:07.199949    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/ea883d51-c080-4b94-90e3-c5e26c612f53-kube-state-metrics-tls\") pod \"kube-state-metrics-74cb57df7-wzrfr\" (UID: \"ea883d51-c080-4b94-90e3-c5e26c612f53\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-wzrfr"
Sep 07 11:52:07.200700 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:52:07.200680    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/67bac748-66ed-4dcb-8a9c-1814e8074836-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-66czn\" (UID: \"67bac748-66ed-4dcb-8a9c-1814e8074836\") " pod="openshift-monitoring/node-exporter-66czn"
Sep 07 11:52:07.204586 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:52:07.204492    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 11:52:07.206219 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:52:07.206197    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6wcm6\" (UniqueName: \"kubernetes.io/projected/67bac748-66ed-4dcb-8a9c-1814e8074836-kube-api-access-6wcm6\") pod \"node-exporter-66czn\" (UID: \"67bac748-66ed-4dcb-8a9c-1814e8074836\") " pod="openshift-monitoring/node-exporter-66czn"
Sep 07 11:52:07.206302 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:52:07.206284    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ztmb8\" (UniqueName: \"kubernetes.io/projected/ea883d51-c080-4b94-90e3-c5e26c612f53-kube-api-access-ztmb8\") pod \"kube-state-metrics-74cb57df7-wzrfr\" (UID: \"ea883d51-c080-4b94-90e3-c5e26c612f53\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-wzrfr"
Sep 07 11:52:07.238135 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:52:07.238114    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-74cb57df7-wzrfr"
Sep 07 11:52:07.307053 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:52:07.307027    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 11:52:07.406827 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:52:07.406789    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 11:52:07.508952 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:52:07.508893    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-74cb57df7-wzrfr"]
Sep 07 11:52:07.511646 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:52:07.511619    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podea883d51_c080_4b94_90e3_c5e26c612f53.slice/crio-0e42146d8df182531234c704a0cb68e6e47bf5b8997db53447d6df99630aff07 WatchSource:0}: Error finding container 0e42146d8df182531234c704a0cb68e6e47bf5b8997db53447d6df99630aff07: Status 404 returned error can't find the container with id 0e42146d8df182531234c704a0cb68e6e47bf5b8997db53447d6df99630aff07
Sep 07 11:52:07.531160 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:52:07.531126    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 11:52:07.702735 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:52:07.702684    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/67bac748-66ed-4dcb-8a9c-1814e8074836-node-exporter-tls\") pod \"node-exporter-66czn\" (UID: \"67bac748-66ed-4dcb-8a9c-1814e8074836\") " pod="openshift-monitoring/node-exporter-66czn"
Sep 07 11:52:07.704865 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:52:07.704848    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/67bac748-66ed-4dcb-8a9c-1814e8074836-node-exporter-tls\") pod \"node-exporter-66czn\" (UID: \"67bac748-66ed-4dcb-8a9c-1814e8074836\") " pod="openshift-monitoring/node-exporter-66czn"
Sep 07 11:52:07.842313 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:52:07.842251    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-66czn"
Sep 07 11:52:07.852787 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:52:07.852759    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod67bac748_66ed_4dcb_8a9c_1814e8074836.slice/crio-93e286c0fba27b3d16cdb9d0f5324d83bee896d974eedb688e958dbe4f43e96a WatchSource:0}: Error finding container 93e286c0fba27b3d16cdb9d0f5324d83bee896d974eedb688e958dbe4f43e96a: Status 404 returned error can't find the container with id 93e286c0fba27b3d16cdb9d0f5324d83bee896d974eedb688e958dbe4f43e96a
Sep 07 11:52:07.890650 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:52:07.890619    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 11:52:07.965623 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:52:07.965595    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-74cb57df7-wzrfr" event={"ID":"ea883d51-c080-4b94-90e3-c5e26c612f53","Type":"ContainerStarted","Data":"0e42146d8df182531234c704a0cb68e6e47bf5b8997db53447d6df99630aff07"}
Sep 07 11:52:07.966758 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:52:07.966735    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-66czn" event={"ID":"67bac748-66ed-4dcb-8a9c-1814e8074836","Type":"ContainerStarted","Data":"93e286c0fba27b3d16cdb9d0f5324d83bee896d974eedb688e958dbe4f43e96a"}
Sep 07 11:52:07.977238 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:52:07.977215    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 11:52:07.985961 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:52:07.985936    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 07 11:52:07.989368 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:52:07.989350    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 07 11:52:07.991363 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:52:07.991332    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Sep 07 11:52:07.991363 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:52:07.991355    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Sep 07 11:52:07.991523 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:52:07.991332    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Sep 07 11:52:07.991688 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:52:07.991673    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Sep 07 11:52:07.991764 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:52:07.991706    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-5qlcc\""
Sep 07 11:52:07.992006 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:52:07.991991    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Sep 07 11:52:07.992132 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:52:07.992116    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Sep 07 11:52:07.992208 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:52:07.992174    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Sep 07 11:52:07.992380 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:52:07.992366    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Sep 07 11:52:07.997125 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:52:07.997107    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Sep 07 11:52:08.002479 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:52:08.002460    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 07 11:52:08.106564 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:52:08.106502    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/57a9b372-5c0c-48e0-b0a2-49814095da99-config-out\") pod \"alertmanager-main-0\" (UID: \"57a9b372-5c0c-48e0-b0a2-49814095da99\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 07 11:52:08.106564 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:52:08.106535    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/57a9b372-5c0c-48e0-b0a2-49814095da99-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"57a9b372-5c0c-48e0-b0a2-49814095da99\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 07 11:52:08.106564 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:52:08.106561    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/57a9b372-5c0c-48e0-b0a2-49814095da99-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"57a9b372-5c0c-48e0-b0a2-49814095da99\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 07 11:52:08.106802 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:52:08.106648    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6qnvb\" (UniqueName: \"kubernetes.io/projected/57a9b372-5c0c-48e0-b0a2-49814095da99-kube-api-access-6qnvb\") pod \"alertmanager-main-0\" (UID: \"57a9b372-5c0c-48e0-b0a2-49814095da99\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 07 11:52:08.106802 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:52:08.106677    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/57a9b372-5c0c-48e0-b0a2-49814095da99-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"57a9b372-5c0c-48e0-b0a2-49814095da99\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 07 11:52:08.106802 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:52:08.106705    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/57a9b372-5c0c-48e0-b0a2-49814095da99-web-config\") pod \"alertmanager-main-0\" (UID: \"57a9b372-5c0c-48e0-b0a2-49814095da99\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 07 11:52:08.106802 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:52:08.106734    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/57a9b372-5c0c-48e0-b0a2-49814095da99-tls-assets\") pod \"alertmanager-main-0\" (UID: \"57a9b372-5c0c-48e0-b0a2-49814095da99\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 07 11:52:08.106802 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:52:08.106783    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/57a9b372-5c0c-48e0-b0a2-49814095da99-config-volume\") pod \"alertmanager-main-0\" (UID: \"57a9b372-5c0c-48e0-b0a2-49814095da99\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 07 11:52:08.107048 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:52:08.106818    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/57a9b372-5c0c-48e0-b0a2-49814095da99-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"57a9b372-5c0c-48e0-b0a2-49814095da99\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 07 11:52:08.107048 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:52:08.106848    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/57a9b372-5c0c-48e0-b0a2-49814095da99-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"57a9b372-5c0c-48e0-b0a2-49814095da99\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 07 11:52:08.107048 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:52:08.106889    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/57a9b372-5c0c-48e0-b0a2-49814095da99-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"57a9b372-5c0c-48e0-b0a2-49814095da99\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 07 11:52:08.107048 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:52:08.106919    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/57a9b372-5c0c-48e0-b0a2-49814095da99-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"57a9b372-5c0c-48e0-b0a2-49814095da99\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 07 11:52:08.107048 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:52:08.106954    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/57a9b372-5c0c-48e0-b0a2-49814095da99-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"57a9b372-5c0c-48e0-b0a2-49814095da99\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 07 11:52:08.207812 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:52:08.207776    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/57a9b372-5c0c-48e0-b0a2-49814095da99-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"57a9b372-5c0c-48e0-b0a2-49814095da99\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 07 11:52:08.207951 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:52:08.207817    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/57a9b372-5c0c-48e0-b0a2-49814095da99-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"57a9b372-5c0c-48e0-b0a2-49814095da99\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 07 11:52:08.207951 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:52:08.207868    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/57a9b372-5c0c-48e0-b0a2-49814095da99-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"57a9b372-5c0c-48e0-b0a2-49814095da99\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 07 11:52:08.207951 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:52:08.207898    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/57a9b372-5c0c-48e0-b0a2-49814095da99-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"57a9b372-5c0c-48e0-b0a2-49814095da99\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 07 11:52:08.207951 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:52:08.207937    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/57a9b372-5c0c-48e0-b0a2-49814095da99-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"57a9b372-5c0c-48e0-b0a2-49814095da99\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 07 11:52:08.208152 ip-10-0-135-173 kubenswrapper[2563]: E0907 11:52:08.207957    2563 secret.go:189] Couldn't get secret openshift-monitoring/alertmanager-main-tls: secret "alertmanager-main-tls" not found
Sep 07 11:52:08.208152 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:52:08.207970    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/57a9b372-5c0c-48e0-b0a2-49814095da99-config-out\") pod \"alertmanager-main-0\" (UID: \"57a9b372-5c0c-48e0-b0a2-49814095da99\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 07 11:52:08.208152 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:52:08.208021    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/57a9b372-5c0c-48e0-b0a2-49814095da99-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"57a9b372-5c0c-48e0-b0a2-49814095da99\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 07 11:52:08.208152 ip-10-0-135-173 kubenswrapper[2563]: E0907 11:52:08.208032    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/57a9b372-5c0c-48e0-b0a2-49814095da99-secret-alertmanager-main-tls podName:57a9b372-5c0c-48e0-b0a2-49814095da99 nodeName:}" failed. No retries permitted until 2026-09-07 11:52:08.70801468 +0000 UTC m=+300.226746471 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "secret-alertmanager-main-tls" (UniqueName: "kubernetes.io/secret/57a9b372-5c0c-48e0-b0a2-49814095da99-secret-alertmanager-main-tls") pod "alertmanager-main-0" (UID: "57a9b372-5c0c-48e0-b0a2-49814095da99") : secret "alertmanager-main-tls" not found
Sep 07 11:52:08.208152 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:52:08.208068    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/57a9b372-5c0c-48e0-b0a2-49814095da99-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"57a9b372-5c0c-48e0-b0a2-49814095da99\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 07 11:52:08.208152 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:52:08.208097    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6qnvb\" (UniqueName: \"kubernetes.io/projected/57a9b372-5c0c-48e0-b0a2-49814095da99-kube-api-access-6qnvb\") pod \"alertmanager-main-0\" (UID: \"57a9b372-5c0c-48e0-b0a2-49814095da99\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 07 11:52:08.208152 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:52:08.208125    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/57a9b372-5c0c-48e0-b0a2-49814095da99-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"57a9b372-5c0c-48e0-b0a2-49814095da99\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 07 11:52:08.208517 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:52:08.208156    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/57a9b372-5c0c-48e0-b0a2-49814095da99-web-config\") pod \"alertmanager-main-0\" (UID: \"57a9b372-5c0c-48e0-b0a2-49814095da99\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 07 11:52:08.208517 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:52:08.208183    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/57a9b372-5c0c-48e0-b0a2-49814095da99-tls-assets\") pod \"alertmanager-main-0\" (UID: \"57a9b372-5c0c-48e0-b0a2-49814095da99\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 07 11:52:08.208517 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:52:08.208221    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/57a9b372-5c0c-48e0-b0a2-49814095da99-config-volume\") pod \"alertmanager-main-0\" (UID: \"57a9b372-5c0c-48e0-b0a2-49814095da99\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 07 11:52:08.210909 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:52:08.209715    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/57a9b372-5c0c-48e0-b0a2-49814095da99-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"57a9b372-5c0c-48e0-b0a2-49814095da99\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 07 11:52:08.210909 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:52:08.210593    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/57a9b372-5c0c-48e0-b0a2-49814095da99-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"57a9b372-5c0c-48e0-b0a2-49814095da99\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 07 11:52:08.211210 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:52:08.211179    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/57a9b372-5c0c-48e0-b0a2-49814095da99-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"57a9b372-5c0c-48e0-b0a2-49814095da99\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 07 11:52:08.212013 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:52:08.211971    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/57a9b372-5c0c-48e0-b0a2-49814095da99-config-volume\") pod \"alertmanager-main-0\" (UID: \"57a9b372-5c0c-48e0-b0a2-49814095da99\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 07 11:52:08.214705 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:52:08.214362    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/57a9b372-5c0c-48e0-b0a2-49814095da99-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"57a9b372-5c0c-48e0-b0a2-49814095da99\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 07 11:52:08.214705 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:52:08.214666    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/57a9b372-5c0c-48e0-b0a2-49814095da99-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"57a9b372-5c0c-48e0-b0a2-49814095da99\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 07 11:52:08.214705 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:52:08.214668    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/57a9b372-5c0c-48e0-b0a2-49814095da99-web-config\") pod \"alertmanager-main-0\" (UID: \"57a9b372-5c0c-48e0-b0a2-49814095da99\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 07 11:52:08.214926 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:52:08.214709    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/57a9b372-5c0c-48e0-b0a2-49814095da99-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"57a9b372-5c0c-48e0-b0a2-49814095da99\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 07 11:52:08.214926 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:52:08.214749    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/57a9b372-5c0c-48e0-b0a2-49814095da99-tls-assets\") pod \"alertmanager-main-0\" (UID: \"57a9b372-5c0c-48e0-b0a2-49814095da99\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 07 11:52:08.215217 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:52:08.215195    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/57a9b372-5c0c-48e0-b0a2-49814095da99-config-out\") pod \"alertmanager-main-0\" (UID: \"57a9b372-5c0c-48e0-b0a2-49814095da99\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 07 11:52:08.216118 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:52:08.216094    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/57a9b372-5c0c-48e0-b0a2-49814095da99-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"57a9b372-5c0c-48e0-b0a2-49814095da99\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 07 11:52:08.218211 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:52:08.218188    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6qnvb\" (UniqueName: \"kubernetes.io/projected/57a9b372-5c0c-48e0-b0a2-49814095da99-kube-api-access-6qnvb\") pod \"alertmanager-main-0\" (UID: \"57a9b372-5c0c-48e0-b0a2-49814095da99\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 07 11:52:08.625838 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:52:08.625803    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 11:52:08.713515 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:52:08.713476    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/57a9b372-5c0c-48e0-b0a2-49814095da99-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"57a9b372-5c0c-48e0-b0a2-49814095da99\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 07 11:52:08.716273 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:52:08.716245    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/57a9b372-5c0c-48e0-b0a2-49814095da99-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"57a9b372-5c0c-48e0-b0a2-49814095da99\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 07 11:52:08.894220 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:52:08.894148    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-65bdc89fcb-8tfm6"]
Sep 07 11:52:08.897855 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:52:08.897832    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-65bdc89fcb-8tfm6"
Sep 07 11:52:08.899756 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:52:08.899739    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\""
Sep 07 11:52:08.899849 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:52:08.899788    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\""
Sep 07 11:52:08.900093 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:52:08.900074    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\""
Sep 07 11:52:08.900192 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:52:08.900142    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\""
Sep 07 11:52:08.900261 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:52:08.900248    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-7sdtb\""
Sep 07 11:52:08.900351 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:52:08.900336    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-ffmrn85pvp6pm\""
Sep 07 11:52:08.900447 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:52:08.900369    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\""
Sep 07 11:52:08.902065 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:52:08.902043    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-5qlcc\""
Sep 07 11:52:08.906715 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:52:08.906647    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-65bdc89fcb-8tfm6"]
Sep 07 11:52:08.910062 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:52:08.910037    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 07 11:52:09.016398 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:52:09.016368    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/00d4ac48-8eff-4658-963c-b338ff9e2714-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-65bdc89fcb-8tfm6\" (UID: \"00d4ac48-8eff-4658-963c-b338ff9e2714\") " pod="openshift-monitoring/thanos-querier-65bdc89fcb-8tfm6"
Sep 07 11:52:09.016578 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:52:09.016404    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/00d4ac48-8eff-4658-963c-b338ff9e2714-secret-thanos-querier-tls\") pod \"thanos-querier-65bdc89fcb-8tfm6\" (UID: \"00d4ac48-8eff-4658-963c-b338ff9e2714\") " pod="openshift-monitoring/thanos-querier-65bdc89fcb-8tfm6"
Sep 07 11:52:09.016578 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:52:09.016455    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/00d4ac48-8eff-4658-963c-b338ff9e2714-secret-grpc-tls\") pod \"thanos-querier-65bdc89fcb-8tfm6\" (UID: \"00d4ac48-8eff-4658-963c-b338ff9e2714\") " pod="openshift-monitoring/thanos-querier-65bdc89fcb-8tfm6"
Sep 07 11:52:09.016578 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:52:09.016502    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/00d4ac48-8eff-4658-963c-b338ff9e2714-metrics-client-ca\") pod \"thanos-querier-65bdc89fcb-8tfm6\" (UID: \"00d4ac48-8eff-4658-963c-b338ff9e2714\") " pod="openshift-monitoring/thanos-querier-65bdc89fcb-8tfm6"
Sep 07 11:52:09.016578 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:52:09.016564    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/00d4ac48-8eff-4658-963c-b338ff9e2714-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-65bdc89fcb-8tfm6\" (UID: \"00d4ac48-8eff-4658-963c-b338ff9e2714\") " pod="openshift-monitoring/thanos-querier-65bdc89fcb-8tfm6"
Sep 07 11:52:09.016710 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:52:09.016599    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/00d4ac48-8eff-4658-963c-b338ff9e2714-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-65bdc89fcb-8tfm6\" (UID: \"00d4ac48-8eff-4658-963c-b338ff9e2714\") " pod="openshift-monitoring/thanos-querier-65bdc89fcb-8tfm6"
Sep 07 11:52:09.016710 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:52:09.016640    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/00d4ac48-8eff-4658-963c-b338ff9e2714-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-65bdc89fcb-8tfm6\" (UID: \"00d4ac48-8eff-4658-963c-b338ff9e2714\") " pod="openshift-monitoring/thanos-querier-65bdc89fcb-8tfm6"
Sep 07 11:52:09.016710 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:52:09.016697    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jt9gb\" (UniqueName: \"kubernetes.io/projected/00d4ac48-8eff-4658-963c-b338ff9e2714-kube-api-access-jt9gb\") pod \"thanos-querier-65bdc89fcb-8tfm6\" (UID: \"00d4ac48-8eff-4658-963c-b338ff9e2714\") " pod="openshift-monitoring/thanos-querier-65bdc89fcb-8tfm6"
Sep 07 11:52:09.068137 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:52:09.068113    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-4nfbc_b34ca85d-ac51-4724-a508-1026106b3f3a/console-operator/2.log"
Sep 07 11:52:09.069112 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:52:09.069095    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-4nfbc_b34ca85d-ac51-4724-a508-1026106b3f3a/console-operator/2.log"
Sep 07 11:52:09.082591 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:52:09.082574    2563 kubelet.go:1648] "Image garbage collection succeeded"
Sep 07 11:52:09.117793 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:52:09.117768    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/00d4ac48-8eff-4658-963c-b338ff9e2714-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-65bdc89fcb-8tfm6\" (UID: \"00d4ac48-8eff-4658-963c-b338ff9e2714\") " pod="openshift-monitoring/thanos-querier-65bdc89fcb-8tfm6"
Sep 07 11:52:09.117902 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:52:09.117816    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jt9gb\" (UniqueName: \"kubernetes.io/projected/00d4ac48-8eff-4658-963c-b338ff9e2714-kube-api-access-jt9gb\") pod \"thanos-querier-65bdc89fcb-8tfm6\" (UID: \"00d4ac48-8eff-4658-963c-b338ff9e2714\") " pod="openshift-monitoring/thanos-querier-65bdc89fcb-8tfm6"
Sep 07 11:52:09.117902 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:52:09.117873    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/00d4ac48-8eff-4658-963c-b338ff9e2714-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-65bdc89fcb-8tfm6\" (UID: \"00d4ac48-8eff-4658-963c-b338ff9e2714\") " pod="openshift-monitoring/thanos-querier-65bdc89fcb-8tfm6"
Sep 07 11:52:09.117902 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:52:09.117892    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/00d4ac48-8eff-4658-963c-b338ff9e2714-secret-thanos-querier-tls\") pod \"thanos-querier-65bdc89fcb-8tfm6\" (UID: \"00d4ac48-8eff-4658-963c-b338ff9e2714\") " pod="openshift-monitoring/thanos-querier-65bdc89fcb-8tfm6"
Sep 07 11:52:09.118055 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:52:09.117926    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/00d4ac48-8eff-4658-963c-b338ff9e2714-secret-grpc-tls\") pod \"thanos-querier-65bdc89fcb-8tfm6\" (UID: \"00d4ac48-8eff-4658-963c-b338ff9e2714\") " pod="openshift-monitoring/thanos-querier-65bdc89fcb-8tfm6"
Sep 07 11:52:09.118055 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:52:09.117947    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/00d4ac48-8eff-4658-963c-b338ff9e2714-metrics-client-ca\") pod \"thanos-querier-65bdc89fcb-8tfm6\" (UID: \"00d4ac48-8eff-4658-963c-b338ff9e2714\") " pod="openshift-monitoring/thanos-querier-65bdc89fcb-8tfm6"
Sep 07 11:52:09.118055 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:52:09.117971    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/00d4ac48-8eff-4658-963c-b338ff9e2714-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-65bdc89fcb-8tfm6\" (UID: \"00d4ac48-8eff-4658-963c-b338ff9e2714\") " pod="openshift-monitoring/thanos-querier-65bdc89fcb-8tfm6"
Sep 07 11:52:09.118055 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:52:09.117989    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/00d4ac48-8eff-4658-963c-b338ff9e2714-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-65bdc89fcb-8tfm6\" (UID: \"00d4ac48-8eff-4658-963c-b338ff9e2714\") " pod="openshift-monitoring/thanos-querier-65bdc89fcb-8tfm6"
Sep 07 11:52:09.119084 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:52:09.118956    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/00d4ac48-8eff-4658-963c-b338ff9e2714-metrics-client-ca\") pod \"thanos-querier-65bdc89fcb-8tfm6\" (UID: \"00d4ac48-8eff-4658-963c-b338ff9e2714\") " pod="openshift-monitoring/thanos-querier-65bdc89fcb-8tfm6"
Sep 07 11:52:09.120482 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:52:09.120460    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/00d4ac48-8eff-4658-963c-b338ff9e2714-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-65bdc89fcb-8tfm6\" (UID: \"00d4ac48-8eff-4658-963c-b338ff9e2714\") " pod="openshift-monitoring/thanos-querier-65bdc89fcb-8tfm6"
Sep 07 11:52:09.120997 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:52:09.120962    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/00d4ac48-8eff-4658-963c-b338ff9e2714-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-65bdc89fcb-8tfm6\" (UID: \"00d4ac48-8eff-4658-963c-b338ff9e2714\") " pod="openshift-monitoring/thanos-querier-65bdc89fcb-8tfm6"
Sep 07 11:52:09.121520 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:52:09.121490    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/00d4ac48-8eff-4658-963c-b338ff9e2714-secret-grpc-tls\") pod \"thanos-querier-65bdc89fcb-8tfm6\" (UID: \"00d4ac48-8eff-4658-963c-b338ff9e2714\") " pod="openshift-monitoring/thanos-querier-65bdc89fcb-8tfm6"
Sep 07 11:52:09.122171 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:52:09.122153    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/00d4ac48-8eff-4658-963c-b338ff9e2714-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-65bdc89fcb-8tfm6\" (UID: \"00d4ac48-8eff-4658-963c-b338ff9e2714\") " pod="openshift-monitoring/thanos-querier-65bdc89fcb-8tfm6"
Sep 07 11:52:09.123332 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:52:09.122852    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/00d4ac48-8eff-4658-963c-b338ff9e2714-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-65bdc89fcb-8tfm6\" (UID: \"00d4ac48-8eff-4658-963c-b338ff9e2714\") " pod="openshift-monitoring/thanos-querier-65bdc89fcb-8tfm6"
Sep 07 11:52:09.123332 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:52:09.123094    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/00d4ac48-8eff-4658-963c-b338ff9e2714-secret-thanos-querier-tls\") pod \"thanos-querier-65bdc89fcb-8tfm6\" (UID: \"00d4ac48-8eff-4658-963c-b338ff9e2714\") " pod="openshift-monitoring/thanos-querier-65bdc89fcb-8tfm6"
Sep 07 11:52:09.126400 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:52:09.126355    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jt9gb\" (UniqueName: \"kubernetes.io/projected/00d4ac48-8eff-4658-963c-b338ff9e2714-kube-api-access-jt9gb\") pod \"thanos-querier-65bdc89fcb-8tfm6\" (UID: \"00d4ac48-8eff-4658-963c-b338ff9e2714\") " pod="openshift-monitoring/thanos-querier-65bdc89fcb-8tfm6"
Sep 07 11:52:09.209526 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:52:09.209447    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-65bdc89fcb-8tfm6"
Sep 07 11:52:09.331623 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:52:09.331586    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 07 11:52:09.334089 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:52:09.334057    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod57a9b372_5c0c_48e0_b0a2_49814095da99.slice/crio-bc552b4d6c357e1b7720db0b396eba16cf91371f9b314c304994d9e607d68e18 WatchSource:0}: Error finding container bc552b4d6c357e1b7720db0b396eba16cf91371f9b314c304994d9e607d68e18: Status 404 returned error can't find the container with id bc552b4d6c357e1b7720db0b396eba16cf91371f9b314c304994d9e607d68e18
Sep 07 11:52:09.496471 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:52:09.496438    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-65bdc89fcb-8tfm6"]
Sep 07 11:52:09.500008 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:52:09.499981    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod00d4ac48_8eff_4658_963c_b338ff9e2714.slice/crio-88c880d683eea6eb11fb8a78f5bf857952e3d48b2b9e1893b652314c99a676e1 WatchSource:0}: Error finding container 88c880d683eea6eb11fb8a78f5bf857952e3d48b2b9e1893b652314c99a676e1: Status 404 returned error can't find the container with id 88c880d683eea6eb11fb8a78f5bf857952e3d48b2b9e1893b652314c99a676e1
Sep 07 11:52:09.843571 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:52:09.843492    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 11:52:09.945098 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:52:09.945053    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 11:52:09.978845 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:52:09.978810    2563 generic.go:356] "Generic (PLEG): container finished" podID="67bac748-66ed-4dcb-8a9c-1814e8074836" containerID="d7f571145ea357afc44201e636a209ebab31703e0bb2dcc10bbd229b537c95a2" exitCode=0
Sep 07 11:52:09.979006 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:52:09.978977    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-66czn" event={"ID":"67bac748-66ed-4dcb-8a9c-1814e8074836","Type":"ContainerDied","Data":"d7f571145ea357afc44201e636a209ebab31703e0bb2dcc10bbd229b537c95a2"}
Sep 07 11:52:09.981381 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:52:09.981347    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"57a9b372-5c0c-48e0-b0a2-49814095da99","Type":"ContainerStarted","Data":"bc552b4d6c357e1b7720db0b396eba16cf91371f9b314c304994d9e607d68e18"}
Sep 07 11:52:09.983151 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:52:09.983125    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-65bdc89fcb-8tfm6" event={"ID":"00d4ac48-8eff-4658-963c-b338ff9e2714","Type":"ContainerStarted","Data":"88c880d683eea6eb11fb8a78f5bf857952e3d48b2b9e1893b652314c99a676e1"}
Sep 07 11:52:09.986068 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:52:09.986041    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-74cb57df7-wzrfr" event={"ID":"ea883d51-c080-4b94-90e3-c5e26c612f53","Type":"ContainerStarted","Data":"277d5aeacc139783d23e570012cb0ad9ed55e79913f846be5a5afc1039ffd459"}
Sep 07 11:52:09.986160 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:52:09.986098    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-74cb57df7-wzrfr" event={"ID":"ea883d51-c080-4b94-90e3-c5e26c612f53","Type":"ContainerStarted","Data":"23be4042e2bd2c114f027d2d1bfbafa3964b4356ab85ac56fe00747ab3689f42"}
Sep 07 11:52:09.986160 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:52:09.986121    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-74cb57df7-wzrfr" event={"ID":"ea883d51-c080-4b94-90e3-c5e26c612f53","Type":"ContainerStarted","Data":"f52b78a182c7b7eee160eb446a31474e09f2bcf0d56ed03ad003c914399b8248"}
Sep 07 11:52:10.017120 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:52:10.017077    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-74cb57df7-wzrfr" podStartSLOduration=2.476120225 podStartE2EDuration="4.017061787s" totalImagesPullingTime="1.540941562s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 11:52:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 11:52:10.014833506 +0000 UTC m=+301.533565321" watchObservedRunningTime="2026-09-07 11:52:10.017061787 +0000 UTC m=+301.535793608"
Sep 07 11:52:10.088457 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:52:10.088404    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 11:52:10.290356 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:52:10.290316    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 11:52:10.992072 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:52:10.992038    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-66czn" event={"ID":"67bac748-66ed-4dcb-8a9c-1814e8074836","Type":"ContainerStarted","Data":"57c172ec136e6899b3c05c64a7f1e6ee3e3b67716dfa02f9c3a51f29ef5a3a6d"}
Sep 07 11:52:10.992528 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:52:10.992096    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-66czn" event={"ID":"67bac748-66ed-4dcb-8a9c-1814e8074836","Type":"ContainerStarted","Data":"7072ca21a52a61d120038548fcd2036cd0c4fc68bd7a651d865ac0ffed31f5c7"}
Sep 07 11:52:10.993481 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:52:10.993454    2563 generic.go:356] "Generic (PLEG): container finished" podID="57a9b372-5c0c-48e0-b0a2-49814095da99" containerID="bd31040e3f5fe153bb53c4bdc1a4af3beb7c6ea8d8806a8711ca3c409d5c62b0" exitCode=0
Sep 07 11:52:10.993587 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:52:10.993535    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"57a9b372-5c0c-48e0-b0a2-49814095da99","Type":"ContainerDied","Data":"bd31040e3f5fe153bb53c4bdc1a4af3beb7c6ea8d8806a8711ca3c409d5c62b0"}
Sep 07 11:52:10.994624 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:52:10.994605    2563 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 07 11:52:11.008157 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:52:11.008109    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-66czn" podStartSLOduration=3.726523761 podStartE2EDuration="5.008087832s" totalImagesPullingTime="1.202364158s" totalInitContainerRuntime="79.199913ms" isStatefulPod=true podCreationTimestamp="2026-09-07 11:52:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 11:52:11.007154958 +0000 UTC m=+302.525886768" watchObservedRunningTime="2026-09-07 11:52:11.008087832 +0000 UTC m=+302.526819646"
Sep 07 11:52:11.378559 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:52:11.378472    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 11:52:11.479133 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:52:11.479090    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 11:52:11.687698 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:52:11.687665    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-85bbbbbf4-wt826"]
Sep 07 11:52:11.688822 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:52:11.688796    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 11:52:11.692128 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:52:11.692109    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-wt826"
Sep 07 11:52:11.694212 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:52:11.693824    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\""
Sep 07 11:52:11.694212 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:52:11.694055    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-tfvwg\""
Sep 07 11:52:11.696903 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:52:11.696844    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-85bbbbbf4-wt826"]
Sep 07 11:52:11.743463 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:52:11.743409    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j54gs\" (UniqueName: \"kubernetes.io/projected/2d925afb-5916-41ec-86db-9c11538cfa85-kube-api-access-j54gs\") pod \"monitoring-plugin-85bbbbbf4-wt826\" (UID: \"2d925afb-5916-41ec-86db-9c11538cfa85\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-wt826"
Sep 07 11:52:11.743567 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:52:11.743522    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/2d925afb-5916-41ec-86db-9c11538cfa85-monitoring-plugin-cert\") pod \"monitoring-plugin-85bbbbbf4-wt826\" (UID: \"2d925afb-5916-41ec-86db-9c11538cfa85\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-wt826"
Sep 07 11:52:11.805667 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:52:11.805597    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 11:52:11.844590 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:52:11.844551    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j54gs\" (UniqueName: \"kubernetes.io/projected/2d925afb-5916-41ec-86db-9c11538cfa85-kube-api-access-j54gs\") pod \"monitoring-plugin-85bbbbbf4-wt826\" (UID: \"2d925afb-5916-41ec-86db-9c11538cfa85\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-wt826"
Sep 07 11:52:11.844779 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:52:11.844631    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/2d925afb-5916-41ec-86db-9c11538cfa85-monitoring-plugin-cert\") pod \"monitoring-plugin-85bbbbbf4-wt826\" (UID: \"2d925afb-5916-41ec-86db-9c11538cfa85\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-wt826"
Sep 07 11:52:11.847556 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:52:11.847526    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/2d925afb-5916-41ec-86db-9c11538cfa85-monitoring-plugin-cert\") pod \"monitoring-plugin-85bbbbbf4-wt826\" (UID: \"2d925afb-5916-41ec-86db-9c11538cfa85\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-wt826"
Sep 07 11:52:11.856867 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:52:11.856840    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j54gs\" (UniqueName: \"kubernetes.io/projected/2d925afb-5916-41ec-86db-9c11538cfa85-kube-api-access-j54gs\") pod \"monitoring-plugin-85bbbbbf4-wt826\" (UID: \"2d925afb-5916-41ec-86db-9c11538cfa85\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-wt826"
Sep 07 11:52:11.999130 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:52:11.999093    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-65bdc89fcb-8tfm6" event={"ID":"00d4ac48-8eff-4658-963c-b338ff9e2714","Type":"ContainerStarted","Data":"f3e022ed53ff337bae135cb7fdc130e61de993374719923e38d1f35b6b5001bc"}
Sep 07 11:52:11.999507 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:52:11.999177    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-65bdc89fcb-8tfm6" event={"ID":"00d4ac48-8eff-4658-963c-b338ff9e2714","Type":"ContainerStarted","Data":"107f173eeced42e5cec70007225683c59ea6c530c88b8efdbbab8f64d21d3edf"}
Sep 07 11:52:11.999507 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:52:11.999221    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-65bdc89fcb-8tfm6" event={"ID":"00d4ac48-8eff-4658-963c-b338ff9e2714","Type":"ContainerStarted","Data":"e8467458c375d28418b21e3104efb4a669904504f66e3c76f02694259aa089bd"}
Sep 07 11:52:12.027698 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:52:12.027667    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-wt826"
Sep 07 11:52:12.303466 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:52:12.302360    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-85bbbbbf4-wt826"]
Sep 07 11:52:12.530114 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:52:12.530082    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2d925afb_5916_41ec_86db_9c11538cfa85.slice/crio-7779f546a9e87e0b3fd19a71381a785635ca1e38c9c83e5cf444cd8334498381 WatchSource:0}: Error finding container 7779f546a9e87e0b3fd19a71381a785635ca1e38c9c83e5cf444cd8334498381: Status 404 returned error can't find the container with id 7779f546a9e87e0b3fd19a71381a785635ca1e38c9c83e5cf444cd8334498381
Sep 07 11:52:12.558017 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:52:12.557961    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-676764f5b5-dsf9f"
Sep 07 11:52:13.006454 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:52:13.006405    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"57a9b372-5c0c-48e0-b0a2-49814095da99","Type":"ContainerStarted","Data":"7a3d701ece196e6e116809dc39653365323a106702438f2d0f49bf8fa28b7e3a"}
Sep 07 11:52:13.006825 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:52:13.006515    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"57a9b372-5c0c-48e0-b0a2-49814095da99","Type":"ContainerStarted","Data":"230b0afbfc819e47df67117ce1da56cf708e92f6cb15077c68a29e90e4cc89eb"}
Sep 07 11:52:13.006825 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:52:13.006567    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"57a9b372-5c0c-48e0-b0a2-49814095da99","Type":"ContainerStarted","Data":"52d8b7bb0ed9420e3691f9dd8ae3cec051bb21e091fdd26964047e9b10844f2f"}
Sep 07 11:52:13.006825 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:52:13.006607    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"57a9b372-5c0c-48e0-b0a2-49814095da99","Type":"ContainerStarted","Data":"c1e38b4b7e9c090a29836bbf25c4de203bab3b7ad31f2d2024d6b599e31254d5"}
Sep 07 11:52:13.006825 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:52:13.006648    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"57a9b372-5c0c-48e0-b0a2-49814095da99","Type":"ContainerStarted","Data":"df4d5f3cbec9461b05eac65ccd7ae1e1b5076a14e395ac487ded763915493caa"}
Sep 07 11:52:13.007675 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:52:13.007642    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-wt826" event={"ID":"2d925afb-5916-41ec-86db-9c11538cfa85","Type":"ContainerStarted","Data":"7779f546a9e87e0b3fd19a71381a785635ca1e38c9c83e5cf444cd8334498381"}
Sep 07 11:52:13.011677 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:52:13.011323    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-65bdc89fcb-8tfm6" event={"ID":"00d4ac48-8eff-4658-963c-b338ff9e2714","Type":"ContainerStarted","Data":"50d68d4db06bb728df58266f0ca157dde728e6af1f5573c90fc19570a0edaaca"}
Sep 07 11:52:13.011677 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:52:13.011391    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-65bdc89fcb-8tfm6" event={"ID":"00d4ac48-8eff-4658-963c-b338ff9e2714","Type":"ContainerStarted","Data":"d1a11967e18399b63d21aad3f3bab10a172a76077887801d3dab3d1524d06736"}
Sep 07 11:52:13.011677 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:52:13.011450    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-65bdc89fcb-8tfm6" event={"ID":"00d4ac48-8eff-4658-963c-b338ff9e2714","Type":"ContainerStarted","Data":"f7275926558d725f245f563b5bf0e12e6916f7b0327d399ae664bbeb6b2940c8"}
Sep 07 11:52:13.011839 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:52:13.011731    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-65bdc89fcb-8tfm6"
Sep 07 11:52:13.031383 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:52:13.031342    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-65bdc89fcb-8tfm6" podStartSLOduration=2.0435248010000002 podStartE2EDuration="5.031327258s" totalImagesPullingTime="2.987802457s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 11:52:08 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 11:52:13.030272422 +0000 UTC m=+304.549004233" watchObservedRunningTime="2026-09-07 11:52:13.031327258 +0000 UTC m=+304.550059074"
Sep 07 11:52:13.136847 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:52:13.136742    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 11:52:13.301929 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:52:13.301890    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 11:52:13.401333 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:52:13.401251    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 11:52:13.503029 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:52:13.502987    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 11:52:14.018840 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:52:14.018808    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"57a9b372-5c0c-48e0-b0a2-49814095da99","Type":"ContainerStarted","Data":"26d835dca455ab83cd5168987b5846d693f0097ac987b8495e5010b46a1eb3b4"}
Sep 07 11:52:14.020340 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:52:14.020310    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-wt826" event={"ID":"2d925afb-5916-41ec-86db-9c11538cfa85","Type":"ContainerStarted","Data":"875a25a1be75679a81a49247e7d6f8f6050dac3033dbc09f8faec9d4319b49fa"}
Sep 07 11:52:14.020601 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:52:14.020582    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-wt826"
Sep 07 11:52:14.025341 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:52:14.025321    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-wt826"
Sep 07 11:52:14.043661 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:52:14.043465    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=4.130731002 podStartE2EDuration="7.043445962s" totalImagesPullingTime="2.893403382s" totalInitContainerRuntime="19.311578ms" isStatefulPod=false podCreationTimestamp="2026-09-07 11:52:07 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 11:52:14.042396982 +0000 UTC m=+305.561128794" watchObservedRunningTime="2026-09-07 11:52:14.043445962 +0000 UTC m=+305.562177776"
Sep 07 11:52:14.060520 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:52:14.060478    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-wt826" podStartSLOduration=1.813020998 podStartE2EDuration="3.060464225s" totalImagesPullingTime="1.247443227s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 11:52:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 11:52:14.059507614 +0000 UTC m=+305.578239507" watchObservedRunningTime="2026-09-07 11:52:14.060464225 +0000 UTC m=+305.579196063"
Sep 07 11:52:14.926713 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:52:14.926677    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-74ccddf898-jp5pm"
Sep 07 11:52:15.034542 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:52:15.034509    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xf2gj"
Sep 07 11:52:15.044467 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:52:15.044412    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 11:52:15.143552 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:52:15.143528    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 11:52:15.548923 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:52:15.548889    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 11:52:15.649682 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:52:15.649645    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 11:52:17.035141 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:52:17.035112    2563 scope.go:117] "RemoveContainer" containerID="725f5bd2022473854b9b6803069247eb819f5bf8e09a47410ca95ea96dacab2b"
Sep 07 11:52:17.580294 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:52:17.580249    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-676764f5b5-dsf9f" podUID="9ae06e91-9199-4863-a2ab-aa8397e8cbbb" containerName="registry" containerID="cri-o://d88e772a3f606d24efae5bc9e4925f3260099ffe19b7e03e6eb039253688b227" gracePeriod=30
Sep 07 11:52:17.817800 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:52:17.817776    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-676764f5b5-dsf9f"
Sep 07 11:52:17.902803 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:52:17.902726    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/9ae06e91-9199-4863-a2ab-aa8397e8cbbb-installation-pull-secrets\") pod \"9ae06e91-9199-4863-a2ab-aa8397e8cbbb\" (UID: \"9ae06e91-9199-4863-a2ab-aa8397e8cbbb\") "
Sep 07 11:52:17.902803 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:52:17.902764    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/9ae06e91-9199-4863-a2ab-aa8397e8cbbb-registry-certificates\") pod \"9ae06e91-9199-4863-a2ab-aa8397e8cbbb\" (UID: \"9ae06e91-9199-4863-a2ab-aa8397e8cbbb\") "
Sep 07 11:52:17.902803 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:52:17.902795    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/9ae06e91-9199-4863-a2ab-aa8397e8cbbb-ca-trust-extracted\") pod \"9ae06e91-9199-4863-a2ab-aa8397e8cbbb\" (UID: \"9ae06e91-9199-4863-a2ab-aa8397e8cbbb\") "
Sep 07 11:52:17.903041 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:52:17.902830    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/9ae06e91-9199-4863-a2ab-aa8397e8cbbb-bound-sa-token\") pod \"9ae06e91-9199-4863-a2ab-aa8397e8cbbb\" (UID: \"9ae06e91-9199-4863-a2ab-aa8397e8cbbb\") "
Sep 07 11:52:17.903041 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:52:17.902876    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9ae06e91-9199-4863-a2ab-aa8397e8cbbb-registry-tls\") pod \"9ae06e91-9199-4863-a2ab-aa8397e8cbbb\" (UID: \"9ae06e91-9199-4863-a2ab-aa8397e8cbbb\") "
Sep 07 11:52:17.903041 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:52:17.902898    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-twzgv\" (UniqueName: \"kubernetes.io/projected/9ae06e91-9199-4863-a2ab-aa8397e8cbbb-kube-api-access-twzgv\") pod \"9ae06e91-9199-4863-a2ab-aa8397e8cbbb\" (UID: \"9ae06e91-9199-4863-a2ab-aa8397e8cbbb\") "
Sep 07 11:52:17.903041 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:52:17.902953    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/9ae06e91-9199-4863-a2ab-aa8397e8cbbb-trusted-ca\") pod \"9ae06e91-9199-4863-a2ab-aa8397e8cbbb\" (UID: \"9ae06e91-9199-4863-a2ab-aa8397e8cbbb\") "
Sep 07 11:52:17.903041 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:52:17.903010    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/9ae06e91-9199-4863-a2ab-aa8397e8cbbb-image-registry-private-configuration\") pod \"9ae06e91-9199-4863-a2ab-aa8397e8cbbb\" (UID: \"9ae06e91-9199-4863-a2ab-aa8397e8cbbb\") "
Sep 07 11:52:17.903269 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:52:17.903189    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9ae06e91-9199-4863-a2ab-aa8397e8cbbb-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "9ae06e91-9199-4863-a2ab-aa8397e8cbbb" (UID: "9ae06e91-9199-4863-a2ab-aa8397e8cbbb"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 07 11:52:17.903325 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:52:17.903306    2563 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/9ae06e91-9199-4863-a2ab-aa8397e8cbbb-registry-certificates\") on node \"ip-10-0-135-173.ec2.internal\" DevicePath \"\""
Sep 07 11:52:17.903774 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:52:17.903746    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9ae06e91-9199-4863-a2ab-aa8397e8cbbb-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "9ae06e91-9199-4863-a2ab-aa8397e8cbbb" (UID: "9ae06e91-9199-4863-a2ab-aa8397e8cbbb"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 07 11:52:17.905183 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:52:17.905156    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9ae06e91-9199-4863-a2ab-aa8397e8cbbb-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "9ae06e91-9199-4863-a2ab-aa8397e8cbbb" (UID: "9ae06e91-9199-4863-a2ab-aa8397e8cbbb"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 07 11:52:17.905389 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:52:17.905359    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9ae06e91-9199-4863-a2ab-aa8397e8cbbb-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "9ae06e91-9199-4863-a2ab-aa8397e8cbbb" (UID: "9ae06e91-9199-4863-a2ab-aa8397e8cbbb"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 11:52:17.905741 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:52:17.905630    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9ae06e91-9199-4863-a2ab-aa8397e8cbbb-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "9ae06e91-9199-4863-a2ab-aa8397e8cbbb" (UID: "9ae06e91-9199-4863-a2ab-aa8397e8cbbb"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 07 11:52:17.905741 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:52:17.905662    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9ae06e91-9199-4863-a2ab-aa8397e8cbbb-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "9ae06e91-9199-4863-a2ab-aa8397e8cbbb" (UID: "9ae06e91-9199-4863-a2ab-aa8397e8cbbb"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 11:52:17.905741 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:52:17.905706    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9ae06e91-9199-4863-a2ab-aa8397e8cbbb-kube-api-access-twzgv" (OuterVolumeSpecName: "kube-api-access-twzgv") pod "9ae06e91-9199-4863-a2ab-aa8397e8cbbb" (UID: "9ae06e91-9199-4863-a2ab-aa8397e8cbbb"). InnerVolumeSpecName "kube-api-access-twzgv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 11:52:17.911592 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:52:17.911570    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9ae06e91-9199-4863-a2ab-aa8397e8cbbb-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "9ae06e91-9199-4863-a2ab-aa8397e8cbbb" (UID: "9ae06e91-9199-4863-a2ab-aa8397e8cbbb"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 11:52:18.004342 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:52:18.004314    2563 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/9ae06e91-9199-4863-a2ab-aa8397e8cbbb-bound-sa-token\") on node \"ip-10-0-135-173.ec2.internal\" DevicePath \"\""
Sep 07 11:52:18.004342 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:52:18.004342    2563 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9ae06e91-9199-4863-a2ab-aa8397e8cbbb-registry-tls\") on node \"ip-10-0-135-173.ec2.internal\" DevicePath \"\""
Sep 07 11:52:18.004487 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:52:18.004356    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-twzgv\" (UniqueName: \"kubernetes.io/projected/9ae06e91-9199-4863-a2ab-aa8397e8cbbb-kube-api-access-twzgv\") on node \"ip-10-0-135-173.ec2.internal\" DevicePath \"\""
Sep 07 11:52:18.004487 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:52:18.004372    2563 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/9ae06e91-9199-4863-a2ab-aa8397e8cbbb-trusted-ca\") on node \"ip-10-0-135-173.ec2.internal\" DevicePath \"\""
Sep 07 11:52:18.004487 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:52:18.004386    2563 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/9ae06e91-9199-4863-a2ab-aa8397e8cbbb-image-registry-private-configuration\") on node \"ip-10-0-135-173.ec2.internal\" DevicePath \"\""
Sep 07 11:52:18.004487 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:52:18.004400    2563 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/9ae06e91-9199-4863-a2ab-aa8397e8cbbb-installation-pull-secrets\") on node \"ip-10-0-135-173.ec2.internal\" DevicePath \"\""
Sep 07 11:52:18.004487 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:52:18.004438    2563 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/9ae06e91-9199-4863-a2ab-aa8397e8cbbb-ca-trust-extracted\") on node \"ip-10-0-135-173.ec2.internal\" DevicePath \"\""
Sep 07 11:52:18.038328 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:52:18.038308    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-4nfbc_b34ca85d-ac51-4724-a508-1026106b3f3a/console-operator/2.log"
Sep 07 11:52:18.038674 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:52:18.038397    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-85b454874b-4nfbc" event={"ID":"b34ca85d-ac51-4724-a508-1026106b3f3a","Type":"ContainerStarted","Data":"2a967dbf82a2ed2674935445b9e6f91a121df376181b02e71052040cae999d82"}
Sep 07 11:52:18.038753 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:52:18.038731    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-85b454874b-4nfbc"
Sep 07 11:52:18.039773 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:52:18.039751    2563 generic.go:356] "Generic (PLEG): container finished" podID="9ae06e91-9199-4863-a2ab-aa8397e8cbbb" containerID="d88e772a3f606d24efae5bc9e4925f3260099ffe19b7e03e6eb039253688b227" exitCode=0
Sep 07 11:52:18.039859 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:52:18.039784    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-676764f5b5-dsf9f"
Sep 07 11:52:18.039859 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:52:18.039830    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-676764f5b5-dsf9f" event={"ID":"9ae06e91-9199-4863-a2ab-aa8397e8cbbb","Type":"ContainerDied","Data":"d88e772a3f606d24efae5bc9e4925f3260099ffe19b7e03e6eb039253688b227"}
Sep 07 11:52:18.039926 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:52:18.039875    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-676764f5b5-dsf9f" event={"ID":"9ae06e91-9199-4863-a2ab-aa8397e8cbbb","Type":"ContainerDied","Data":"b8962cbfcdc6ac731ce93cea66541f8ceecac8294a71a703f6cdf9cd99db88c2"}
Sep 07 11:52:18.039926 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:52:18.039894    2563 scope.go:117] "RemoveContainer" containerID="d88e772a3f606d24efae5bc9e4925f3260099ffe19b7e03e6eb039253688b227"
Sep 07 11:52:18.049070 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:52:18.049047    2563 scope.go:117] "RemoveContainer" containerID="d88e772a3f606d24efae5bc9e4925f3260099ffe19b7e03e6eb039253688b227"
Sep 07 11:52:18.049331 ip-10-0-135-173 kubenswrapper[2563]: E0907 11:52:18.049313    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d88e772a3f606d24efae5bc9e4925f3260099ffe19b7e03e6eb039253688b227\": container with ID starting with d88e772a3f606d24efae5bc9e4925f3260099ffe19b7e03e6eb039253688b227 not found: ID does not exist" containerID="d88e772a3f606d24efae5bc9e4925f3260099ffe19b7e03e6eb039253688b227"
Sep 07 11:52:18.049396 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:52:18.049338    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d88e772a3f606d24efae5bc9e4925f3260099ffe19b7e03e6eb039253688b227"} err="failed to get container status \"d88e772a3f606d24efae5bc9e4925f3260099ffe19b7e03e6eb039253688b227\": rpc error: code = NotFound desc = could not find container \"d88e772a3f606d24efae5bc9e4925f3260099ffe19b7e03e6eb039253688b227\": container with ID starting with d88e772a3f606d24efae5bc9e4925f3260099ffe19b7e03e6eb039253688b227 not found: ID does not exist"
Sep 07 11:52:18.053833 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:52:18.053793    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-85b454874b-4nfbc" podStartSLOduration=50.587976535 podStartE2EDuration="53.053781297s" totalImagesPullingTime="2.465804762s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 11:51:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 11:52:18.052394146 +0000 UTC m=+309.571125962" watchObservedRunningTime="2026-09-07 11:52:18.053781297 +0000 UTC m=+309.572513145"
Sep 07 11:52:18.062969 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:52:18.062950    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-676764f5b5-dsf9f"]
Sep 07 11:52:18.064808 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:52:18.064789    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-676764f5b5-dsf9f"]
Sep 07 11:52:18.087732 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:52:18.087712    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-85b454874b-4nfbc"
Sep 07 11:52:18.275318 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:52:18.275282    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 11:52:18.374875 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:52:18.374847    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 11:52:18.476707 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:52:18.476675    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 11:52:18.577073 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:52:18.576995    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 11:52:18.950329 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:52:18.950291    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 11:52:19.006020 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:52:19.005991    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 11:52:19.026333 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:52:19.026312    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-65bdc89fcb-8tfm6"
Sep 07 11:52:19.038866 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:52:19.038840    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9ae06e91-9199-4863-a2ab-aa8397e8cbbb" path="/var/lib/kubelet/pods/9ae06e91-9199-4863-a2ab-aa8397e8cbbb/volumes"
Sep 07 11:52:19.341375 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:52:19.341293    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 11:52:19.564650 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:52:19.564619    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 11:52:19.663412 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:52:19.663333    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 11:52:19.763762 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:52:19.763723    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 11:52:19.864328 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:52:19.864292    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 11:52:19.974304 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:52:19.974275    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 11:52:20.073392 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:52:20.073358    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 11:52:20.172079 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:52:20.172046    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 11:52:20.271540 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:52:20.271459    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 11:52:32.151258 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:52:32.151225    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 11:52:32.205574 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:52:32.205545    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 11:52:34.566885 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:52:34.566851    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 11:52:34.615354 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:52:34.615319    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 11:52:36.895504 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:52:36.895466    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 11:52:36.949267 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:52:36.949231    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 11:52:39.118322 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:52:39.118239    2563 generic.go:356] "Generic (PLEG): container finished" podID="b6023824-522a-4bed-b277-7394abe09bbc" containerID="66ca367eafbc2c34a13a95755721461617f1f4e756dcd9f3cfb0728b362c31cf" exitCode=0
Sep 07 11:52:39.118700 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:52:39.118319    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-76bcd6ccf5-zls5m" event={"ID":"b6023824-522a-4bed-b277-7394abe09bbc","Type":"ContainerDied","Data":"66ca367eafbc2c34a13a95755721461617f1f4e756dcd9f3cfb0728b362c31cf"}
Sep 07 11:52:39.118700 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:52:39.118655    2563 scope.go:117] "RemoveContainer" containerID="66ca367eafbc2c34a13a95755721461617f1f4e756dcd9f3cfb0728b362c31cf"
Sep 07 11:52:39.774508 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:52:39.774454    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 11:52:40.125076 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:52:40.124989    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-76bcd6ccf5-zls5m" event={"ID":"b6023824-522a-4bed-b277-7394abe09bbc","Type":"ContainerStarted","Data":"d8930af04dee23160970377a1b52bc0d4c6b58a8d6265e632cdef1fb92c9247e"}
Sep 07 11:52:44.167622 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:52:44.167580    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 11:52:44.219932 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:52:44.219900    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 11:52:46.369847 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:52:46.369793    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 11:52:46.470972 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:52:46.470933    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 11:52:53.965858 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:52:53.965818    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 11:52:59.966606 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:52:59.966566    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 11:53:06.892603 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:53:06.892518    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 11:53:06.943617 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:53:06.943590    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 11:53:09.109383 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:53:09.109293    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 11:53:21.673284 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:53:21.673248    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 11:53:21.704192 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:53:21.704166    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3acaae8d-5930-49c6-9b9e-d286ac6a75d5-metrics-certs\") pod \"network-metrics-daemon-xf2gj\" (UID: \"3acaae8d-5930-49c6-9b9e-d286ac6a75d5\") " pod="openshift-multus/network-metrics-daemon-xf2gj"
Sep 07 11:53:21.706498 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:53:21.706471    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3acaae8d-5930-49c6-9b9e-d286ac6a75d5-metrics-certs\") pod \"network-metrics-daemon-xf2gj\" (UID: \"3acaae8d-5930-49c6-9b9e-d286ac6a75d5\") " pod="openshift-multus/network-metrics-daemon-xf2gj"
Sep 07 11:53:21.774278 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:53:21.774243    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 11:53:21.937638 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:53:21.937552    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-gtblx\""
Sep 07 11:53:21.945961 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:53:21.945938    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xf2gj"
Sep 07 11:53:22.214500 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:53:22.214467    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-xf2gj"]
Sep 07 11:53:22.217139 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:53:22.217113    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3acaae8d_5930_49c6_9b9e_d286ac6a75d5.slice/crio-d09afb30edee38e5c1da127e6889c8a740f9a13f0a9bfc75d1e46d1da1dbd370 WatchSource:0}: Error finding container d09afb30edee38e5c1da127e6889c8a740f9a13f0a9bfc75d1e46d1da1dbd370: Status 404 returned error can't find the container with id d09afb30edee38e5c1da127e6889c8a740f9a13f0a9bfc75d1e46d1da1dbd370
Sep 07 11:53:22.288608 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:53:22.288575    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-xf2gj" event={"ID":"3acaae8d-5930-49c6-9b9e-d286ac6a75d5","Type":"ContainerStarted","Data":"d09afb30edee38e5c1da127e6889c8a740f9a13f0a9bfc75d1e46d1da1dbd370"}
Sep 07 11:53:23.294280 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:53:23.294206    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-xf2gj" event={"ID":"3acaae8d-5930-49c6-9b9e-d286ac6a75d5","Type":"ContainerStarted","Data":"c0e872c28249430c170db05be94956cc7a2efecf11078365728af2944fd66ce1"}
Sep 07 11:53:23.294280 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:53:23.294262    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-xf2gj" event={"ID":"3acaae8d-5930-49c6-9b9e-d286ac6a75d5","Type":"ContainerStarted","Data":"994b84c76981af5df3f8f8a7070b4ee9ef2cb1c000ef6c416f7d68a55f4c1643"}
Sep 07 11:53:23.308109 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:53:23.308059    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-xf2gj" podStartSLOduration=373.48289917 podStartE2EDuration="6m14.308043284s" totalImagesPullingTime="825.144114ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 11:47:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 11:53:23.306877384 +0000 UTC m=+374.825609196" watchObservedRunningTime="2026-09-07 11:53:23.308043284 +0000 UTC m=+374.826775098"
Sep 07 11:53:23.388574 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:53:23.388546    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 11:53:23.445866 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:53:23.445836    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 11:53:26.119056 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:53:26.118999    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 11:53:26.217213 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:53:26.217174    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 11:53:27.174372 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:53:27.174338    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 07 11:53:27.174885 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:53:27.174624    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="57a9b372-5c0c-48e0-b0a2-49814095da99" containerName="alertmanager" containerID="cri-o://df4d5f3cbec9461b05eac65ccd7ae1e1b5076a14e395ac487ded763915493caa" gracePeriod=120
Sep 07 11:53:27.174885 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:53:27.174687    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="57a9b372-5c0c-48e0-b0a2-49814095da99" containerName="kube-rbac-proxy-metric" containerID="cri-o://7a3d701ece196e6e116809dc39653365323a106702438f2d0f49bf8fa28b7e3a" gracePeriod=120
Sep 07 11:53:27.174885 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:53:27.174729    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="57a9b372-5c0c-48e0-b0a2-49814095da99" containerName="prom-label-proxy" containerID="cri-o://26d835dca455ab83cd5168987b5846d693f0097ac987b8495e5010b46a1eb3b4" gracePeriod=120
Sep 07 11:53:27.174885 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:53:27.174771    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="57a9b372-5c0c-48e0-b0a2-49814095da99" containerName="kube-rbac-proxy" containerID="cri-o://230b0afbfc819e47df67117ce1da56cf708e92f6cb15077c68a29e90e4cc89eb" gracePeriod=120
Sep 07 11:53:27.174885 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:53:27.174805    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="57a9b372-5c0c-48e0-b0a2-49814095da99" containerName="config-reloader" containerID="cri-o://c1e38b4b7e9c090a29836bbf25c4de203bab3b7ad31f2d2024d6b599e31254d5" gracePeriod=120
Sep 07 11:53:27.174885 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:53:27.174755    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="57a9b372-5c0c-48e0-b0a2-49814095da99" containerName="kube-rbac-proxy-web" containerID="cri-o://52d8b7bb0ed9420e3691f9dd8ae3cec051bb21e091fdd26964047e9b10844f2f" gracePeriod=120
Sep 07 11:53:27.317552 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:53:27.317523    2563 generic.go:356] "Generic (PLEG): container finished" podID="57a9b372-5c0c-48e0-b0a2-49814095da99" containerID="26d835dca455ab83cd5168987b5846d693f0097ac987b8495e5010b46a1eb3b4" exitCode=0
Sep 07 11:53:27.317552 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:53:27.317546    2563 generic.go:356] "Generic (PLEG): container finished" podID="57a9b372-5c0c-48e0-b0a2-49814095da99" containerID="230b0afbfc819e47df67117ce1da56cf708e92f6cb15077c68a29e90e4cc89eb" exitCode=0
Sep 07 11:53:27.317552 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:53:27.317553    2563 generic.go:356] "Generic (PLEG): container finished" podID="57a9b372-5c0c-48e0-b0a2-49814095da99" containerID="c1e38b4b7e9c090a29836bbf25c4de203bab3b7ad31f2d2024d6b599e31254d5" exitCode=0
Sep 07 11:53:27.317711 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:53:27.317558    2563 generic.go:356] "Generic (PLEG): container finished" podID="57a9b372-5c0c-48e0-b0a2-49814095da99" containerID="df4d5f3cbec9461b05eac65ccd7ae1e1b5076a14e395ac487ded763915493caa" exitCode=0
Sep 07 11:53:27.317711 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:53:27.317590    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"57a9b372-5c0c-48e0-b0a2-49814095da99","Type":"ContainerDied","Data":"26d835dca455ab83cd5168987b5846d693f0097ac987b8495e5010b46a1eb3b4"}
Sep 07 11:53:27.317711 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:53:27.317649    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"57a9b372-5c0c-48e0-b0a2-49814095da99","Type":"ContainerDied","Data":"230b0afbfc819e47df67117ce1da56cf708e92f6cb15077c68a29e90e4cc89eb"}
Sep 07 11:53:27.317711 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:53:27.317678    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"57a9b372-5c0c-48e0-b0a2-49814095da99","Type":"ContainerDied","Data":"c1e38b4b7e9c090a29836bbf25c4de203bab3b7ad31f2d2024d6b599e31254d5"}
Sep 07 11:53:27.317711 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:53:27.317703    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"57a9b372-5c0c-48e0-b0a2-49814095da99","Type":"ContainerDied","Data":"df4d5f3cbec9461b05eac65ccd7ae1e1b5076a14e395ac487ded763915493caa"}
Sep 07 11:53:28.327853 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:53:28.327823    2563 generic.go:356] "Generic (PLEG): container finished" podID="57a9b372-5c0c-48e0-b0a2-49814095da99" containerID="7a3d701ece196e6e116809dc39653365323a106702438f2d0f49bf8fa28b7e3a" exitCode=0
Sep 07 11:53:28.327853 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:53:28.327849    2563 generic.go:356] "Generic (PLEG): container finished" podID="57a9b372-5c0c-48e0-b0a2-49814095da99" containerID="52d8b7bb0ed9420e3691f9dd8ae3cec051bb21e091fdd26964047e9b10844f2f" exitCode=0
Sep 07 11:53:28.328205 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:53:28.327869    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"57a9b372-5c0c-48e0-b0a2-49814095da99","Type":"ContainerDied","Data":"7a3d701ece196e6e116809dc39653365323a106702438f2d0f49bf8fa28b7e3a"}
Sep 07 11:53:28.328205 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:53:28.327922    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"57a9b372-5c0c-48e0-b0a2-49814095da99","Type":"ContainerDied","Data":"52d8b7bb0ed9420e3691f9dd8ae3cec051bb21e091fdd26964047e9b10844f2f"}
Sep 07 11:53:28.413041 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:53:28.413021    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 07 11:53:28.459712 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:53:28.459687    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6qnvb\" (UniqueName: \"kubernetes.io/projected/57a9b372-5c0c-48e0-b0a2-49814095da99-kube-api-access-6qnvb\") pod \"57a9b372-5c0c-48e0-b0a2-49814095da99\" (UID: \"57a9b372-5c0c-48e0-b0a2-49814095da99\") "
Sep 07 11:53:28.459870 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:53:28.459721    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/57a9b372-5c0c-48e0-b0a2-49814095da99-alertmanager-trusted-ca-bundle\") pod \"57a9b372-5c0c-48e0-b0a2-49814095da99\" (UID: \"57a9b372-5c0c-48e0-b0a2-49814095da99\") "
Sep 07 11:53:28.459870 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:53:28.459741    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/57a9b372-5c0c-48e0-b0a2-49814095da99-secret-alertmanager-kube-rbac-proxy\") pod \"57a9b372-5c0c-48e0-b0a2-49814095da99\" (UID: \"57a9b372-5c0c-48e0-b0a2-49814095da99\") "
Sep 07 11:53:28.459870 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:53:28.459768    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/57a9b372-5c0c-48e0-b0a2-49814095da99-secret-alertmanager-kube-rbac-proxy-metric\") pod \"57a9b372-5c0c-48e0-b0a2-49814095da99\" (UID: \"57a9b372-5c0c-48e0-b0a2-49814095da99\") "
Sep 07 11:53:28.459870 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:53:28.459799    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/57a9b372-5c0c-48e0-b0a2-49814095da99-alertmanager-main-db\") pod \"57a9b372-5c0c-48e0-b0a2-49814095da99\" (UID: \"57a9b372-5c0c-48e0-b0a2-49814095da99\") "
Sep 07 11:53:28.459870 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:53:28.459832    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/57a9b372-5c0c-48e0-b0a2-49814095da99-metrics-client-ca\") pod \"57a9b372-5c0c-48e0-b0a2-49814095da99\" (UID: \"57a9b372-5c0c-48e0-b0a2-49814095da99\") "
Sep 07 11:53:28.459870 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:53:28.459857    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/57a9b372-5c0c-48e0-b0a2-49814095da99-web-config\") pod \"57a9b372-5c0c-48e0-b0a2-49814095da99\" (UID: \"57a9b372-5c0c-48e0-b0a2-49814095da99\") "
Sep 07 11:53:28.460180 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:53:28.459883    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/57a9b372-5c0c-48e0-b0a2-49814095da99-secret-alertmanager-main-tls\") pod \"57a9b372-5c0c-48e0-b0a2-49814095da99\" (UID: \"57a9b372-5c0c-48e0-b0a2-49814095da99\") "
Sep 07 11:53:28.460180 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:53:28.459908    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/57a9b372-5c0c-48e0-b0a2-49814095da99-tls-assets\") pod \"57a9b372-5c0c-48e0-b0a2-49814095da99\" (UID: \"57a9b372-5c0c-48e0-b0a2-49814095da99\") "
Sep 07 11:53:28.460180 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:53:28.459941    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/57a9b372-5c0c-48e0-b0a2-49814095da99-config-out\") pod \"57a9b372-5c0c-48e0-b0a2-49814095da99\" (UID: \"57a9b372-5c0c-48e0-b0a2-49814095da99\") "
Sep 07 11:53:28.460180 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:53:28.459968    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/57a9b372-5c0c-48e0-b0a2-49814095da99-secret-alertmanager-kube-rbac-proxy-web\") pod \"57a9b372-5c0c-48e0-b0a2-49814095da99\" (UID: \"57a9b372-5c0c-48e0-b0a2-49814095da99\") "
Sep 07 11:53:28.460180 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:53:28.459998    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/57a9b372-5c0c-48e0-b0a2-49814095da99-cluster-tls-config\") pod \"57a9b372-5c0c-48e0-b0a2-49814095da99\" (UID: \"57a9b372-5c0c-48e0-b0a2-49814095da99\") "
Sep 07 11:53:28.460180 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:53:28.460047    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/57a9b372-5c0c-48e0-b0a2-49814095da99-config-volume\") pod \"57a9b372-5c0c-48e0-b0a2-49814095da99\" (UID: \"57a9b372-5c0c-48e0-b0a2-49814095da99\") "
Sep 07 11:53:28.460180 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:53:28.460119    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/57a9b372-5c0c-48e0-b0a2-49814095da99-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "57a9b372-5c0c-48e0-b0a2-49814095da99" (UID: "57a9b372-5c0c-48e0-b0a2-49814095da99"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 07 11:53:28.460528 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:53:28.460254    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/57a9b372-5c0c-48e0-b0a2-49814095da99-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "57a9b372-5c0c-48e0-b0a2-49814095da99" (UID: "57a9b372-5c0c-48e0-b0a2-49814095da99"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 11:53:28.460528 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:53:28.460355    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/57a9b372-5c0c-48e0-b0a2-49814095da99-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "57a9b372-5c0c-48e0-b0a2-49814095da99" (UID: "57a9b372-5c0c-48e0-b0a2-49814095da99"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 07 11:53:28.460528 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:53:28.460358    2563 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/57a9b372-5c0c-48e0-b0a2-49814095da99-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-135-173.ec2.internal\" DevicePath \"\""
Sep 07 11:53:28.460528 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:53:28.460387    2563 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/57a9b372-5c0c-48e0-b0a2-49814095da99-alertmanager-main-db\") on node \"ip-10-0-135-173.ec2.internal\" DevicePath \"\""
Sep 07 11:53:28.462472 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:53:28.462442    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/57a9b372-5c0c-48e0-b0a2-49814095da99-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "57a9b372-5c0c-48e0-b0a2-49814095da99" (UID: "57a9b372-5c0c-48e0-b0a2-49814095da99"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 07 11:53:28.462752 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:53:28.462679    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/57a9b372-5c0c-48e0-b0a2-49814095da99-config-volume" (OuterVolumeSpecName: "config-volume") pod "57a9b372-5c0c-48e0-b0a2-49814095da99" (UID: "57a9b372-5c0c-48e0-b0a2-49814095da99"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 07 11:53:28.463479 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:53:28.463446    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/57a9b372-5c0c-48e0-b0a2-49814095da99-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "57a9b372-5c0c-48e0-b0a2-49814095da99" (UID: "57a9b372-5c0c-48e0-b0a2-49814095da99"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 11:53:28.463479 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:53:28.463453    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/57a9b372-5c0c-48e0-b0a2-49814095da99-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "57a9b372-5c0c-48e0-b0a2-49814095da99" (UID: "57a9b372-5c0c-48e0-b0a2-49814095da99"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 07 11:53:28.463770 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:53:28.463741    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/57a9b372-5c0c-48e0-b0a2-49814095da99-kube-api-access-6qnvb" (OuterVolumeSpecName: "kube-api-access-6qnvb") pod "57a9b372-5c0c-48e0-b0a2-49814095da99" (UID: "57a9b372-5c0c-48e0-b0a2-49814095da99"). InnerVolumeSpecName "kube-api-access-6qnvb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 11:53:28.464233 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:53:28.464212    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/57a9b372-5c0c-48e0-b0a2-49814095da99-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "57a9b372-5c0c-48e0-b0a2-49814095da99" (UID: "57a9b372-5c0c-48e0-b0a2-49814095da99"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 07 11:53:28.464630 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:53:28.464603    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/57a9b372-5c0c-48e0-b0a2-49814095da99-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "57a9b372-5c0c-48e0-b0a2-49814095da99" (UID: "57a9b372-5c0c-48e0-b0a2-49814095da99"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 07 11:53:28.464915 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:53:28.464889    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/57a9b372-5c0c-48e0-b0a2-49814095da99-config-out" (OuterVolumeSpecName: "config-out") pod "57a9b372-5c0c-48e0-b0a2-49814095da99" (UID: "57a9b372-5c0c-48e0-b0a2-49814095da99"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 11:53:28.468044 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:53:28.468024    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/57a9b372-5c0c-48e0-b0a2-49814095da99-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "57a9b372-5c0c-48e0-b0a2-49814095da99" (UID: "57a9b372-5c0c-48e0-b0a2-49814095da99"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 07 11:53:28.473837 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:53:28.473809    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/57a9b372-5c0c-48e0-b0a2-49814095da99-web-config" (OuterVolumeSpecName: "web-config") pod "57a9b372-5c0c-48e0-b0a2-49814095da99" (UID: "57a9b372-5c0c-48e0-b0a2-49814095da99"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 07 11:53:28.561039 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:53:28.560965    2563 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/57a9b372-5c0c-48e0-b0a2-49814095da99-config-volume\") on node \"ip-10-0-135-173.ec2.internal\" DevicePath \"\""
Sep 07 11:53:28.561039 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:53:28.560999    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6qnvb\" (UniqueName: \"kubernetes.io/projected/57a9b372-5c0c-48e0-b0a2-49814095da99-kube-api-access-6qnvb\") on node \"ip-10-0-135-173.ec2.internal\" DevicePath \"\""
Sep 07 11:53:28.561039 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:53:28.561013    2563 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/57a9b372-5c0c-48e0-b0a2-49814095da99-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-135-173.ec2.internal\" DevicePath \"\""
Sep 07 11:53:28.561039 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:53:28.561026    2563 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/57a9b372-5c0c-48e0-b0a2-49814095da99-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-135-173.ec2.internal\" DevicePath \"\""
Sep 07 11:53:28.561039 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:53:28.561042    2563 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/57a9b372-5c0c-48e0-b0a2-49814095da99-metrics-client-ca\") on node \"ip-10-0-135-173.ec2.internal\" DevicePath \"\""
Sep 07 11:53:28.561268 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:53:28.561055    2563 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/57a9b372-5c0c-48e0-b0a2-49814095da99-web-config\") on node \"ip-10-0-135-173.ec2.internal\" DevicePath \"\""
Sep 07 11:53:28.561268 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:53:28.561067    2563 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/57a9b372-5c0c-48e0-b0a2-49814095da99-secret-alertmanager-main-tls\") on node \"ip-10-0-135-173.ec2.internal\" DevicePath \"\""
Sep 07 11:53:28.561268 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:53:28.561078    2563 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/57a9b372-5c0c-48e0-b0a2-49814095da99-tls-assets\") on node \"ip-10-0-135-173.ec2.internal\" DevicePath \"\""
Sep 07 11:53:28.561268 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:53:28.561089    2563 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/57a9b372-5c0c-48e0-b0a2-49814095da99-config-out\") on node \"ip-10-0-135-173.ec2.internal\" DevicePath \"\""
Sep 07 11:53:28.561268 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:53:28.561100    2563 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/57a9b372-5c0c-48e0-b0a2-49814095da99-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-135-173.ec2.internal\" DevicePath \"\""
Sep 07 11:53:28.561268 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:53:28.561113    2563 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/57a9b372-5c0c-48e0-b0a2-49814095da99-cluster-tls-config\") on node \"ip-10-0-135-173.ec2.internal\" DevicePath \"\""
Sep 07 11:53:29.335366 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:53:29.335285    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"57a9b372-5c0c-48e0-b0a2-49814095da99","Type":"ContainerDied","Data":"bc552b4d6c357e1b7720db0b396eba16cf91371f9b314c304994d9e607d68e18"}
Sep 07 11:53:29.335366 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:53:29.335326    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 07 11:53:29.335997 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:53:29.335393    2563 scope.go:117] "RemoveContainer" containerID="26d835dca455ab83cd5168987b5846d693f0097ac987b8495e5010b46a1eb3b4"
Sep 07 11:53:29.343110 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:53:29.343097    2563 scope.go:117] "RemoveContainer" containerID="7a3d701ece196e6e116809dc39653365323a106702438f2d0f49bf8fa28b7e3a"
Sep 07 11:53:29.349745 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:53:29.349727    2563 scope.go:117] "RemoveContainer" containerID="230b0afbfc819e47df67117ce1da56cf708e92f6cb15077c68a29e90e4cc89eb"
Sep 07 11:53:29.353343 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:53:29.353322    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 07 11:53:29.357334 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:53:29.357311    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 07 11:53:29.357728 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:53:29.357711    2563 scope.go:117] "RemoveContainer" containerID="52d8b7bb0ed9420e3691f9dd8ae3cec051bb21e091fdd26964047e9b10844f2f"
Sep 07 11:53:29.364027 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:53:29.364010    2563 scope.go:117] "RemoveContainer" containerID="c1e38b4b7e9c090a29836bbf25c4de203bab3b7ad31f2d2024d6b599e31254d5"
Sep 07 11:53:29.370384 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:53:29.370367    2563 scope.go:117] "RemoveContainer" containerID="df4d5f3cbec9461b05eac65ccd7ae1e1b5076a14e395ac487ded763915493caa"
Sep 07 11:53:29.377251 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:53:29.377237    2563 scope.go:117] "RemoveContainer" containerID="bd31040e3f5fe153bb53c4bdc1a4af3beb7c6ea8d8806a8711ca3c409d5c62b0"
Sep 07 11:53:29.381446 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:53:29.381427    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 07 11:53:29.381690 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:53:29.381679    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9ae06e91-9199-4863-a2ab-aa8397e8cbbb" containerName="registry"
Sep 07 11:53:29.381743 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:53:29.381691    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="9ae06e91-9199-4863-a2ab-aa8397e8cbbb" containerName="registry"
Sep 07 11:53:29.381743 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:53:29.381703    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="57a9b372-5c0c-48e0-b0a2-49814095da99" containerName="init-config-reloader"
Sep 07 11:53:29.381743 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:53:29.381709    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="57a9b372-5c0c-48e0-b0a2-49814095da99" containerName="init-config-reloader"
Sep 07 11:53:29.381743 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:53:29.381717    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="57a9b372-5c0c-48e0-b0a2-49814095da99" containerName="kube-rbac-proxy-web"
Sep 07 11:53:29.381743 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:53:29.381723    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="57a9b372-5c0c-48e0-b0a2-49814095da99" containerName="kube-rbac-proxy-web"
Sep 07 11:53:29.381743 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:53:29.381731    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="57a9b372-5c0c-48e0-b0a2-49814095da99" containerName="kube-rbac-proxy-metric"
Sep 07 11:53:29.381743 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:53:29.381736    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="57a9b372-5c0c-48e0-b0a2-49814095da99" containerName="kube-rbac-proxy-metric"
Sep 07 11:53:29.381956 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:53:29.381747    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="57a9b372-5c0c-48e0-b0a2-49814095da99" containerName="alertmanager"
Sep 07 11:53:29.381956 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:53:29.381753    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="57a9b372-5c0c-48e0-b0a2-49814095da99" containerName="alertmanager"
Sep 07 11:53:29.381956 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:53:29.381759    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="57a9b372-5c0c-48e0-b0a2-49814095da99" containerName="config-reloader"
Sep 07 11:53:29.381956 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:53:29.381764    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="57a9b372-5c0c-48e0-b0a2-49814095da99" containerName="config-reloader"
Sep 07 11:53:29.381956 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:53:29.381770    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="57a9b372-5c0c-48e0-b0a2-49814095da99" containerName="kube-rbac-proxy"
Sep 07 11:53:29.381956 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:53:29.381774    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="57a9b372-5c0c-48e0-b0a2-49814095da99" containerName="kube-rbac-proxy"
Sep 07 11:53:29.381956 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:53:29.381783    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="57a9b372-5c0c-48e0-b0a2-49814095da99" containerName="prom-label-proxy"
Sep 07 11:53:29.381956 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:53:29.381788    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="57a9b372-5c0c-48e0-b0a2-49814095da99" containerName="prom-label-proxy"
Sep 07 11:53:29.381956 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:53:29.381833    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="57a9b372-5c0c-48e0-b0a2-49814095da99" containerName="alertmanager"
Sep 07 11:53:29.381956 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:53:29.381842    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="57a9b372-5c0c-48e0-b0a2-49814095da99" containerName="config-reloader"
Sep 07 11:53:29.381956 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:53:29.381848    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="57a9b372-5c0c-48e0-b0a2-49814095da99" containerName="prom-label-proxy"
Sep 07 11:53:29.381956 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:53:29.381853    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="57a9b372-5c0c-48e0-b0a2-49814095da99" containerName="kube-rbac-proxy"
Sep 07 11:53:29.381956 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:53:29.381858    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="9ae06e91-9199-4863-a2ab-aa8397e8cbbb" containerName="registry"
Sep 07 11:53:29.381956 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:53:29.381864    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="57a9b372-5c0c-48e0-b0a2-49814095da99" containerName="kube-rbac-proxy-web"
Sep 07 11:53:29.381956 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:53:29.381870    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="57a9b372-5c0c-48e0-b0a2-49814095da99" containerName="kube-rbac-proxy-metric"
Sep 07 11:53:29.386655 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:53:29.386639    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 07 11:53:29.388328 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:53:29.388308    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Sep 07 11:53:29.388442 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:53:29.388308    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Sep 07 11:53:29.388633 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:53:29.388613    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Sep 07 11:53:29.388695 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:53:29.388613    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-5qlcc\""
Sep 07 11:53:29.388832 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:53:29.388816    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Sep 07 11:53:29.388944 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:53:29.388930    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Sep 07 11:53:29.389004 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:53:29.388991    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Sep 07 11:53:29.389258 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:53:29.389243    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Sep 07 11:53:29.389855 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:53:29.389835    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Sep 07 11:53:29.394145 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:53:29.394108    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Sep 07 11:53:29.397194 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:53:29.397174    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 07 11:53:29.466191 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:53:29.466165    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/5aade16a-29eb-43c8-8bc5-9889f0356fbf-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"5aade16a-29eb-43c8-8bc5-9889f0356fbf\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 07 11:53:29.466301 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:53:29.466200    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5aade16a-29eb-43c8-8bc5-9889f0356fbf-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"5aade16a-29eb-43c8-8bc5-9889f0356fbf\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 07 11:53:29.466301 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:53:29.466226    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/5aade16a-29eb-43c8-8bc5-9889f0356fbf-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"5aade16a-29eb-43c8-8bc5-9889f0356fbf\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 07 11:53:29.466301 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:53:29.466283    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/5aade16a-29eb-43c8-8bc5-9889f0356fbf-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"5aade16a-29eb-43c8-8bc5-9889f0356fbf\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 07 11:53:29.466398 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:53:29.466345    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/5aade16a-29eb-43c8-8bc5-9889f0356fbf-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"5aade16a-29eb-43c8-8bc5-9889f0356fbf\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 07 11:53:29.466398 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:53:29.466370    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/5aade16a-29eb-43c8-8bc5-9889f0356fbf-tls-assets\") pod \"alertmanager-main-0\" (UID: \"5aade16a-29eb-43c8-8bc5-9889f0356fbf\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 07 11:53:29.466398 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:53:29.466386    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/5aade16a-29eb-43c8-8bc5-9889f0356fbf-config-out\") pod \"alertmanager-main-0\" (UID: \"5aade16a-29eb-43c8-8bc5-9889f0356fbf\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 07 11:53:29.466521 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:53:29.466401    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/5aade16a-29eb-43c8-8bc5-9889f0356fbf-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"5aade16a-29eb-43c8-8bc5-9889f0356fbf\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 07 11:53:29.466521 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:53:29.466453    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/5aade16a-29eb-43c8-8bc5-9889f0356fbf-config-volume\") pod \"alertmanager-main-0\" (UID: \"5aade16a-29eb-43c8-8bc5-9889f0356fbf\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 07 11:53:29.466521 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:53:29.466479    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5aade16a-29eb-43c8-8bc5-9889f0356fbf-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"5aade16a-29eb-43c8-8bc5-9889f0356fbf\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 07 11:53:29.466521 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:53:29.466505    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/5aade16a-29eb-43c8-8bc5-9889f0356fbf-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"5aade16a-29eb-43c8-8bc5-9889f0356fbf\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 07 11:53:29.466521 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:53:29.466519    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/5aade16a-29eb-43c8-8bc5-9889f0356fbf-web-config\") pod \"alertmanager-main-0\" (UID: \"5aade16a-29eb-43c8-8bc5-9889f0356fbf\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 07 11:53:29.466671 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:53:29.466535    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9z8vg\" (UniqueName: \"kubernetes.io/projected/5aade16a-29eb-43c8-8bc5-9889f0356fbf-kube-api-access-9z8vg\") pod \"alertmanager-main-0\" (UID: \"5aade16a-29eb-43c8-8bc5-9889f0356fbf\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 07 11:53:29.567694 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:53:29.567670    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/5aade16a-29eb-43c8-8bc5-9889f0356fbf-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"5aade16a-29eb-43c8-8bc5-9889f0356fbf\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 07 11:53:29.567802 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:53:29.567698    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/5aade16a-29eb-43c8-8bc5-9889f0356fbf-web-config\") pod \"alertmanager-main-0\" (UID: \"5aade16a-29eb-43c8-8bc5-9889f0356fbf\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 07 11:53:29.567802 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:53:29.567715    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9z8vg\" (UniqueName: \"kubernetes.io/projected/5aade16a-29eb-43c8-8bc5-9889f0356fbf-kube-api-access-9z8vg\") pod \"alertmanager-main-0\" (UID: \"5aade16a-29eb-43c8-8bc5-9889f0356fbf\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 07 11:53:29.567802 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:53:29.567772    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/5aade16a-29eb-43c8-8bc5-9889f0356fbf-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"5aade16a-29eb-43c8-8bc5-9889f0356fbf\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 07 11:53:29.567926 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:53:29.567806    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5aade16a-29eb-43c8-8bc5-9889f0356fbf-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"5aade16a-29eb-43c8-8bc5-9889f0356fbf\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 07 11:53:29.567926 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:53:29.567854    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/5aade16a-29eb-43c8-8bc5-9889f0356fbf-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"5aade16a-29eb-43c8-8bc5-9889f0356fbf\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 07 11:53:29.567926 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:53:29.567889    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/5aade16a-29eb-43c8-8bc5-9889f0356fbf-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"5aade16a-29eb-43c8-8bc5-9889f0356fbf\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 07 11:53:29.568065 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:53:29.567922    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/5aade16a-29eb-43c8-8bc5-9889f0356fbf-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"5aade16a-29eb-43c8-8bc5-9889f0356fbf\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 07 11:53:29.568065 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:53:29.567955    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/5aade16a-29eb-43c8-8bc5-9889f0356fbf-tls-assets\") pod \"alertmanager-main-0\" (UID: \"5aade16a-29eb-43c8-8bc5-9889f0356fbf\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 07 11:53:29.568065 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:53:29.567980    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/5aade16a-29eb-43c8-8bc5-9889f0356fbf-config-out\") pod \"alertmanager-main-0\" (UID: \"5aade16a-29eb-43c8-8bc5-9889f0356fbf\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 07 11:53:29.568065 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:53:29.568006    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/5aade16a-29eb-43c8-8bc5-9889f0356fbf-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"5aade16a-29eb-43c8-8bc5-9889f0356fbf\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 07 11:53:29.568065 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:53:29.568034    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/5aade16a-29eb-43c8-8bc5-9889f0356fbf-config-volume\") pod \"alertmanager-main-0\" (UID: \"5aade16a-29eb-43c8-8bc5-9889f0356fbf\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 07 11:53:29.568065 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:53:29.568062    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5aade16a-29eb-43c8-8bc5-9889f0356fbf-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"5aade16a-29eb-43c8-8bc5-9889f0356fbf\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 07 11:53:29.568909 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:53:29.568882    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5aade16a-29eb-43c8-8bc5-9889f0356fbf-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"5aade16a-29eb-43c8-8bc5-9889f0356fbf\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 07 11:53:29.569014 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:53:29.568885    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5aade16a-29eb-43c8-8bc5-9889f0356fbf-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"5aade16a-29eb-43c8-8bc5-9889f0356fbf\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 07 11:53:29.571008 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:53:29.570745    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/5aade16a-29eb-43c8-8bc5-9889f0356fbf-web-config\") pod \"alertmanager-main-0\" (UID: \"5aade16a-29eb-43c8-8bc5-9889f0356fbf\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 07 11:53:29.571008 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:53:29.570768    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/5aade16a-29eb-43c8-8bc5-9889f0356fbf-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"5aade16a-29eb-43c8-8bc5-9889f0356fbf\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 07 11:53:29.571008 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:53:29.570942    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/5aade16a-29eb-43c8-8bc5-9889f0356fbf-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"5aade16a-29eb-43c8-8bc5-9889f0356fbf\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 07 11:53:29.571008 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:53:29.570990    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/5aade16a-29eb-43c8-8bc5-9889f0356fbf-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"5aade16a-29eb-43c8-8bc5-9889f0356fbf\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 07 11:53:29.571008 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:53:29.571007    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/5aade16a-29eb-43c8-8bc5-9889f0356fbf-config-out\") pod \"alertmanager-main-0\" (UID: \"5aade16a-29eb-43c8-8bc5-9889f0356fbf\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 07 11:53:29.571272 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:53:29.571035    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/5aade16a-29eb-43c8-8bc5-9889f0356fbf-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"5aade16a-29eb-43c8-8bc5-9889f0356fbf\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 07 11:53:29.571272 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:53:29.571141    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/5aade16a-29eb-43c8-8bc5-9889f0356fbf-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"5aade16a-29eb-43c8-8bc5-9889f0356fbf\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 07 11:53:29.571550 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:53:29.571531    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/5aade16a-29eb-43c8-8bc5-9889f0356fbf-config-volume\") pod \"alertmanager-main-0\" (UID: \"5aade16a-29eb-43c8-8bc5-9889f0356fbf\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 07 11:53:29.571688 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:53:29.571671    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/5aade16a-29eb-43c8-8bc5-9889f0356fbf-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"5aade16a-29eb-43c8-8bc5-9889f0356fbf\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 07 11:53:29.572623 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:53:29.572607    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/5aade16a-29eb-43c8-8bc5-9889f0356fbf-tls-assets\") pod \"alertmanager-main-0\" (UID: \"5aade16a-29eb-43c8-8bc5-9889f0356fbf\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 07 11:53:29.575036 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:53:29.575016    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9z8vg\" (UniqueName: \"kubernetes.io/projected/5aade16a-29eb-43c8-8bc5-9889f0356fbf-kube-api-access-9z8vg\") pod \"alertmanager-main-0\" (UID: \"5aade16a-29eb-43c8-8bc5-9889f0356fbf\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 07 11:53:29.696019 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:53:29.695998    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 07 11:53:29.975293 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:53:29.974674    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 07 11:53:29.978674 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:53:29.978650    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5aade16a_29eb_43c8_8bc5_9889f0356fbf.slice/crio-f3bc4694c5bef77d34a75429d4709d934f2c59d75fc4d8e6562cfcf8e620e8c2 WatchSource:0}: Error finding container f3bc4694c5bef77d34a75429d4709d934f2c59d75fc4d8e6562cfcf8e620e8c2: Status 404 returned error can't find the container with id f3bc4694c5bef77d34a75429d4709d934f2c59d75fc4d8e6562cfcf8e620e8c2
Sep 07 11:53:30.341366 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:53:30.341281    2563 generic.go:356] "Generic (PLEG): container finished" podID="5aade16a-29eb-43c8-8bc5-9889f0356fbf" containerID="f917c1f2c6e5bd3a7c0fb8b214016ecc7d280324ee2eb14175b511660eb83ca5" exitCode=0
Sep 07 11:53:30.341366 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:53:30.341346    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"5aade16a-29eb-43c8-8bc5-9889f0356fbf","Type":"ContainerDied","Data":"f917c1f2c6e5bd3a7c0fb8b214016ecc7d280324ee2eb14175b511660eb83ca5"}
Sep 07 11:53:30.341873 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:53:30.341433    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"5aade16a-29eb-43c8-8bc5-9889f0356fbf","Type":"ContainerStarted","Data":"f3bc4694c5bef77d34a75429d4709d934f2c59d75fc4d8e6562cfcf8e620e8c2"}
Sep 07 11:53:31.044496 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:53:31.044462    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="57a9b372-5c0c-48e0-b0a2-49814095da99" path="/var/lib/kubelet/pods/57a9b372-5c0c-48e0-b0a2-49814095da99/volumes"
Sep 07 11:53:31.210573 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:53:31.210538    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-786fcc8696-69dmr"]
Sep 07 11:53:31.214766 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:53:31.214743    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-786fcc8696-69dmr"
Sep 07 11:53:31.216392 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:53:31.216369    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\""
Sep 07 11:53:31.216520 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:53:31.216408    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\""
Sep 07 11:53:31.216687 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:53:31.216669    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\""
Sep 07 11:53:31.216805 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:53:31.216686    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\""
Sep 07 11:53:31.216805 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:53:31.216670    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-mdksl\""
Sep 07 11:53:31.216805 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:53:31.216787    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\""
Sep 07 11:53:31.221668 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:53:31.221600    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\""
Sep 07 11:53:31.225742 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:53:31.225722    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-786fcc8696-69dmr"]
Sep 07 11:53:31.282958 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:53:31.282925    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/0ea1f45f-67b1-4a1e-86bb-d59f207a7265-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-786fcc8696-69dmr\" (UID: \"0ea1f45f-67b1-4a1e-86bb-d59f207a7265\") " pod="openshift-monitoring/telemeter-client-786fcc8696-69dmr"
Sep 07 11:53:31.283095 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:53:31.282968    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0ea1f45f-67b1-4a1e-86bb-d59f207a7265-metrics-client-ca\") pod \"telemeter-client-786fcc8696-69dmr\" (UID: \"0ea1f45f-67b1-4a1e-86bb-d59f207a7265\") " pod="openshift-monitoring/telemeter-client-786fcc8696-69dmr"
Sep 07 11:53:31.283095 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:53:31.282995    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/0ea1f45f-67b1-4a1e-86bb-d59f207a7265-secret-telemeter-client\") pod \"telemeter-client-786fcc8696-69dmr\" (UID: \"0ea1f45f-67b1-4a1e-86bb-d59f207a7265\") " pod="openshift-monitoring/telemeter-client-786fcc8696-69dmr"
Sep 07 11:53:31.283095 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:53:31.283030    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/0ea1f45f-67b1-4a1e-86bb-d59f207a7265-federate-client-tls\") pod \"telemeter-client-786fcc8696-69dmr\" (UID: \"0ea1f45f-67b1-4a1e-86bb-d59f207a7265\") " pod="openshift-monitoring/telemeter-client-786fcc8696-69dmr"
Sep 07 11:53:31.283095 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:53:31.283065    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0ea1f45f-67b1-4a1e-86bb-d59f207a7265-serving-certs-ca-bundle\") pod \"telemeter-client-786fcc8696-69dmr\" (UID: \"0ea1f45f-67b1-4a1e-86bb-d59f207a7265\") " pod="openshift-monitoring/telemeter-client-786fcc8696-69dmr"
Sep 07 11:53:31.283240 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:53:31.283103    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0ea1f45f-67b1-4a1e-86bb-d59f207a7265-telemeter-trusted-ca-bundle\") pod \"telemeter-client-786fcc8696-69dmr\" (UID: \"0ea1f45f-67b1-4a1e-86bb-d59f207a7265\") " pod="openshift-monitoring/telemeter-client-786fcc8696-69dmr"
Sep 07 11:53:31.283240 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:53:31.283138    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hvr25\" (UniqueName: \"kubernetes.io/projected/0ea1f45f-67b1-4a1e-86bb-d59f207a7265-kube-api-access-hvr25\") pod \"telemeter-client-786fcc8696-69dmr\" (UID: \"0ea1f45f-67b1-4a1e-86bb-d59f207a7265\") " pod="openshift-monitoring/telemeter-client-786fcc8696-69dmr"
Sep 07 11:53:31.283240 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:53:31.283156    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/0ea1f45f-67b1-4a1e-86bb-d59f207a7265-telemeter-client-tls\") pod \"telemeter-client-786fcc8696-69dmr\" (UID: \"0ea1f45f-67b1-4a1e-86bb-d59f207a7265\") " pod="openshift-monitoring/telemeter-client-786fcc8696-69dmr"
Sep 07 11:53:31.349331 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:53:31.349251    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"5aade16a-29eb-43c8-8bc5-9889f0356fbf","Type":"ContainerStarted","Data":"fdbea286fbdd78299df17dc0601f59f7909acc3efcfe4321d08999c1169fb08f"}
Sep 07 11:53:31.349331 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:53:31.349315    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"5aade16a-29eb-43c8-8bc5-9889f0356fbf","Type":"ContainerStarted","Data":"9d63a0a5bda813de826019b3e8bb9f82aa7072b947bca71556890dfc59b44072"}
Sep 07 11:53:31.349759 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:53:31.349341    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"5aade16a-29eb-43c8-8bc5-9889f0356fbf","Type":"ContainerStarted","Data":"19ae5d2ebb3e936c5ff739f7424966b95df3d99bc5c016f53ef5d1fe091fe54f"}
Sep 07 11:53:31.349759 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:53:31.349371    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"5aade16a-29eb-43c8-8bc5-9889f0356fbf","Type":"ContainerStarted","Data":"64718d3ba959f2ebc5db4816fb8c08e81de98634e57e2a8cb166e3e9c5ac7a5f"}
Sep 07 11:53:31.349759 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:53:31.349396    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"5aade16a-29eb-43c8-8bc5-9889f0356fbf","Type":"ContainerStarted","Data":"770a8a0b469d101e62c972d23fcfef0e361c9cb4e81b078599497cc73a5de33f"}
Sep 07 11:53:31.349759 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:53:31.349433    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"5aade16a-29eb-43c8-8bc5-9889f0356fbf","Type":"ContainerStarted","Data":"7a54970d51b40e4660f43adddb18c520680db1069ff1441f5450ccd3c1a35035"}
Sep 07 11:53:31.371873 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:53:31.371822    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.356439099 podStartE2EDuration="2.371802532s" totalImagesPullingTime="0s" totalInitContainerRuntime="15.363433ms" isStatefulPod=false podCreationTimestamp="2026-09-07 11:53:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 11:53:31.371287705 +0000 UTC m=+382.890019519" watchObservedRunningTime="2026-09-07 11:53:31.371802532 +0000 UTC m=+382.890534346"
Sep 07 11:53:31.384556 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:53:31.384529    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/0ea1f45f-67b1-4a1e-86bb-d59f207a7265-telemeter-client-tls\") pod \"telemeter-client-786fcc8696-69dmr\" (UID: \"0ea1f45f-67b1-4a1e-86bb-d59f207a7265\") " pod="openshift-monitoring/telemeter-client-786fcc8696-69dmr"
Sep 07 11:53:31.384936 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:53:31.384889    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/0ea1f45f-67b1-4a1e-86bb-d59f207a7265-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-786fcc8696-69dmr\" (UID: \"0ea1f45f-67b1-4a1e-86bb-d59f207a7265\") " pod="openshift-monitoring/telemeter-client-786fcc8696-69dmr"
Sep 07 11:53:31.385053 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:53:31.384949    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0ea1f45f-67b1-4a1e-86bb-d59f207a7265-metrics-client-ca\") pod \"telemeter-client-786fcc8696-69dmr\" (UID: \"0ea1f45f-67b1-4a1e-86bb-d59f207a7265\") " pod="openshift-monitoring/telemeter-client-786fcc8696-69dmr"
Sep 07 11:53:31.385053 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:53:31.384990    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/0ea1f45f-67b1-4a1e-86bb-d59f207a7265-secret-telemeter-client\") pod \"telemeter-client-786fcc8696-69dmr\" (UID: \"0ea1f45f-67b1-4a1e-86bb-d59f207a7265\") " pod="openshift-monitoring/telemeter-client-786fcc8696-69dmr"
Sep 07 11:53:31.385053 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:53:31.385006    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/0ea1f45f-67b1-4a1e-86bb-d59f207a7265-federate-client-tls\") pod \"telemeter-client-786fcc8696-69dmr\" (UID: \"0ea1f45f-67b1-4a1e-86bb-d59f207a7265\") " pod="openshift-monitoring/telemeter-client-786fcc8696-69dmr"
Sep 07 11:53:31.385053 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:53:31.385031    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0ea1f45f-67b1-4a1e-86bb-d59f207a7265-serving-certs-ca-bundle\") pod \"telemeter-client-786fcc8696-69dmr\" (UID: \"0ea1f45f-67b1-4a1e-86bb-d59f207a7265\") " pod="openshift-monitoring/telemeter-client-786fcc8696-69dmr"
Sep 07 11:53:31.385254 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:53:31.385089    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0ea1f45f-67b1-4a1e-86bb-d59f207a7265-telemeter-trusted-ca-bundle\") pod \"telemeter-client-786fcc8696-69dmr\" (UID: \"0ea1f45f-67b1-4a1e-86bb-d59f207a7265\") " pod="openshift-monitoring/telemeter-client-786fcc8696-69dmr"
Sep 07 11:53:31.385254 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:53:31.385218    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hvr25\" (UniqueName: \"kubernetes.io/projected/0ea1f45f-67b1-4a1e-86bb-d59f207a7265-kube-api-access-hvr25\") pod \"telemeter-client-786fcc8696-69dmr\" (UID: \"0ea1f45f-67b1-4a1e-86bb-d59f207a7265\") " pod="openshift-monitoring/telemeter-client-786fcc8696-69dmr"
Sep 07 11:53:31.386103 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:53:31.385842    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0ea1f45f-67b1-4a1e-86bb-d59f207a7265-metrics-client-ca\") pod \"telemeter-client-786fcc8696-69dmr\" (UID: \"0ea1f45f-67b1-4a1e-86bb-d59f207a7265\") " pod="openshift-monitoring/telemeter-client-786fcc8696-69dmr"
Sep 07 11:53:31.386256 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:53:31.386232    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0ea1f45f-67b1-4a1e-86bb-d59f207a7265-serving-certs-ca-bundle\") pod \"telemeter-client-786fcc8696-69dmr\" (UID: \"0ea1f45f-67b1-4a1e-86bb-d59f207a7265\") " pod="openshift-monitoring/telemeter-client-786fcc8696-69dmr"
Sep 07 11:53:31.386677 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:53:31.386643    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0ea1f45f-67b1-4a1e-86bb-d59f207a7265-telemeter-trusted-ca-bundle\") pod \"telemeter-client-786fcc8696-69dmr\" (UID: \"0ea1f45f-67b1-4a1e-86bb-d59f207a7265\") " pod="openshift-monitoring/telemeter-client-786fcc8696-69dmr"
Sep 07 11:53:31.387558 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:53:31.387524    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/0ea1f45f-67b1-4a1e-86bb-d59f207a7265-telemeter-client-tls\") pod \"telemeter-client-786fcc8696-69dmr\" (UID: \"0ea1f45f-67b1-4a1e-86bb-d59f207a7265\") " pod="openshift-monitoring/telemeter-client-786fcc8696-69dmr"
Sep 07 11:53:31.387878 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:53:31.387853    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/0ea1f45f-67b1-4a1e-86bb-d59f207a7265-secret-telemeter-client\") pod \"telemeter-client-786fcc8696-69dmr\" (UID: \"0ea1f45f-67b1-4a1e-86bb-d59f207a7265\") " pod="openshift-monitoring/telemeter-client-786fcc8696-69dmr"
Sep 07 11:53:31.388392 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:53:31.388369    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/0ea1f45f-67b1-4a1e-86bb-d59f207a7265-federate-client-tls\") pod \"telemeter-client-786fcc8696-69dmr\" (UID: \"0ea1f45f-67b1-4a1e-86bb-d59f207a7265\") " pod="openshift-monitoring/telemeter-client-786fcc8696-69dmr"
Sep 07 11:53:31.388734 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:53:31.388713    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/0ea1f45f-67b1-4a1e-86bb-d59f207a7265-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-786fcc8696-69dmr\" (UID: \"0ea1f45f-67b1-4a1e-86bb-d59f207a7265\") " pod="openshift-monitoring/telemeter-client-786fcc8696-69dmr"
Sep 07 11:53:31.391593 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:53:31.391566    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hvr25\" (UniqueName: \"kubernetes.io/projected/0ea1f45f-67b1-4a1e-86bb-d59f207a7265-kube-api-access-hvr25\") pod \"telemeter-client-786fcc8696-69dmr\" (UID: \"0ea1f45f-67b1-4a1e-86bb-d59f207a7265\") " pod="openshift-monitoring/telemeter-client-786fcc8696-69dmr"
Sep 07 11:53:31.523971 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:53:31.523942    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-786fcc8696-69dmr"
Sep 07 11:53:31.796858 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:53:31.796712    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-786fcc8696-69dmr"]
Sep 07 11:53:31.799173 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:53:31.799149    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0ea1f45f_67b1_4a1e_86bb_d59f207a7265.slice/crio-462a170429d28c3f91370e69e9a821a51d623849084b01250a4da6e3d4eaf4de WatchSource:0}: Error finding container 462a170429d28c3f91370e69e9a821a51d623849084b01250a4da6e3d4eaf4de: Status 404 returned error can't find the container with id 462a170429d28c3f91370e69e9a821a51d623849084b01250a4da6e3d4eaf4de
Sep 07 11:53:32.354557 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:53:32.354518    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-786fcc8696-69dmr" event={"ID":"0ea1f45f-67b1-4a1e-86bb-d59f207a7265","Type":"ContainerStarted","Data":"462a170429d28c3f91370e69e9a821a51d623849084b01250a4da6e3d4eaf4de"}
Sep 07 11:53:34.371202 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:53:34.371169    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-786fcc8696-69dmr" event={"ID":"0ea1f45f-67b1-4a1e-86bb-d59f207a7265","Type":"ContainerStarted","Data":"fb44fc9ec98f1f97c8299f55d0250a8e5611e9c25358dfe56bcbf6f73d1b2f66"}
Sep 07 11:53:34.371617 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:53:34.371250    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-786fcc8696-69dmr" event={"ID":"0ea1f45f-67b1-4a1e-86bb-d59f207a7265","Type":"ContainerStarted","Data":"3b3f9b47a893d92786af5a09acb135209a99aa302b748df2c6d3b9885c0f2433"}
Sep 07 11:53:34.371617 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:53:34.371272    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-786fcc8696-69dmr" event={"ID":"0ea1f45f-67b1-4a1e-86bb-d59f207a7265","Type":"ContainerStarted","Data":"1ce9bf165ead3d06314d9453d97eb87e20fd682421a1eb349c47363a16e56719"}
Sep 07 11:53:34.393075 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:53:34.393029    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-786fcc8696-69dmr" podStartSLOduration=1.652191612 podStartE2EDuration="3.393009312s" totalImagesPullingTime="1.7408177s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 11:53:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 11:53:34.390808633 +0000 UTC m=+385.909540461" watchObservedRunningTime="2026-09-07 11:53:34.393009312 +0000 UTC m=+385.911741241"
Sep 07 11:53:34.528874 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:53:34.528833    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 11:53:34.580947 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:53:34.580917    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 11:53:34.730940 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:53:34.730905    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 11:53:34.783683 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:53:34.783653    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 11:53:35.271842 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:53:35.271814    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 11:53:35.372219 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:53:35.372183    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 11:53:42.435632 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:53:42.435597    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 11:53:42.490944 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:53:42.490906    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 11:53:45.816969 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:53:45.816933    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 11:53:45.868370 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:53:45.868337    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 11:53:45.923112 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:53:45.923079    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 11:54:09.651088 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:54:09.651049    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 11:54:09.703769 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:54:09.703739    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 11:54:41.412192 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:54:41.412149    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 11:54:41.470070 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:54:41.470036    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 11:54:53.303980 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:54:53.303941    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 11:54:53.354235 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:54:53.354208    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 11:54:55.234802 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:54:55.234754    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 11:54:55.287676 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:54:55.287645    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 11:54:55.378049 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:54:55.378007    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 11:54:55.428393 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:54:55.428350    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 11:54:55.980439 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:54:55.980389    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 11:55:06.114449 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:55:06.114396    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 11:55:06.161223 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:55:06.161183    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 11:55:06.214505 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:55:06.214473    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 11:55:06.263101 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:55:06.263077    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 11:55:06.893728 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:55:06.893688    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 11:55:06.995457 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:55:06.995402    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 11:55:57.445762 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:55:57.445719    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 11:55:57.539524 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:55:57.539488    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 11:56:17.284984 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:56:17.284944    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 11:56:19.015678 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:56:19.015649    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 11:56:19.850412 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:56:19.850375    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 11:56:22.624388 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:56:22.624354    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 11:56:22.724987 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:56:22.724953    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 11:56:23.727516 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:56:23.727474    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 11:56:23.836121 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:56:23.836085    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 11:56:26.939480 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:56:26.939445    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 11:56:26.994197 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:56:26.994164    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 11:56:27.282041 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:56:27.282009    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-675t5"]
Sep 07 11:56:27.285291 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:56:27.285275    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-675t5"
Sep 07 11:56:27.287364 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:56:27.287340    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Sep 07 11:56:27.287574 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:56:27.287560    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-z9cjf\""
Sep 07 11:56:27.287574 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:56:27.287568    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Sep 07 11:56:27.292752 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:56:27.292731    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-675t5"]
Sep 07 11:56:27.414759 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:56:27.414726    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bc9w9\" (UniqueName: \"kubernetes.io/projected/2ef7d547-c667-400c-bbf8-84f445e1643d-kube-api-access-bc9w9\") pod \"cert-manager-cainjector-559cdd8c87-675t5\" (UID: \"2ef7d547-c667-400c-bbf8-84f445e1643d\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-675t5"
Sep 07 11:56:27.414885 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:56:27.414766    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/2ef7d547-c667-400c-bbf8-84f445e1643d-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-675t5\" (UID: \"2ef7d547-c667-400c-bbf8-84f445e1643d\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-675t5"
Sep 07 11:56:27.515587 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:56:27.515557    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bc9w9\" (UniqueName: \"kubernetes.io/projected/2ef7d547-c667-400c-bbf8-84f445e1643d-kube-api-access-bc9w9\") pod \"cert-manager-cainjector-559cdd8c87-675t5\" (UID: \"2ef7d547-c667-400c-bbf8-84f445e1643d\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-675t5"
Sep 07 11:56:27.515693 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:56:27.515593    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/2ef7d547-c667-400c-bbf8-84f445e1643d-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-675t5\" (UID: \"2ef7d547-c667-400c-bbf8-84f445e1643d\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-675t5"
Sep 07 11:56:27.523024 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:56:27.523001    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/2ef7d547-c667-400c-bbf8-84f445e1643d-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-675t5\" (UID: \"2ef7d547-c667-400c-bbf8-84f445e1643d\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-675t5"
Sep 07 11:56:27.523118 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:56:27.523068    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bc9w9\" (UniqueName: \"kubernetes.io/projected/2ef7d547-c667-400c-bbf8-84f445e1643d-kube-api-access-bc9w9\") pod \"cert-manager-cainjector-559cdd8c87-675t5\" (UID: \"2ef7d547-c667-400c-bbf8-84f445e1643d\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-675t5"
Sep 07 11:56:27.594542 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:56:27.594488    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-675t5"
Sep 07 11:56:27.861640 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:56:27.861618    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-675t5"]
Sep 07 11:56:27.864129 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:56:27.864105    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2ef7d547_c667_400c_bbf8_84f445e1643d.slice/crio-43da816c4006f91c1ad9da67f158d45127a7311742ed9975451e87e0b561834d WatchSource:0}: Error finding container 43da816c4006f91c1ad9da67f158d45127a7311742ed9975451e87e0b561834d: Status 404 returned error can't find the container with id 43da816c4006f91c1ad9da67f158d45127a7311742ed9975451e87e0b561834d
Sep 07 11:56:28.037509 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:56:28.037474    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-675t5" event={"ID":"2ef7d547-c667-400c-bbf8-84f445e1643d","Type":"ContainerStarted","Data":"43da816c4006f91c1ad9da67f158d45127a7311742ed9975451e87e0b561834d"}
Sep 07 11:56:29.427223 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:56:29.427177    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 11:56:29.527847 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:56:29.527812    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 11:56:31.052486 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:56:31.052394    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-675t5" event={"ID":"2ef7d547-c667-400c-bbf8-84f445e1643d","Type":"ContainerStarted","Data":"e0a5ee8e3c30ac7f799d6f65554e02207feca6e3d8beb80c2c8f925390db6ad6"}
Sep 07 11:56:31.066627 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:56:31.066579    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-675t5" podStartSLOduration=1.173531618 podStartE2EDuration="4.066564596s" totalImagesPullingTime="2.893032978s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 11:56:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 11:56:31.064927518 +0000 UTC m=+562.583659330" watchObservedRunningTime="2026-09-07 11:56:31.066564596 +0000 UTC m=+562.585296409"
Sep 07 11:56:31.533122 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:56:31.533085    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 11:56:33.247342 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:56:33.247312    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 11:56:34.271804 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:56:34.271770    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 11:56:39.841713 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:56:39.841673    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 11:56:39.940363 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:56:39.940331    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 11:56:42.057234 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:56:42.057199    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-tf2ht"]
Sep 07 11:56:42.060551 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:56:42.060528    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-tf2ht"
Sep 07 11:56:42.062310 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:56:42.062289    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Sep 07 11:56:42.062396 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:56:42.062325    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-2vgvm\""
Sep 07 11:56:42.062542 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:56:42.062527    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Sep 07 11:56:42.069495 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:56:42.069476    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-tf2ht"]
Sep 07 11:56:42.132007 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:56:42.131979    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/117fafc0-57de-4666-845f-f12646393515-tmp\") pod \"openshift-lws-operator-fd8ccff4c-tf2ht\" (UID: \"117fafc0-57de-4666-845f-f12646393515\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-tf2ht"
Sep 07 11:56:42.132159 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:56:42.132027    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hgnwl\" (UniqueName: \"kubernetes.io/projected/117fafc0-57de-4666-845f-f12646393515-kube-api-access-hgnwl\") pod \"openshift-lws-operator-fd8ccff4c-tf2ht\" (UID: \"117fafc0-57de-4666-845f-f12646393515\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-tf2ht"
Sep 07 11:56:42.233247 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:56:42.233205    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/117fafc0-57de-4666-845f-f12646393515-tmp\") pod \"openshift-lws-operator-fd8ccff4c-tf2ht\" (UID: \"117fafc0-57de-4666-845f-f12646393515\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-tf2ht"
Sep 07 11:56:42.233471 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:56:42.233303    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hgnwl\" (UniqueName: \"kubernetes.io/projected/117fafc0-57de-4666-845f-f12646393515-kube-api-access-hgnwl\") pod \"openshift-lws-operator-fd8ccff4c-tf2ht\" (UID: \"117fafc0-57de-4666-845f-f12646393515\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-tf2ht"
Sep 07 11:56:42.233725 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:56:42.233705    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/117fafc0-57de-4666-845f-f12646393515-tmp\") pod \"openshift-lws-operator-fd8ccff4c-tf2ht\" (UID: \"117fafc0-57de-4666-845f-f12646393515\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-tf2ht"
Sep 07 11:56:42.240811 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:56:42.240784    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hgnwl\" (UniqueName: \"kubernetes.io/projected/117fafc0-57de-4666-845f-f12646393515-kube-api-access-hgnwl\") pod \"openshift-lws-operator-fd8ccff4c-tf2ht\" (UID: \"117fafc0-57de-4666-845f-f12646393515\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-tf2ht"
Sep 07 11:56:42.369776 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:56:42.369706    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-tf2ht"
Sep 07 11:56:42.640296 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:56:42.640275    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-tf2ht"]
Sep 07 11:56:42.643016 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:56:42.642991    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod117fafc0_57de_4666_845f_f12646393515.slice/crio-889921f9539e10e49a5ba745863951a1d15a685e9d6858762a59af054f5f70c8 WatchSource:0}: Error finding container 889921f9539e10e49a5ba745863951a1d15a685e9d6858762a59af054f5f70c8: Status 404 returned error can't find the container with id 889921f9539e10e49a5ba745863951a1d15a685e9d6858762a59af054f5f70c8
Sep 07 11:56:43.097176 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:56:43.097142    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-tf2ht" event={"ID":"117fafc0-57de-4666-845f-f12646393515","Type":"ContainerStarted","Data":"889921f9539e10e49a5ba745863951a1d15a685e9d6858762a59af054f5f70c8"}
Sep 07 11:56:45.106651 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:56:45.106612    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-tf2ht" event={"ID":"117fafc0-57de-4666-845f-f12646393515","Type":"ContainerStarted","Data":"7bd437fa776a54b21a4c96b49b909dee5c703c82fd081af90016e09e21b4eeff"}
Sep 07 11:56:45.121391 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:56:45.121351    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-tf2ht" podStartSLOduration=0.791355277 podStartE2EDuration="3.121334868s" totalImagesPullingTime="2.329979591s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 11:56:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 11:56:45.119908216 +0000 UTC m=+576.638640025" watchObservedRunningTime="2026-09-07 11:56:45.121334868 +0000 UTC m=+576.640066683"
Sep 07 11:56:45.749702 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:56:45.749665    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 11:56:45.846437 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:56:45.846375    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 11:56:47.978088 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:56:47.978056    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 11:56:58.027635 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:56:58.027596    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 11:57:00.146181 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:57:00.146148    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 11:57:00.246559 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:57:00.246528    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 11:57:00.769824 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:57:00.769780    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 11:57:00.965766 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:57:00.965733    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 11:57:01.073992 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:57:01.073916    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 11:57:01.118173 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:57:01.118142    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-648db6df7b-c82m6"]
Sep 07 11:57:01.125806 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:57:01.125780    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-648db6df7b-c82m6"
Sep 07 11:57:01.127625 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:57:01.127602    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Sep 07 11:57:01.127989 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:57:01.127968    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-qnxxt\""
Sep 07 11:57:01.128072 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:57:01.127990    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Sep 07 11:57:01.128072 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:57:01.127997    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Sep 07 11:57:01.128153 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:57:01.128141    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Sep 07 11:57:01.147263 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:57:01.147236    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-648db6df7b-c82m6"]
Sep 07 11:57:01.171820 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:57:01.171789    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9fe85f81-1199-4c0b-b58e-036142982b1a-tmp\") pod \"opendatahub-operator-controller-manager-648db6df7b-c82m6\" (UID: \"9fe85f81-1199-4c0b-b58e-036142982b1a\") " pod="opendatahub/opendatahub-operator-controller-manager-648db6df7b-c82m6"
Sep 07 11:57:01.171953 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:57:01.171826    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/9fe85f81-1199-4c0b-b58e-036142982b1a-manifests\") pod \"opendatahub-operator-controller-manager-648db6df7b-c82m6\" (UID: \"9fe85f81-1199-4c0b-b58e-036142982b1a\") " pod="opendatahub/opendatahub-operator-controller-manager-648db6df7b-c82m6"
Sep 07 11:57:01.171953 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:57:01.171875    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7cbbn\" (UniqueName: \"kubernetes.io/projected/9fe85f81-1199-4c0b-b58e-036142982b1a-kube-api-access-7cbbn\") pod \"opendatahub-operator-controller-manager-648db6df7b-c82m6\" (UID: \"9fe85f81-1199-4c0b-b58e-036142982b1a\") " pod="opendatahub/opendatahub-operator-controller-manager-648db6df7b-c82m6"
Sep 07 11:57:01.171953 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:57:01.171916    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/9fe85f81-1199-4c0b-b58e-036142982b1a-apiservice-cert\") pod \"opendatahub-operator-controller-manager-648db6df7b-c82m6\" (UID: \"9fe85f81-1199-4c0b-b58e-036142982b1a\") " pod="opendatahub/opendatahub-operator-controller-manager-648db6df7b-c82m6"
Sep 07 11:57:01.171953 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:57:01.171947    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9fe85f81-1199-4c0b-b58e-036142982b1a-webhook-cert\") pod \"opendatahub-operator-controller-manager-648db6df7b-c82m6\" (UID: \"9fe85f81-1199-4c0b-b58e-036142982b1a\") " pod="opendatahub/opendatahub-operator-controller-manager-648db6df7b-c82m6"
Sep 07 11:57:01.265579 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:57:01.265544    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 11:57:01.272658 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:57:01.272631    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7cbbn\" (UniqueName: \"kubernetes.io/projected/9fe85f81-1199-4c0b-b58e-036142982b1a-kube-api-access-7cbbn\") pod \"opendatahub-operator-controller-manager-648db6df7b-c82m6\" (UID: \"9fe85f81-1199-4c0b-b58e-036142982b1a\") " pod="opendatahub/opendatahub-operator-controller-manager-648db6df7b-c82m6"
Sep 07 11:57:01.272778 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:57:01.272681    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/9fe85f81-1199-4c0b-b58e-036142982b1a-apiservice-cert\") pod \"opendatahub-operator-controller-manager-648db6df7b-c82m6\" (UID: \"9fe85f81-1199-4c0b-b58e-036142982b1a\") " pod="opendatahub/opendatahub-operator-controller-manager-648db6df7b-c82m6"
Sep 07 11:57:01.272778 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:57:01.272706    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9fe85f81-1199-4c0b-b58e-036142982b1a-webhook-cert\") pod \"opendatahub-operator-controller-manager-648db6df7b-c82m6\" (UID: \"9fe85f81-1199-4c0b-b58e-036142982b1a\") " pod="opendatahub/opendatahub-operator-controller-manager-648db6df7b-c82m6"
Sep 07 11:57:01.272778 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:57:01.272734    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9fe85f81-1199-4c0b-b58e-036142982b1a-tmp\") pod \"opendatahub-operator-controller-manager-648db6df7b-c82m6\" (UID: \"9fe85f81-1199-4c0b-b58e-036142982b1a\") " pod="opendatahub/opendatahub-operator-controller-manager-648db6df7b-c82m6"
Sep 07 11:57:01.272778 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:57:01.272753    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/9fe85f81-1199-4c0b-b58e-036142982b1a-manifests\") pod \"opendatahub-operator-controller-manager-648db6df7b-c82m6\" (UID: \"9fe85f81-1199-4c0b-b58e-036142982b1a\") " pod="opendatahub/opendatahub-operator-controller-manager-648db6df7b-c82m6"
Sep 07 11:57:01.273181 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:57:01.273154    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/9fe85f81-1199-4c0b-b58e-036142982b1a-manifests\") pod \"opendatahub-operator-controller-manager-648db6df7b-c82m6\" (UID: \"9fe85f81-1199-4c0b-b58e-036142982b1a\") " pod="opendatahub/opendatahub-operator-controller-manager-648db6df7b-c82m6"
Sep 07 11:57:01.275007 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:57:01.274981    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9fe85f81-1199-4c0b-b58e-036142982b1a-tmp\") pod \"opendatahub-operator-controller-manager-648db6df7b-c82m6\" (UID: \"9fe85f81-1199-4c0b-b58e-036142982b1a\") " pod="opendatahub/opendatahub-operator-controller-manager-648db6df7b-c82m6"
Sep 07 11:57:01.275219 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:57:01.275200    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9fe85f81-1199-4c0b-b58e-036142982b1a-webhook-cert\") pod \"opendatahub-operator-controller-manager-648db6df7b-c82m6\" (UID: \"9fe85f81-1199-4c0b-b58e-036142982b1a\") " pod="opendatahub/opendatahub-operator-controller-manager-648db6df7b-c82m6"
Sep 07 11:57:01.275288 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:57:01.275238    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/9fe85f81-1199-4c0b-b58e-036142982b1a-apiservice-cert\") pod \"opendatahub-operator-controller-manager-648db6df7b-c82m6\" (UID: \"9fe85f81-1199-4c0b-b58e-036142982b1a\") " pod="opendatahub/opendatahub-operator-controller-manager-648db6df7b-c82m6"
Sep 07 11:57:01.281008 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:57:01.280985    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7cbbn\" (UniqueName: \"kubernetes.io/projected/9fe85f81-1199-4c0b-b58e-036142982b1a-kube-api-access-7cbbn\") pod \"opendatahub-operator-controller-manager-648db6df7b-c82m6\" (UID: \"9fe85f81-1199-4c0b-b58e-036142982b1a\") " pod="opendatahub/opendatahub-operator-controller-manager-648db6df7b-c82m6"
Sep 07 11:57:01.365023 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:57:01.364950    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 11:57:01.436003 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:57:01.435973    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-648db6df7b-c82m6"
Sep 07 11:57:01.466825 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:57:01.466791    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 11:57:01.563548 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:57:01.563516    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 11:57:01.663323 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:57:01.663229    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 11:57:01.715612 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:57:01.715587    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-648db6df7b-c82m6"]
Sep 07 11:57:01.717791 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:57:01.717761    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9fe85f81_1199_4c0b_b58e_036142982b1a.slice/crio-b462afd13a841b6eae1aeb875788d377da5c69abf4ce0771fa3c96171c21bb54 WatchSource:0}: Error finding container b462afd13a841b6eae1aeb875788d377da5c69abf4ce0771fa3c96171c21bb54: Status 404 returned error can't find the container with id b462afd13a841b6eae1aeb875788d377da5c69abf4ce0771fa3c96171c21bb54
Sep 07 11:57:01.763353 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:57:01.763322    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 11:57:02.177685 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:57:02.177652    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-648db6df7b-c82m6" event={"ID":"9fe85f81-1199-4c0b-b58e-036142982b1a","Type":"ContainerStarted","Data":"b462afd13a841b6eae1aeb875788d377da5c69abf4ce0771fa3c96171c21bb54"}
Sep 07 11:57:05.196299 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:57:05.196267    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-648db6df7b-c82m6" event={"ID":"9fe85f81-1199-4c0b-b58e-036142982b1a","Type":"ContainerStarted","Data":"45e185eee6b860e3bbb93cad50e110a633743b8f77ab0d4f538c01ddec5509a4"}
Sep 07 11:57:06.201893 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:57:06.201859    2563 generic.go:356] "Generic (PLEG): container finished" podID="9fe85f81-1199-4c0b-b58e-036142982b1a" containerID="45e185eee6b860e3bbb93cad50e110a633743b8f77ab0d4f538c01ddec5509a4" exitCode=0
Sep 07 11:57:06.202358 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:57:06.201924    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-648db6df7b-c82m6" event={"ID":"9fe85f81-1199-4c0b-b58e-036142982b1a","Type":"ContainerDied","Data":"45e185eee6b860e3bbb93cad50e110a633743b8f77ab0d4f538c01ddec5509a4"}
Sep 07 11:57:07.208835 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:57:07.208796    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-648db6df7b-c82m6" event={"ID":"9fe85f81-1199-4c0b-b58e-036142982b1a","Type":"ContainerStarted","Data":"3107022da8a1f75ba138e614e4134028cef579bed0f5f0202036fb135cda36cf"}
Sep 07 11:57:07.209286 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:57:07.208910    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-648db6df7b-c82m6"
Sep 07 11:57:07.228679 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:57:07.228633    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-648db6df7b-c82m6" podStartSLOduration=1.821897117 podStartE2EDuration="6.228619779s" totalImagesPullingTime="3.322805198s" totalInitContainerRuntime="1.083917464s" isStatefulPod=false podCreationTimestamp="2026-09-07 11:57:01 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 11:57:07.226803551 +0000 UTC m=+598.745535364" watchObservedRunningTime="2026-09-07 11:57:07.228619779 +0000 UTC m=+598.747351591"
Sep 07 11:57:08.259146 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:57:08.259110    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 11:57:08.360014 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:57:08.359979    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 11:57:09.101585 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:57:09.101560    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-4nfbc_b34ca85d-ac51-4724-a508-1026106b3f3a/console-operator/2.log"
Sep 07 11:57:09.103805 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:57:09.101804    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-4nfbc_b34ca85d-ac51-4724-a508-1026106b3f3a/console-operator/2.log"
Sep 07 11:57:11.817079 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:57:11.817033    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 11:57:11.914197 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:57:11.914161    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 11:57:12.245937 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:57:12.245887    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 11:57:12.346880 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:57:12.346843    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 11:57:12.677444 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:57:12.677347    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 11:57:12.781991 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:57:12.781964    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 11:57:18.218017 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:57:18.217988    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-648db6df7b-c82m6"
Sep 07 11:57:20.123690 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:57:20.123653    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 11:57:20.240950 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:57:20.240909    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 11:57:20.447154 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:57:20.447121    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 11:57:20.450394 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:57:20.450371    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-5b6777d477-c7pps"]
Sep 07 11:57:20.458661 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:57:20.458638    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-5b6777d477-c7pps"
Sep 07 11:57:20.460471 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:57:20.460398    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Sep 07 11:57:20.460471 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:57:20.460443    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Sep 07 11:57:20.460636 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:57:20.460446    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Sep 07 11:57:20.460834 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:57:20.460818    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Sep 07 11:57:20.461080 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:57:20.461064    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-56w79\""
Sep 07 11:57:20.466986 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:57:20.466690    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-5b6777d477-c7pps"]
Sep 07 11:57:20.533549 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:57:20.533523    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4a58840b-efff-43a4-ac2f-906a5f1b4bdc-tls-certs\") pod \"kube-auth-proxy-5b6777d477-c7pps\" (UID: \"4a58840b-efff-43a4-ac2f-906a5f1b4bdc\") " pod="openshift-ingress/kube-auth-proxy-5b6777d477-c7pps"
Sep 07 11:57:20.533688 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:57:20.533561    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4lz4b\" (UniqueName: \"kubernetes.io/projected/4a58840b-efff-43a4-ac2f-906a5f1b4bdc-kube-api-access-4lz4b\") pod \"kube-auth-proxy-5b6777d477-c7pps\" (UID: \"4a58840b-efff-43a4-ac2f-906a5f1b4bdc\") " pod="openshift-ingress/kube-auth-proxy-5b6777d477-c7pps"
Sep 07 11:57:20.533688 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:57:20.533603    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4a58840b-efff-43a4-ac2f-906a5f1b4bdc-tmp\") pod \"kube-auth-proxy-5b6777d477-c7pps\" (UID: \"4a58840b-efff-43a4-ac2f-906a5f1b4bdc\") " pod="openshift-ingress/kube-auth-proxy-5b6777d477-c7pps"
Sep 07 11:57:20.635146 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:57:20.635101    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4lz4b\" (UniqueName: \"kubernetes.io/projected/4a58840b-efff-43a4-ac2f-906a5f1b4bdc-kube-api-access-4lz4b\") pod \"kube-auth-proxy-5b6777d477-c7pps\" (UID: \"4a58840b-efff-43a4-ac2f-906a5f1b4bdc\") " pod="openshift-ingress/kube-auth-proxy-5b6777d477-c7pps"
Sep 07 11:57:20.635335 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:57:20.635218    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4a58840b-efff-43a4-ac2f-906a5f1b4bdc-tmp\") pod \"kube-auth-proxy-5b6777d477-c7pps\" (UID: \"4a58840b-efff-43a4-ac2f-906a5f1b4bdc\") " pod="openshift-ingress/kube-auth-proxy-5b6777d477-c7pps"
Sep 07 11:57:20.635335 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:57:20.635329    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4a58840b-efff-43a4-ac2f-906a5f1b4bdc-tls-certs\") pod \"kube-auth-proxy-5b6777d477-c7pps\" (UID: \"4a58840b-efff-43a4-ac2f-906a5f1b4bdc\") " pod="openshift-ingress/kube-auth-proxy-5b6777d477-c7pps"
Sep 07 11:57:20.638648 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:57:20.637870    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4a58840b-efff-43a4-ac2f-906a5f1b4bdc-tmp\") pod \"kube-auth-proxy-5b6777d477-c7pps\" (UID: \"4a58840b-efff-43a4-ac2f-906a5f1b4bdc\") " pod="openshift-ingress/kube-auth-proxy-5b6777d477-c7pps"
Sep 07 11:57:20.638648 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:57:20.638271    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4a58840b-efff-43a4-ac2f-906a5f1b4bdc-tls-certs\") pod \"kube-auth-proxy-5b6777d477-c7pps\" (UID: \"4a58840b-efff-43a4-ac2f-906a5f1b4bdc\") " pod="openshift-ingress/kube-auth-proxy-5b6777d477-c7pps"
Sep 07 11:57:20.642969 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:57:20.642945    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4lz4b\" (UniqueName: \"kubernetes.io/projected/4a58840b-efff-43a4-ac2f-906a5f1b4bdc-kube-api-access-4lz4b\") pod \"kube-auth-proxy-5b6777d477-c7pps\" (UID: \"4a58840b-efff-43a4-ac2f-906a5f1b4bdc\") " pod="openshift-ingress/kube-auth-proxy-5b6777d477-c7pps"
Sep 07 11:57:20.661216 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:57:20.661195    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 11:57:20.770053 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:57:20.769977    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-5b6777d477-c7pps"
Sep 07 11:57:20.808196 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:57:20.808156    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-59499dc5ff-46274"]
Sep 07 11:57:20.812126 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:57:20.812105    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-59499dc5ff-46274"
Sep 07 11:57:20.813879 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:57:20.813857    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Sep 07 11:57:20.813999 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:57:20.813856    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Sep 07 11:57:20.813999 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:57:20.813861    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Sep 07 11:57:20.813999 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:57:20.813860    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-tslk4\""
Sep 07 11:57:20.817886 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:57:20.817863    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-59499dc5ff-46274"]
Sep 07 11:57:20.939347 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:57:20.939315    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/ee12ae27-62c1-43cb-98ec-319b3243ff28-metrics-cert\") pod \"lws-controller-manager-59499dc5ff-46274\" (UID: \"ee12ae27-62c1-43cb-98ec-319b3243ff28\") " pod="openshift-lws-operator/lws-controller-manager-59499dc5ff-46274"
Sep 07 11:57:20.939551 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:57:20.939494    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ee12ae27-62c1-43cb-98ec-319b3243ff28-cert\") pod \"lws-controller-manager-59499dc5ff-46274\" (UID: \"ee12ae27-62c1-43cb-98ec-319b3243ff28\") " pod="openshift-lws-operator/lws-controller-manager-59499dc5ff-46274"
Sep 07 11:57:20.939551 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:57:20.939525    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/ee12ae27-62c1-43cb-98ec-319b3243ff28-manager-config\") pod \"lws-controller-manager-59499dc5ff-46274\" (UID: \"ee12ae27-62c1-43cb-98ec-319b3243ff28\") " pod="openshift-lws-operator/lws-controller-manager-59499dc5ff-46274"
Sep 07 11:57:20.939664 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:57:20.939550    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-84x7z\" (UniqueName: \"kubernetes.io/projected/ee12ae27-62c1-43cb-98ec-319b3243ff28-kube-api-access-84x7z\") pod \"lws-controller-manager-59499dc5ff-46274\" (UID: \"ee12ae27-62c1-43cb-98ec-319b3243ff28\") " pod="openshift-lws-operator/lws-controller-manager-59499dc5ff-46274"
Sep 07 11:57:21.040263 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:57:21.040183    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/ee12ae27-62c1-43cb-98ec-319b3243ff28-metrics-cert\") pod \"lws-controller-manager-59499dc5ff-46274\" (UID: \"ee12ae27-62c1-43cb-98ec-319b3243ff28\") " pod="openshift-lws-operator/lws-controller-manager-59499dc5ff-46274"
Sep 07 11:57:21.040429 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:57:21.040267    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ee12ae27-62c1-43cb-98ec-319b3243ff28-cert\") pod \"lws-controller-manager-59499dc5ff-46274\" (UID: \"ee12ae27-62c1-43cb-98ec-319b3243ff28\") " pod="openshift-lws-operator/lws-controller-manager-59499dc5ff-46274"
Sep 07 11:57:21.040429 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:57:21.040294    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/ee12ae27-62c1-43cb-98ec-319b3243ff28-manager-config\") pod \"lws-controller-manager-59499dc5ff-46274\" (UID: \"ee12ae27-62c1-43cb-98ec-319b3243ff28\") " pod="openshift-lws-operator/lws-controller-manager-59499dc5ff-46274"
Sep 07 11:57:21.040429 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:57:21.040317    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-84x7z\" (UniqueName: \"kubernetes.io/projected/ee12ae27-62c1-43cb-98ec-319b3243ff28-kube-api-access-84x7z\") pod \"lws-controller-manager-59499dc5ff-46274\" (UID: \"ee12ae27-62c1-43cb-98ec-319b3243ff28\") " pod="openshift-lws-operator/lws-controller-manager-59499dc5ff-46274"
Sep 07 11:57:21.041072 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:57:21.041048    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/ee12ae27-62c1-43cb-98ec-319b3243ff28-manager-config\") pod \"lws-controller-manager-59499dc5ff-46274\" (UID: \"ee12ae27-62c1-43cb-98ec-319b3243ff28\") " pod="openshift-lws-operator/lws-controller-manager-59499dc5ff-46274"
Sep 07 11:57:21.042813 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:57:21.042786    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/ee12ae27-62c1-43cb-98ec-319b3243ff28-metrics-cert\") pod \"lws-controller-manager-59499dc5ff-46274\" (UID: \"ee12ae27-62c1-43cb-98ec-319b3243ff28\") " pod="openshift-lws-operator/lws-controller-manager-59499dc5ff-46274"
Sep 07 11:57:21.043082 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:57:21.043061    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ee12ae27-62c1-43cb-98ec-319b3243ff28-cert\") pod \"lws-controller-manager-59499dc5ff-46274\" (UID: \"ee12ae27-62c1-43cb-98ec-319b3243ff28\") " pod="openshift-lws-operator/lws-controller-manager-59499dc5ff-46274"
Sep 07 11:57:21.046383 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:57:21.046357    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-5b6777d477-c7pps"]
Sep 07 11:57:21.049396 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:57:21.049375    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4a58840b_efff_43a4_ac2f_906a5f1b4bdc.slice/crio-11974dbdec0ae55aa13eec3702f262a3c9f115d9d429d3c427a8d73d89618668 WatchSource:0}: Error finding container 11974dbdec0ae55aa13eec3702f262a3c9f115d9d429d3c427a8d73d89618668: Status 404 returned error can't find the container with id 11974dbdec0ae55aa13eec3702f262a3c9f115d9d429d3c427a8d73d89618668
Sep 07 11:57:21.049769 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:57:21.049745    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-84x7z\" (UniqueName: \"kubernetes.io/projected/ee12ae27-62c1-43cb-98ec-319b3243ff28-kube-api-access-84x7z\") pod \"lws-controller-manager-59499dc5ff-46274\" (UID: \"ee12ae27-62c1-43cb-98ec-319b3243ff28\") " pod="openshift-lws-operator/lws-controller-manager-59499dc5ff-46274"
Sep 07 11:57:21.051476 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:57:21.051458    2563 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 07 11:57:21.125787 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:57:21.125751    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-59499dc5ff-46274"
Sep 07 11:57:21.279079 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:57:21.279041    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-5b6777d477-c7pps" event={"ID":"4a58840b-efff-43a4-ac2f-906a5f1b4bdc","Type":"ContainerStarted","Data":"11974dbdec0ae55aa13eec3702f262a3c9f115d9d429d3c427a8d73d89618668"}
Sep 07 11:57:21.404484 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:57:21.404385    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-59499dc5ff-46274"]
Sep 07 11:57:21.408500 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:57:21.408474    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podee12ae27_62c1_43cb_98ec_319b3243ff28.slice/crio-7de5b1580b52a6ddbacc13a89431f73382357de50e29db8cbcc4ec782804812e WatchSource:0}: Error finding container 7de5b1580b52a6ddbacc13a89431f73382357de50e29db8cbcc4ec782804812e: Status 404 returned error can't find the container with id 7de5b1580b52a6ddbacc13a89431f73382357de50e29db8cbcc4ec782804812e
Sep 07 11:57:21.669930 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:57:21.669849    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 11:57:21.849248 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:57:21.849210    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 11:57:22.288220 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:57:22.288183    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-59499dc5ff-46274" event={"ID":"ee12ae27-62c1-43cb-98ec-319b3243ff28","Type":"ContainerStarted","Data":"7de5b1580b52a6ddbacc13a89431f73382357de50e29db8cbcc4ec782804812e"}
Sep 07 11:57:22.692184 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:57:22.692104    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 11:57:25.305244 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:57:25.305206    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-5b6777d477-c7pps" event={"ID":"4a58840b-efff-43a4-ac2f-906a5f1b4bdc","Type":"ContainerStarted","Data":"e3f4218222a2deaae3e187f376601f4fdd9f0dedae98a40394381341bd780eb0"}
Sep 07 11:57:25.306803 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:57:25.306775    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-59499dc5ff-46274" event={"ID":"ee12ae27-62c1-43cb-98ec-319b3243ff28","Type":"ContainerStarted","Data":"6c2d4bf6e55d259d4c79a0d0aa4a771b9848d7dc6bc9740e132d6e7b50771f66"}
Sep 07 11:57:25.306925 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:57:25.306837    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-59499dc5ff-46274"
Sep 07 11:57:25.320133 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:57:25.320093    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-5b6777d477-c7pps" podStartSLOduration=2.047197582 podStartE2EDuration="5.320079132s" totalImagesPullingTime="3.27288155s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 11:57:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 11:57:25.31901623 +0000 UTC m=+616.837748043" watchObservedRunningTime="2026-09-07 11:57:25.320079132 +0000 UTC m=+616.838810944"
Sep 07 11:57:25.335338 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:57:25.335285    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-59499dc5ff-46274" podStartSLOduration=2.426218171 podStartE2EDuration="5.335271135s" totalImagesPullingTime="2.909052964s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 11:57:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 11:57:25.333476275 +0000 UTC m=+616.852208087" watchObservedRunningTime="2026-09-07 11:57:25.335271135 +0000 UTC m=+616.854002948"
Sep 07 11:57:26.349270 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:57:26.349234    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 11:57:26.450109 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:57:26.450072    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 11:57:26.630917 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:57:26.630835    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 11:57:26.759376 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:57:26.759339    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 11:57:26.860315 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:57:26.860284    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 11:57:31.254711 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:57:31.254618    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 11:57:31.355117 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:57:31.355085    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 11:57:31.455320 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:57:31.455273    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 11:57:31.555322 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:57:31.555227    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 11:57:32.114064 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:57:32.114025    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 11:57:32.178720 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:57:32.178680    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 11:57:32.303254 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:57:32.303222    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 11:57:33.801899 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:57:33.801865    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 11:57:33.946656 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:57:33.946618    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 11:57:34.047744 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:57:34.047709    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 11:57:34.548402 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:57:34.548364    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 11:57:34.597299 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:57:34.597268    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 11:57:35.053040 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:57:35.052998    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 11:57:35.105561 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:57:35.105531    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 11:57:36.314547 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:57:36.314513    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-59499dc5ff-46274"
Sep 07 11:57:40.662515 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:57:40.662480    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 11:57:40.760042 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:57:40.760009    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 11:57:41.742555 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:57:41.742524    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 11:57:41.839767 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:57:41.839729    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 11:57:44.885452 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:57:44.885396    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 11:57:45.072577 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:57:45.072539    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 11:57:45.149920 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:57:45.149830    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 11:57:55.399667 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:57:55.399631    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 11:57:55.452203 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:57:55.452171    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 11:57:58.619495 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:57:58.619462    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 11:58:03.358618 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:58:03.358586    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 11:58:03.409125 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:58:03.409089    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 11:58:05.108515 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:58:05.108476    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 11:58:05.159106 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:58:05.159077    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 11:58:32.249031 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:58:32.248995    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 11:58:32.305388 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:58:32.305352    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 11:58:44.161231 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:58:44.161199    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 11:58:44.959785 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:58:44.959752    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 11:58:45.766589 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:58:45.766551    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 11:58:48.982528 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:58:48.982482    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 11:58:49.078837 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:58:49.078804    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 11:58:49.372697 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:58:49.372621    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 11:58:51.379049 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:58:51.379010    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 11:58:51.478429 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:58:51.478397    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 11:58:52.602524 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:58:52.602488    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 11:58:53.373983 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:58:53.373949    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 11:58:53.761791 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:58:53.761759    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 11:58:55.337331 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:58:55.337288    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 11:58:55.434475 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:58:55.434444    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 11:58:56.372555 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:58:56.372506    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 11:58:56.871688 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:58:56.871654    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 11:58:59.365697 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:58:59.365663    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 11:58:59.413672 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:58:59.413644    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 11:58:59.768465 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:58:59.768433    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 11:58:59.867767 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:58:59.867729    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 11:59:00.166848 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:59:00.166771    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 11:59:00.266916 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:59:00.266884    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 11:59:06.401985 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:59:06.401903    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 11:59:06.506089 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:59:06.506049    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 11:59:08.832019 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:59:08.831985    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 11:59:09.567064 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:59:09.567017    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 11:59:15.380119 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:59:15.380083    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 11:59:15.480024 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:59:15.479968    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 11:59:18.295021 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:59:18.294978    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 11:59:18.400676 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:59:18.400641    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 11:59:20.398831 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:59:20.398795    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 11:59:20.512627 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:59:20.512594    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 11:59:20.790920 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:59:20.790879    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 11:59:20.892002 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:59:20.891972    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 11:59:21.181514 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:59:21.181408    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 11:59:21.282498 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:59:21.282456    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 11:59:27.855640 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:59:27.855605    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 11:59:27.952296 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:59:27.952263    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 11:59:30.239528 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:59:30.239488    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 11:59:30.341070 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:59:30.341031    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 11:59:35.700345 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:59:35.700309    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 11:59:35.968092 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:59:35.968005    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 11:59:36.827961 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:59:36.827922    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 11:59:36.880757 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:59:36.880727    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 11:59:37.774400 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:59:37.774359    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 11:59:37.874115 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:59:37.874084    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 11:59:37.925746 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:59:37.925713    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 11:59:38.161184 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:59:38.161111    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-p4l4w"]
Sep 07 11:59:38.164404 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:59:38.164384    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-85d9db4d5d-p4l4w"
Sep 07 11:59:38.166096 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:59:38.166074    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Sep 07 11:59:38.166096 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:59:38.166092    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Sep 07 11:59:38.166280 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:59:38.166153    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Sep 07 11:59:38.166397 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:59:38.166382    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-f8jzb\""
Sep 07 11:59:38.171474 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:59:38.171213    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-p4l4w"]
Sep 07 11:59:38.199089 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:59:38.199064    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-p4l4w"]
Sep 07 11:59:38.299320 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:59:38.299289    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ztmf6\" (UniqueName: \"kubernetes.io/projected/697c2418-985e-43e8-b5bc-a2fc199cd677-kube-api-access-ztmf6\") pod \"limitador-limitador-85d9db4d5d-p4l4w\" (UID: \"697c2418-985e-43e8-b5bc-a2fc199cd677\") " pod="kuadrant-system/limitador-limitador-85d9db4d5d-p4l4w"
Sep 07 11:59:38.299473 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:59:38.299330    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/697c2418-985e-43e8-b5bc-a2fc199cd677-config-file\") pod \"limitador-limitador-85d9db4d5d-p4l4w\" (UID: \"697c2418-985e-43e8-b5bc-a2fc199cd677\") " pod="kuadrant-system/limitador-limitador-85d9db4d5d-p4l4w"
Sep 07 11:59:38.400823 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:59:38.400792    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ztmf6\" (UniqueName: \"kubernetes.io/projected/697c2418-985e-43e8-b5bc-a2fc199cd677-kube-api-access-ztmf6\") pod \"limitador-limitador-85d9db4d5d-p4l4w\" (UID: \"697c2418-985e-43e8-b5bc-a2fc199cd677\") " pod="kuadrant-system/limitador-limitador-85d9db4d5d-p4l4w"
Sep 07 11:59:38.400924 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:59:38.400827    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/697c2418-985e-43e8-b5bc-a2fc199cd677-config-file\") pod \"limitador-limitador-85d9db4d5d-p4l4w\" (UID: \"697c2418-985e-43e8-b5bc-a2fc199cd677\") " pod="kuadrant-system/limitador-limitador-85d9db4d5d-p4l4w"
Sep 07 11:59:38.401540 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:59:38.401500    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/697c2418-985e-43e8-b5bc-a2fc199cd677-config-file\") pod \"limitador-limitador-85d9db4d5d-p4l4w\" (UID: \"697c2418-985e-43e8-b5bc-a2fc199cd677\") " pod="kuadrant-system/limitador-limitador-85d9db4d5d-p4l4w"
Sep 07 11:59:38.407547 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:59:38.407525    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ztmf6\" (UniqueName: \"kubernetes.io/projected/697c2418-985e-43e8-b5bc-a2fc199cd677-kube-api-access-ztmf6\") pod \"limitador-limitador-85d9db4d5d-p4l4w\" (UID: \"697c2418-985e-43e8-b5bc-a2fc199cd677\") " pod="kuadrant-system/limitador-limitador-85d9db4d5d-p4l4w"
Sep 07 11:59:38.475252 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:59:38.475231    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-85d9db4d5d-p4l4w"
Sep 07 11:59:38.591022 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:59:38.590995    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-b54df94b4-bkcfr"]
Sep 07 11:59:38.594511 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:59:38.594486    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-b54df94b4-bkcfr"
Sep 07 11:59:38.596615 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:59:38.596598    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-gfsh7\""
Sep 07 11:59:38.603392 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:59:38.603370    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-b54df94b4-bkcfr"]
Sep 07 11:59:38.647127 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:59:38.647099    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 11:59:38.703115 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:59:38.703088    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8knk9\" (UniqueName: \"kubernetes.io/projected/628dbf08-1fd9-4446-bdfd-c907df58aaee-kube-api-access-8knk9\") pod \"authorino-b54df94b4-bkcfr\" (UID: \"628dbf08-1fd9-4446-bdfd-c907df58aaee\") " pod="kuadrant-system/authorino-b54df94b4-bkcfr"
Sep 07 11:59:38.741131 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:59:38.741110    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-p4l4w"]
Sep 07 11:59:38.743819 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:59:38.743772    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 11:59:38.743976 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:59:38.743956    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod697c2418_985e_43e8_b5bc_a2fc199cd677.slice/crio-1f86a6854ab56c6b1f7d99ac2197579f3eb9d9fb3bb2aa196508225d2a06c773 WatchSource:0}: Error finding container 1f86a6854ab56c6b1f7d99ac2197579f3eb9d9fb3bb2aa196508225d2a06c773: Status 404 returned error can't find the container with id 1f86a6854ab56c6b1f7d99ac2197579f3eb9d9fb3bb2aa196508225d2a06c773
Sep 07 11:59:38.803792 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:59:38.803771    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8knk9\" (UniqueName: \"kubernetes.io/projected/628dbf08-1fd9-4446-bdfd-c907df58aaee-kube-api-access-8knk9\") pod \"authorino-b54df94b4-bkcfr\" (UID: \"628dbf08-1fd9-4446-bdfd-c907df58aaee\") " pod="kuadrant-system/authorino-b54df94b4-bkcfr"
Sep 07 11:59:38.811382 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:59:38.811356    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8knk9\" (UniqueName: \"kubernetes.io/projected/628dbf08-1fd9-4446-bdfd-c907df58aaee-kube-api-access-8knk9\") pod \"authorino-b54df94b4-bkcfr\" (UID: \"628dbf08-1fd9-4446-bdfd-c907df58aaee\") " pod="kuadrant-system/authorino-b54df94b4-bkcfr"
Sep 07 11:59:38.901600 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:59:38.901570    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-85d9db4d5d-p4l4w" event={"ID":"697c2418-985e-43e8-b5bc-a2fc199cd677","Type":"ContainerStarted","Data":"1f86a6854ab56c6b1f7d99ac2197579f3eb9d9fb3bb2aa196508225d2a06c773"}
Sep 07 11:59:38.904775 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:59:38.904758    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-b54df94b4-bkcfr"
Sep 07 11:59:39.170200 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:59:39.170080    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-b54df94b4-bkcfr"]
Sep 07 11:59:39.172691 ip-10-0-135-173 kubenswrapper[2563]: W0907 11:59:39.172663    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod628dbf08_1fd9_4446_bdfd_c907df58aaee.slice/crio-56eeb3e475098629cf442ebab12a71e7f7b1517c86420a937f5921fd1a4c8fe8 WatchSource:0}: Error finding container 56eeb3e475098629cf442ebab12a71e7f7b1517c86420a937f5921fd1a4c8fe8: Status 404 returned error can't find the container with id 56eeb3e475098629cf442ebab12a71e7f7b1517c86420a937f5921fd1a4c8fe8
Sep 07 11:59:39.910669 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:59:39.910632    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-b54df94b4-bkcfr" event={"ID":"628dbf08-1fd9-4446-bdfd-c907df58aaee","Type":"ContainerStarted","Data":"56eeb3e475098629cf442ebab12a71e7f7b1517c86420a937f5921fd1a4c8fe8"}
Sep 07 11:59:41.518186 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:59:41.518161    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-b54df94b4-bkcfr"]
Sep 07 11:59:41.921968 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:59:41.921859    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-b54df94b4-bkcfr" event={"ID":"628dbf08-1fd9-4446-bdfd-c907df58aaee","Type":"ContainerStarted","Data":"0e705db81d7a581deba11e5645c6b67621247fd9587f612883532f916e55b68f"}
Sep 07 11:59:41.921968 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:59:41.921868    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-b54df94b4-bkcfr" podUID="628dbf08-1fd9-4446-bdfd-c907df58aaee" containerName="authorino" containerID="cri-o://0e705db81d7a581deba11e5645c6b67621247fd9587f612883532f916e55b68f" gracePeriod=30
Sep 07 11:59:41.923411 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:59:41.923376    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-85d9db4d5d-p4l4w" event={"ID":"697c2418-985e-43e8-b5bc-a2fc199cd677","Type":"ContainerStarted","Data":"caf7fbe067cd537066a29c54929488eb2d01840a3e4eea749c22af18647844e6"}
Sep 07 11:59:41.923546 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:59:41.923526    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-85d9db4d5d-p4l4w"
Sep 07 11:59:41.936054 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:59:41.936017    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-b54df94b4-bkcfr" podStartSLOduration=1.590597139 podStartE2EDuration="3.936004546s" totalImagesPullingTime="2.345407407s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 11:59:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 11:59:41.934213539 +0000 UTC m=+753.452945352" watchObservedRunningTime="2026-09-07 11:59:41.936004546 +0000 UTC m=+753.454736355"
Sep 07 11:59:41.948667 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:59:41.948625    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-85d9db4d5d-p4l4w" podStartSLOduration=1.181160958 podStartE2EDuration="3.948612681s" totalImagesPullingTime="2.767451723s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 11:59:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 11:59:41.946536482 +0000 UTC m=+753.465268296" watchObservedRunningTime="2026-09-07 11:59:41.948612681 +0000 UTC m=+753.467344493"
Sep 07 11:59:42.152900 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:59:42.152878    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-b54df94b4-bkcfr"
Sep 07 11:59:42.236296 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:59:42.236268    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8knk9\" (UniqueName: \"kubernetes.io/projected/628dbf08-1fd9-4446-bdfd-c907df58aaee-kube-api-access-8knk9\") pod \"628dbf08-1fd9-4446-bdfd-c907df58aaee\" (UID: \"628dbf08-1fd9-4446-bdfd-c907df58aaee\") "
Sep 07 11:59:42.238348 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:59:42.238326    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/628dbf08-1fd9-4446-bdfd-c907df58aaee-kube-api-access-8knk9" (OuterVolumeSpecName: "kube-api-access-8knk9") pod "628dbf08-1fd9-4446-bdfd-c907df58aaee" (UID: "628dbf08-1fd9-4446-bdfd-c907df58aaee"). InnerVolumeSpecName "kube-api-access-8knk9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 11:59:42.337243 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:59:42.337201    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8knk9\" (UniqueName: \"kubernetes.io/projected/628dbf08-1fd9-4446-bdfd-c907df58aaee-kube-api-access-8knk9\") on node \"ip-10-0-135-173.ec2.internal\" DevicePath \"\""
Sep 07 11:59:42.929260 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:59:42.929189    2563 generic.go:356] "Generic (PLEG): container finished" podID="628dbf08-1fd9-4446-bdfd-c907df58aaee" containerID="0e705db81d7a581deba11e5645c6b67621247fd9587f612883532f916e55b68f" exitCode=0
Sep 07 11:59:42.929260 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:59:42.929229    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-b54df94b4-bkcfr"
Sep 07 11:59:42.929750 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:59:42.929270    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-b54df94b4-bkcfr" event={"ID":"628dbf08-1fd9-4446-bdfd-c907df58aaee","Type":"ContainerDied","Data":"0e705db81d7a581deba11e5645c6b67621247fd9587f612883532f916e55b68f"}
Sep 07 11:59:42.929750 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:59:42.929307    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-b54df94b4-bkcfr" event={"ID":"628dbf08-1fd9-4446-bdfd-c907df58aaee","Type":"ContainerDied","Data":"56eeb3e475098629cf442ebab12a71e7f7b1517c86420a937f5921fd1a4c8fe8"}
Sep 07 11:59:42.929750 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:59:42.929324    2563 scope.go:117] "RemoveContainer" containerID="0e705db81d7a581deba11e5645c6b67621247fd9587f612883532f916e55b68f"
Sep 07 11:59:42.938222 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:59:42.938206    2563 scope.go:117] "RemoveContainer" containerID="0e705db81d7a581deba11e5645c6b67621247fd9587f612883532f916e55b68f"
Sep 07 11:59:42.938491 ip-10-0-135-173 kubenswrapper[2563]: E0907 11:59:42.938475    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0e705db81d7a581deba11e5645c6b67621247fd9587f612883532f916e55b68f\": container with ID starting with 0e705db81d7a581deba11e5645c6b67621247fd9587f612883532f916e55b68f not found: ID does not exist" containerID="0e705db81d7a581deba11e5645c6b67621247fd9587f612883532f916e55b68f"
Sep 07 11:59:42.938556 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:59:42.938501    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0e705db81d7a581deba11e5645c6b67621247fd9587f612883532f916e55b68f"} err="failed to get container status \"0e705db81d7a581deba11e5645c6b67621247fd9587f612883532f916e55b68f\": rpc error: code = NotFound desc = could not find container \"0e705db81d7a581deba11e5645c6b67621247fd9587f612883532f916e55b68f\": container with ID starting with 0e705db81d7a581deba11e5645c6b67621247fd9587f612883532f916e55b68f not found: ID does not exist"
Sep 07 11:59:42.949348 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:59:42.949325    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-b54df94b4-bkcfr"]
Sep 07 11:59:42.953103 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:59:42.953081    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-b54df94b4-bkcfr"]
Sep 07 11:59:43.040044 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:59:43.040020    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="628dbf08-1fd9-4446-bdfd-c907df58aaee" path="/var/lib/kubelet/pods/628dbf08-1fd9-4446-bdfd-c907df58aaee/volumes"
Sep 07 11:59:52.930073 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:59:52.930044    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-85d9db4d5d-p4l4w"
Sep 07 11:59:59.018385 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:59:59.018347    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 11:59:59.117725 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:59:59.117690    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 11:59:59.471988 ip-10-0-135-173 kubenswrapper[2563]: I0907 11:59:59.471943    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:00:11.974903 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:00:11.974864    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:00:12.265471 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:00:12.265361    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:00:20.416343 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:00:20.416301    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:00:20.516939 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:00:20.516903    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:00:20.569003 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:00:20.568973    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:00:20.847308 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:00:20.847270    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:00:20.948794 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:00:20.948762    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:00:24.423112 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:00:24.423073    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:00:24.552910 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:00:24.552876    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:00:57.627989 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:00:57.627951    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:00:57.734498 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:00:57.734458    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:00:59.070766 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:00:59.070730    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:00:59.262550 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:00:59.262509    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:00:59.363477 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:00:59.363380    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:00:59.508612 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:00:59.508580    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:00:59.603269 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:00:59.603234    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:00:59.693377 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:00:59.693287    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:00:59.722323 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:00:59.722291    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-p4l4w"]
Sep 07 12:01:12.054329 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:01:12.054292    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:01:12.152237 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:01:12.152182    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:01:25.829299 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:01:25.829257    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:01:26.404138 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:01:26.404107    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-p4l4w"]
Sep 07 12:01:32.330476 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:01:32.330396    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:01:33.105784 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:01:33.105747    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-p4l4w"]
Sep 07 12:01:37.362021 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:01:37.361970    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:01:39.003967 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:01:39.003928    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-embedding-simulated-kserve-bcc9ff8d8-5nncz"]
Sep 07 12:01:39.004454 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:01:39.004437    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="628dbf08-1fd9-4446-bdfd-c907df58aaee" containerName="authorino"
Sep 07 12:01:39.004543 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:01:39.004458    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="628dbf08-1fd9-4446-bdfd-c907df58aaee" containerName="authorino"
Sep 07 12:01:39.004597 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:01:39.004577    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="628dbf08-1fd9-4446-bdfd-c907df58aaee" containerName="authorino"
Sep 07 12:01:39.007267 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:01:39.007247    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-5nncz"
Sep 07 12:01:39.008868 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:01:39.008846    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-59vv8\""
Sep 07 12:01:39.009148 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:01:39.009132    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Sep 07 12:01:39.009233 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:01:39.009161    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Sep 07 12:01:39.009461 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:01:39.009445    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-embedding-simulated-kserve-self-signed-certs\""
Sep 07 12:01:39.017250 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:01:39.017229    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-embedding-simulated-kserve-bcc9ff8d8-5nncz"]
Sep 07 12:01:39.068937 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:01:39.068909    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e0abbddf-d934-490d-b520-b4a2c0dab142-tmp-dir\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-5nncz\" (UID: \"e0abbddf-d934-490d-b520-b4a2c0dab142\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-5nncz"
Sep 07 12:01:39.069075 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:01:39.068947    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e0abbddf-d934-490d-b520-b4a2c0dab142-home\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-5nncz\" (UID: \"e0abbddf-d934-490d-b520-b4a2c0dab142\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-5nncz"
Sep 07 12:01:39.069075 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:01:39.068987    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e0abbddf-d934-490d-b520-b4a2c0dab142-dshm\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-5nncz\" (UID: \"e0abbddf-d934-490d-b520-b4a2c0dab142\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-5nncz"
Sep 07 12:01:39.069075 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:01:39.069004    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e0abbddf-d934-490d-b520-b4a2c0dab142-tls-certs\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-5nncz\" (UID: \"e0abbddf-d934-490d-b520-b4a2c0dab142\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-5nncz"
Sep 07 12:01:39.069075 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:01:39.069028    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e0abbddf-d934-490d-b520-b4a2c0dab142-model-cache\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-5nncz\" (UID: \"e0abbddf-d934-490d-b520-b4a2c0dab142\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-5nncz"
Sep 07 12:01:39.069075 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:01:39.069051    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6dbc5\" (UniqueName: \"kubernetes.io/projected/e0abbddf-d934-490d-b520-b4a2c0dab142-kube-api-access-6dbc5\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-5nncz\" (UID: \"e0abbddf-d934-490d-b520-b4a2c0dab142\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-5nncz"
Sep 07 12:01:39.170296 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:01:39.170267    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e0abbddf-d934-490d-b520-b4a2c0dab142-dshm\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-5nncz\" (UID: \"e0abbddf-d934-490d-b520-b4a2c0dab142\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-5nncz"
Sep 07 12:01:39.170441 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:01:39.170304    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e0abbddf-d934-490d-b520-b4a2c0dab142-tls-certs\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-5nncz\" (UID: \"e0abbddf-d934-490d-b520-b4a2c0dab142\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-5nncz"
Sep 07 12:01:39.170441 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:01:39.170341    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e0abbddf-d934-490d-b520-b4a2c0dab142-model-cache\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-5nncz\" (UID: \"e0abbddf-d934-490d-b520-b4a2c0dab142\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-5nncz"
Sep 07 12:01:39.170441 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:01:39.170369    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6dbc5\" (UniqueName: \"kubernetes.io/projected/e0abbddf-d934-490d-b520-b4a2c0dab142-kube-api-access-6dbc5\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-5nncz\" (UID: \"e0abbddf-d934-490d-b520-b4a2c0dab142\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-5nncz"
Sep 07 12:01:39.170441 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:01:39.170404    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e0abbddf-d934-490d-b520-b4a2c0dab142-tmp-dir\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-5nncz\" (UID: \"e0abbddf-d934-490d-b520-b4a2c0dab142\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-5nncz"
Sep 07 12:01:39.170666 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:01:39.170478    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e0abbddf-d934-490d-b520-b4a2c0dab142-home\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-5nncz\" (UID: \"e0abbddf-d934-490d-b520-b4a2c0dab142\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-5nncz"
Sep 07 12:01:39.170839 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:01:39.170805    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e0abbddf-d934-490d-b520-b4a2c0dab142-model-cache\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-5nncz\" (UID: \"e0abbddf-d934-490d-b520-b4a2c0dab142\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-5nncz"
Sep 07 12:01:39.170959 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:01:39.170898    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e0abbddf-d934-490d-b520-b4a2c0dab142-tmp-dir\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-5nncz\" (UID: \"e0abbddf-d934-490d-b520-b4a2c0dab142\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-5nncz"
Sep 07 12:01:39.171016 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:01:39.170966    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e0abbddf-d934-490d-b520-b4a2c0dab142-home\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-5nncz\" (UID: \"e0abbddf-d934-490d-b520-b4a2c0dab142\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-5nncz"
Sep 07 12:01:39.172808 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:01:39.172753    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e0abbddf-d934-490d-b520-b4a2c0dab142-dshm\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-5nncz\" (UID: \"e0abbddf-d934-490d-b520-b4a2c0dab142\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-5nncz"
Sep 07 12:01:39.172914 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:01:39.172895    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e0abbddf-d934-490d-b520-b4a2c0dab142-tls-certs\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-5nncz\" (UID: \"e0abbddf-d934-490d-b520-b4a2c0dab142\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-5nncz"
Sep 07 12:01:39.177157 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:01:39.177135    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6dbc5\" (UniqueName: \"kubernetes.io/projected/e0abbddf-d934-490d-b520-b4a2c0dab142-kube-api-access-6dbc5\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-5nncz\" (UID: \"e0abbddf-d934-490d-b520-b4a2c0dab142\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-5nncz"
Sep 07 12:01:39.210158 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:01:39.210132    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:01:39.317240 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:01:39.317181    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-5nncz"
Sep 07 12:01:39.584944 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:01:39.584865    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-embedding-simulated-kserve-bcc9ff8d8-5nncz"]
Sep 07 12:01:39.587885 ip-10-0-135-173 kubenswrapper[2563]: W0907 12:01:39.587853    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode0abbddf_d934_490d_b520_b4a2c0dab142.slice/crio-e43bf96e675b6e54498a75f876943a51fd11fd1862d382086121dd4475cf8afc WatchSource:0}: Error finding container e43bf96e675b6e54498a75f876943a51fd11fd1862d382086121dd4475cf8afc: Status 404 returned error can't find the container with id e43bf96e675b6e54498a75f876943a51fd11fd1862d382086121dd4475cf8afc
Sep 07 12:01:40.196012 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:01:40.195975    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-p4l4w"]
Sep 07 12:01:40.464060 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:01:40.463977    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-5nncz" event={"ID":"e0abbddf-d934-490d-b520-b4a2c0dab142","Type":"ContainerStarted","Data":"e43bf96e675b6e54498a75f876943a51fd11fd1862d382086121dd4475cf8afc"}
Sep 07 12:01:41.475486 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:01:41.475447    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-5nncz" event={"ID":"e0abbddf-d934-490d-b520-b4a2c0dab142","Type":"ContainerStarted","Data":"3707302ce854e5b46c422fa6e5dc564f4209492d49f9e761d0c46d6ef672bb4b"}
Sep 07 12:01:41.490642 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:01:41.490588    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-5nncz" podStartSLOduration=2.497105964 podStartE2EDuration="3.490568734s" totalImagesPullingTime="993.46277ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 12:01:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 12:01:41.489979987 +0000 UTC m=+873.008711802" watchObservedRunningTime="2026-09-07 12:01:41.490568734 +0000 UTC m=+873.009300548"
Sep 07 12:01:48.333898 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:01:48.333854    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:01:49.318002 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:01:49.317952    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-5nncz"
Sep 07 12:01:49.318167 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:01:49.318032    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-5nncz"
Sep 07 12:01:49.330108 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:01:49.330088    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-5nncz"
Sep 07 12:01:49.511753 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:01:49.511720    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-p4l4w"]
Sep 07 12:01:49.525836 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:01:49.525811    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-5nncz"
Sep 07 12:01:51.722173 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:01:51.722123    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:01:54.002024 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:01:54.001985    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-p4l4w"]
Sep 07 12:01:56.292876 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:01:56.292847    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-nj42c"]
Sep 07 12:01:56.296390 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:01:56.296368    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-nj42c"
Sep 07 12:01:56.298268 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:01:56.298243    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\""
Sep 07 12:01:56.306083 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:01:56.306059    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-nj42c"]
Sep 07 12:01:56.413041 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:01:56.412997    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:01:56.427178 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:01:56.427151    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/47d91c30-7252-4e9d-944f-494b41376f94-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-nj42c\" (UID: \"47d91c30-7252-4e9d-944f-494b41376f94\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-nj42c"
Sep 07 12:01:56.427272 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:01:56.427207    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/47d91c30-7252-4e9d-944f-494b41376f94-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-nj42c\" (UID: \"47d91c30-7252-4e9d-944f-494b41376f94\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-nj42c"
Sep 07 12:01:56.427272 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:01:56.427233    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/47d91c30-7252-4e9d-944f-494b41376f94-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-nj42c\" (UID: \"47d91c30-7252-4e9d-944f-494b41376f94\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-nj42c"
Sep 07 12:01:56.427272 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:01:56.427256    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/47d91c30-7252-4e9d-944f-494b41376f94-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-nj42c\" (UID: \"47d91c30-7252-4e9d-944f-494b41376f94\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-nj42c"
Sep 07 12:01:56.427390 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:01:56.427275    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/47d91c30-7252-4e9d-944f-494b41376f94-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-nj42c\" (UID: \"47d91c30-7252-4e9d-944f-494b41376f94\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-nj42c"
Sep 07 12:01:56.427390 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:01:56.427343    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-djrg2\" (UniqueName: \"kubernetes.io/projected/47d91c30-7252-4e9d-944f-494b41376f94-kube-api-access-djrg2\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-nj42c\" (UID: \"47d91c30-7252-4e9d-944f-494b41376f94\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-nj42c"
Sep 07 12:01:56.528810 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:01:56.528763    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/47d91c30-7252-4e9d-944f-494b41376f94-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-nj42c\" (UID: \"47d91c30-7252-4e9d-944f-494b41376f94\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-nj42c"
Sep 07 12:01:56.529063 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:01:56.528839    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/47d91c30-7252-4e9d-944f-494b41376f94-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-nj42c\" (UID: \"47d91c30-7252-4e9d-944f-494b41376f94\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-nj42c"
Sep 07 12:01:56.529063 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:01:56.528860    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/47d91c30-7252-4e9d-944f-494b41376f94-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-nj42c\" (UID: \"47d91c30-7252-4e9d-944f-494b41376f94\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-nj42c"
Sep 07 12:01:56.529063 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:01:56.528880    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/47d91c30-7252-4e9d-944f-494b41376f94-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-nj42c\" (UID: \"47d91c30-7252-4e9d-944f-494b41376f94\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-nj42c"
Sep 07 12:01:56.529063 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:01:56.528908    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/47d91c30-7252-4e9d-944f-494b41376f94-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-nj42c\" (UID: \"47d91c30-7252-4e9d-944f-494b41376f94\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-nj42c"
Sep 07 12:01:56.529063 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:01:56.528964    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-djrg2\" (UniqueName: \"kubernetes.io/projected/47d91c30-7252-4e9d-944f-494b41376f94-kube-api-access-djrg2\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-nj42c\" (UID: \"47d91c30-7252-4e9d-944f-494b41376f94\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-nj42c"
Sep 07 12:01:56.529324 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:01:56.529250    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/47d91c30-7252-4e9d-944f-494b41376f94-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-nj42c\" (UID: \"47d91c30-7252-4e9d-944f-494b41376f94\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-nj42c"
Sep 07 12:01:56.529324 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:01:56.529302    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/47d91c30-7252-4e9d-944f-494b41376f94-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-nj42c\" (UID: \"47d91c30-7252-4e9d-944f-494b41376f94\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-nj42c"
Sep 07 12:01:56.529405 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:01:56.529333    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/47d91c30-7252-4e9d-944f-494b41376f94-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-nj42c\" (UID: \"47d91c30-7252-4e9d-944f-494b41376f94\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-nj42c"
Sep 07 12:01:56.531062 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:01:56.531041    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/47d91c30-7252-4e9d-944f-494b41376f94-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-nj42c\" (UID: \"47d91c30-7252-4e9d-944f-494b41376f94\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-nj42c"
Sep 07 12:01:56.531465 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:01:56.531442    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/47d91c30-7252-4e9d-944f-494b41376f94-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-nj42c\" (UID: \"47d91c30-7252-4e9d-944f-494b41376f94\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-nj42c"
Sep 07 12:01:56.535940 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:01:56.535916    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-djrg2\" (UniqueName: \"kubernetes.io/projected/47d91c30-7252-4e9d-944f-494b41376f94-kube-api-access-djrg2\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-nj42c\" (UID: \"47d91c30-7252-4e9d-944f-494b41376f94\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-nj42c"
Sep 07 12:01:56.626307 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:01:56.626212    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-nj42c"
Sep 07 12:01:56.898159 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:01:56.898131    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-nj42c"]
Sep 07 12:01:56.900208 ip-10-0-135-173 kubenswrapper[2563]: W0907 12:01:56.900183    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod47d91c30_7252_4e9d_944f_494b41376f94.slice/crio-8792119ca8a2f69ce6edc2183542c224e0a6c5d07fcf26558c9af866e47e0165 WatchSource:0}: Error finding container 8792119ca8a2f69ce6edc2183542c224e0a6c5d07fcf26558c9af866e47e0165: Status 404 returned error can't find the container with id 8792119ca8a2f69ce6edc2183542c224e0a6c5d07fcf26558c9af866e47e0165
Sep 07 12:01:57.555763 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:01:57.555730    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-nj42c" event={"ID":"47d91c30-7252-4e9d-944f-494b41376f94","Type":"ContainerStarted","Data":"09e0bc152ffcb61c25a05564298e49bac91f2df22c709d7ff10676d32d66c8b4"}
Sep 07 12:01:57.556132 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:01:57.555778    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-nj42c" event={"ID":"47d91c30-7252-4e9d-944f-494b41376f94","Type":"ContainerStarted","Data":"8792119ca8a2f69ce6edc2183542c224e0a6c5d07fcf26558c9af866e47e0165"}
Sep 07 12:01:57.571587 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:01:57.571541    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-nj42c" podStartSLOduration=1.349743945 podStartE2EDuration="1.571525861s" totalImagesPullingTime="221.781916ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 12:01:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 12:01:57.569548063 +0000 UTC m=+889.088279877" watchObservedRunningTime="2026-09-07 12:01:57.571525861 +0000 UTC m=+889.090257675"
Sep 07 12:01:58.398529 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:01:58.398491    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-p4l4w"]
Sep 07 12:02:00.710048 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:02:00.710006    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:02:00.811072 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:02:00.811034    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:02:03.317884 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:02:03.317852    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-7n5g7"]
Sep 07 12:02:03.321054 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:02:03.321038    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-7n5g7"
Sep 07 12:02:03.323313 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:02:03.323292    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\""
Sep 07 12:02:03.347774 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:02:03.347749    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-7n5g7"]
Sep 07 12:02:03.394342 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:02:03.394315    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/dfdfb25e-f951-4d2c-87b0-26ee1ec10310-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-7n5g7\" (UID: \"dfdfb25e-f951-4d2c-87b0-26ee1ec10310\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-7n5g7"
Sep 07 12:02:03.394474 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:02:03.394359    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dfdfb25e-f951-4d2c-87b0-26ee1ec10310-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-7n5g7\" (UID: \"dfdfb25e-f951-4d2c-87b0-26ee1ec10310\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-7n5g7"
Sep 07 12:02:03.394474 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:02:03.394439    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/dfdfb25e-f951-4d2c-87b0-26ee1ec10310-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-7n5g7\" (UID: \"dfdfb25e-f951-4d2c-87b0-26ee1ec10310\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-7n5g7"
Sep 07 12:02:03.394474 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:02:03.394459    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gjqhp\" (UniqueName: \"kubernetes.io/projected/dfdfb25e-f951-4d2c-87b0-26ee1ec10310-kube-api-access-gjqhp\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-7n5g7\" (UID: \"dfdfb25e-f951-4d2c-87b0-26ee1ec10310\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-7n5g7"
Sep 07 12:02:03.394601 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:02:03.394487    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dfdfb25e-f951-4d2c-87b0-26ee1ec10310-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-7n5g7\" (UID: \"dfdfb25e-f951-4d2c-87b0-26ee1ec10310\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-7n5g7"
Sep 07 12:02:03.394601 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:02:03.394507    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/dfdfb25e-f951-4d2c-87b0-26ee1ec10310-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-7n5g7\" (UID: \"dfdfb25e-f951-4d2c-87b0-26ee1ec10310\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-7n5g7"
Sep 07 12:02:03.433070 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:02:03.433045    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:02:03.495495 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:02:03.495470    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/dfdfb25e-f951-4d2c-87b0-26ee1ec10310-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-7n5g7\" (UID: \"dfdfb25e-f951-4d2c-87b0-26ee1ec10310\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-7n5g7"
Sep 07 12:02:03.495603 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:02:03.495513    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/dfdfb25e-f951-4d2c-87b0-26ee1ec10310-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-7n5g7\" (UID: \"dfdfb25e-f951-4d2c-87b0-26ee1ec10310\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-7n5g7"
Sep 07 12:02:03.495603 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:02:03.495552    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dfdfb25e-f951-4d2c-87b0-26ee1ec10310-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-7n5g7\" (UID: \"dfdfb25e-f951-4d2c-87b0-26ee1ec10310\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-7n5g7"
Sep 07 12:02:03.495716 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:02:03.495611    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/dfdfb25e-f951-4d2c-87b0-26ee1ec10310-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-7n5g7\" (UID: \"dfdfb25e-f951-4d2c-87b0-26ee1ec10310\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-7n5g7"
Sep 07 12:02:03.495716 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:02:03.495642    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gjqhp\" (UniqueName: \"kubernetes.io/projected/dfdfb25e-f951-4d2c-87b0-26ee1ec10310-kube-api-access-gjqhp\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-7n5g7\" (UID: \"dfdfb25e-f951-4d2c-87b0-26ee1ec10310\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-7n5g7"
Sep 07 12:02:03.495716 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:02:03.495681    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dfdfb25e-f951-4d2c-87b0-26ee1ec10310-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-7n5g7\" (UID: \"dfdfb25e-f951-4d2c-87b0-26ee1ec10310\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-7n5g7"
Sep 07 12:02:03.495994 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:02:03.495968    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dfdfb25e-f951-4d2c-87b0-26ee1ec10310-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-7n5g7\" (UID: \"dfdfb25e-f951-4d2c-87b0-26ee1ec10310\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-7n5g7"
Sep 07 12:02:03.496273 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:02:03.496021    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/dfdfb25e-f951-4d2c-87b0-26ee1ec10310-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-7n5g7\" (UID: \"dfdfb25e-f951-4d2c-87b0-26ee1ec10310\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-7n5g7"
Sep 07 12:02:03.496273 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:02:03.496065    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/dfdfb25e-f951-4d2c-87b0-26ee1ec10310-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-7n5g7\" (UID: \"dfdfb25e-f951-4d2c-87b0-26ee1ec10310\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-7n5g7"
Sep 07 12:02:03.497683 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:02:03.497666    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/dfdfb25e-f951-4d2c-87b0-26ee1ec10310-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-7n5g7\" (UID: \"dfdfb25e-f951-4d2c-87b0-26ee1ec10310\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-7n5g7"
Sep 07 12:02:03.497954 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:02:03.497935    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dfdfb25e-f951-4d2c-87b0-26ee1ec10310-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-7n5g7\" (UID: \"dfdfb25e-f951-4d2c-87b0-26ee1ec10310\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-7n5g7"
Sep 07 12:02:03.502344 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:02:03.502317    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gjqhp\" (UniqueName: \"kubernetes.io/projected/dfdfb25e-f951-4d2c-87b0-26ee1ec10310-kube-api-access-gjqhp\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-7n5g7\" (UID: \"dfdfb25e-f951-4d2c-87b0-26ee1ec10310\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-7n5g7"
Sep 07 12:02:03.630188 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:02:03.630123    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-7n5g7"
Sep 07 12:02:03.897897 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:02:03.897867    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-7n5g7"]
Sep 07 12:02:03.900599 ip-10-0-135-173 kubenswrapper[2563]: W0907 12:02:03.900571    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poddfdfb25e_f951_4d2c_87b0_26ee1ec10310.slice/crio-afb2ecfcedf2cb07b719245dcebf5510caa91f6aad30a647eb3721a98b8df43a WatchSource:0}: Error finding container afb2ecfcedf2cb07b719245dcebf5510caa91f6aad30a647eb3721a98b8df43a: Status 404 returned error can't find the container with id afb2ecfcedf2cb07b719245dcebf5510caa91f6aad30a647eb3721a98b8df43a
Sep 07 12:02:04.588655 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:02:04.588623    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-7n5g7" event={"ID":"dfdfb25e-f951-4d2c-87b0-26ee1ec10310","Type":"ContainerStarted","Data":"f428a08f49239e57f7e2e6bf6d797a80a2e3612ae12aa65a15a0d473bb45631d"}
Sep 07 12:02:04.589051 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:02:04.588663    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-7n5g7" event={"ID":"dfdfb25e-f951-4d2c-87b0-26ee1ec10310","Type":"ContainerStarted","Data":"afb2ecfcedf2cb07b719245dcebf5510caa91f6aad30a647eb3721a98b8df43a"}
Sep 07 12:02:04.605571 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:02:04.605524    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-7n5g7" podStartSLOduration=1.433366476 podStartE2EDuration="1.605510162s" totalImagesPullingTime="172.143686ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 12:02:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 12:02:04.603439094 +0000 UTC m=+896.122170899" watchObservedRunningTime="2026-09-07 12:02:04.605510162 +0000 UTC m=+896.124241974"
Sep 07 12:02:05.305991 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:02:05.305954    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-p4l4w"]
Sep 07 12:02:06.627128 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:02:06.627095    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-nj42c"
Sep 07 12:02:06.627560 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:02:06.627147    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-nj42c"
Sep 07 12:02:06.639601 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:02:06.639577    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-nj42c"
Sep 07 12:02:07.613643 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:02:07.613614    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-nj42c"
Sep 07 12:02:09.133411 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:02:09.133322    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-4nfbc_b34ca85d-ac51-4724-a508-1026106b3f3a/console-operator/2.log"
Sep 07 12:02:09.147668 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:02:09.133560    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-4nfbc_b34ca85d-ac51-4724-a508-1026106b3f3a/console-operator/2.log"
Sep 07 12:02:13.630617 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:02:13.630587    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-7n5g7"
Sep 07 12:02:13.631008 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:02:13.630628    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-7n5g7"
Sep 07 12:02:13.643141 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:02:13.643114    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-7n5g7"
Sep 07 12:02:14.647002 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:02:14.646968    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-7n5g7"
Sep 07 12:03:28.896355 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:03:28.896315    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-55cc594f49-5tr5s"]
Sep 07 12:03:28.899746 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:03:28.899728    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-55cc594f49-5tr5s"
Sep 07 12:03:28.901836 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:03:28.901810    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-gfsh7\""
Sep 07 12:03:28.901836 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:03:28.901819    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\""
Sep 07 12:03:28.907301 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:03:28.907277    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-55cc594f49-5tr5s"]
Sep 07 12:03:29.021372 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:03:29.021345    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/8c6c9a5e-f913-4bef-adbc-df934f10b618-tls-cert\") pod \"authorino-55cc594f49-5tr5s\" (UID: \"8c6c9a5e-f913-4bef-adbc-df934f10b618\") " pod="kuadrant-system/authorino-55cc594f49-5tr5s"
Sep 07 12:03:29.021535 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:03:29.021435    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9xmgz\" (UniqueName: \"kubernetes.io/projected/8c6c9a5e-f913-4bef-adbc-df934f10b618-kube-api-access-9xmgz\") pod \"authorino-55cc594f49-5tr5s\" (UID: \"8c6c9a5e-f913-4bef-adbc-df934f10b618\") " pod="kuadrant-system/authorino-55cc594f49-5tr5s"
Sep 07 12:03:29.121811 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:03:29.121785    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9xmgz\" (UniqueName: \"kubernetes.io/projected/8c6c9a5e-f913-4bef-adbc-df934f10b618-kube-api-access-9xmgz\") pod \"authorino-55cc594f49-5tr5s\" (UID: \"8c6c9a5e-f913-4bef-adbc-df934f10b618\") " pod="kuadrant-system/authorino-55cc594f49-5tr5s"
Sep 07 12:03:29.121934 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:03:29.121864    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/8c6c9a5e-f913-4bef-adbc-df934f10b618-tls-cert\") pod \"authorino-55cc594f49-5tr5s\" (UID: \"8c6c9a5e-f913-4bef-adbc-df934f10b618\") " pod="kuadrant-system/authorino-55cc594f49-5tr5s"
Sep 07 12:03:29.124482 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:03:29.124455    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/8c6c9a5e-f913-4bef-adbc-df934f10b618-tls-cert\") pod \"authorino-55cc594f49-5tr5s\" (UID: \"8c6c9a5e-f913-4bef-adbc-df934f10b618\") " pod="kuadrant-system/authorino-55cc594f49-5tr5s"
Sep 07 12:03:29.128825 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:03:29.128800    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9xmgz\" (UniqueName: \"kubernetes.io/projected/8c6c9a5e-f913-4bef-adbc-df934f10b618-kube-api-access-9xmgz\") pod \"authorino-55cc594f49-5tr5s\" (UID: \"8c6c9a5e-f913-4bef-adbc-df934f10b618\") " pod="kuadrant-system/authorino-55cc594f49-5tr5s"
Sep 07 12:03:29.209543 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:03:29.209522    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-55cc594f49-5tr5s"
Sep 07 12:03:29.477085 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:03:29.477061    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-55cc594f49-5tr5s"]
Sep 07 12:03:29.479529 ip-10-0-135-173 kubenswrapper[2563]: W0907 12:03:29.479497    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8c6c9a5e_f913_4bef_adbc_df934f10b618.slice/crio-6b1ddbe3598711510335ddace10ef0dd42a69b33ee9c3ebc4b416495acfed355 WatchSource:0}: Error finding container 6b1ddbe3598711510335ddace10ef0dd42a69b33ee9c3ebc4b416495acfed355: Status 404 returned error can't find the container with id 6b1ddbe3598711510335ddace10ef0dd42a69b33ee9c3ebc4b416495acfed355
Sep 07 12:03:29.480803 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:03:29.480784    2563 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 07 12:03:29.993595 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:03:29.993558    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-55cc594f49-5tr5s" event={"ID":"8c6c9a5e-f913-4bef-adbc-df934f10b618","Type":"ContainerStarted","Data":"6b1ddbe3598711510335ddace10ef0dd42a69b33ee9c3ebc4b416495acfed355"}
Sep 07 12:03:30.890721 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:03:30.890684    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:03:30.989877 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:03:30.989834    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:03:31.000029 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:03:31.000000    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-55cc594f49-5tr5s" event={"ID":"8c6c9a5e-f913-4bef-adbc-df934f10b618","Type":"ContainerStarted","Data":"fd33b36db29a3362500585cc6af11bb33b68364e98a781b376ec03fb837ce0b2"}
Sep 07 12:03:31.018237 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:03:31.018188    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-55cc594f49-5tr5s" podStartSLOduration=2.3931884979999998 podStartE2EDuration="3.018171462s" totalImagesPullingTime="624.982964ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 12:03:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 12:03:31.016307706 +0000 UTC m=+982.535039520" watchObservedRunningTime="2026-09-07 12:03:31.018171462 +0000 UTC m=+982.536903309"
Sep 07 12:03:31.539663 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:03:31.539623    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:03:31.641435 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:03:31.641386    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:03:32.636478 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:03:32.636442    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:03:32.736089 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:03:32.736059    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:03:48.008610 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:03:48.008561    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:03:48.128222 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:03:48.128192    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:03:48.336540 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:03:48.336453    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:03:48.436293 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:03:48.436258    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:03:49.454760 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:03:49.454722    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:03:49.554372 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:03:49.554334    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:03:49.924389 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:03:49.924298    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:03:51.664438 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:03:51.664378    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:03:51.766581 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:03:51.766539    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:03:51.810377 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:03:51.810338    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-p4l4w"]
Sep 07 12:03:52.820268 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:03:52.820231    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:03:52.885045 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:03:52.885009    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:03:53.056000 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:03:53.055968    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:03:53.430530 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:03:53.430492    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:03:53.528316 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:03:53.528277    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:03:54.335597 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:03:54.335568    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7506-140546698491440"]
Sep 07 12:03:54.339139 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:03:54.339123    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7506-140546698491440"
Sep 07 12:03:54.340999 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:03:54.340978    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-c7bf7\""
Sep 07 12:03:54.341680 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:03:54.341180    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8zd29\" (UniqueName: \"kubernetes.io/projected/f18d86ff-b46e-4449-bedd-38a6673c34d9-kube-api-access-8zd29\") pod \"test-curl-7506-140546698491440\" (UID: \"f18d86ff-b46e-4449-bedd-38a6673c34d9\") " pod="opendatahub/test-curl-7506-140546698491440"
Sep 07 12:03:54.342380 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:03:54.342354    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7506-140546698491440"]
Sep 07 12:03:54.442209 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:03:54.442177    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8zd29\" (UniqueName: \"kubernetes.io/projected/f18d86ff-b46e-4449-bedd-38a6673c34d9-kube-api-access-8zd29\") pod \"test-curl-7506-140546698491440\" (UID: \"f18d86ff-b46e-4449-bedd-38a6673c34d9\") " pod="opendatahub/test-curl-7506-140546698491440"
Sep 07 12:03:54.448935 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:03:54.448917    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8zd29\" (UniqueName: \"kubernetes.io/projected/f18d86ff-b46e-4449-bedd-38a6673c34d9-kube-api-access-8zd29\") pod \"test-curl-7506-140546698491440\" (UID: \"f18d86ff-b46e-4449-bedd-38a6673c34d9\") " pod="opendatahub/test-curl-7506-140546698491440"
Sep 07 12:03:54.648971 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:03:54.648900    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7506-140546698491440"
Sep 07 12:03:54.917472 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:03:54.917289    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7506-140546698491440"]
Sep 07 12:03:54.920453 ip-10-0-135-173 kubenswrapper[2563]: W0907 12:03:54.920411    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf18d86ff_b46e_4449_bedd_38a6673c34d9.slice/crio-2bb5bbceed765af133ffe8ba693af8f52079ac5669432475a9f1b19f92a3bfbe WatchSource:0}: Error finding container 2bb5bbceed765af133ffe8ba693af8f52079ac5669432475a9f1b19f92a3bfbe: Status 404 returned error can't find the container with id 2bb5bbceed765af133ffe8ba693af8f52079ac5669432475a9f1b19f92a3bfbe
Sep 07 12:03:55.121977 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:03:55.121939    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7506-140546698491440" event={"ID":"f18d86ff-b46e-4449-bedd-38a6673c34d9","Type":"ContainerStarted","Data":"2bb5bbceed765af133ffe8ba693af8f52079ac5669432475a9f1b19f92a3bfbe"}
Sep 07 12:03:55.900767 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:03:55.900698    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:03:56.000902 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:03:56.000867    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:03:56.133087 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:03:56.133047    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:03:56.432770 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:03:56.432731    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:03:56.640195 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:03:56.640157    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:03:57.135423 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:03:57.135389    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7506-140546698491440" event={"ID":"f18d86ff-b46e-4449-bedd-38a6673c34d9","Type":"ContainerStarted","Data":"a728e493ed4ed66bd8d047db3a715d0f5dd7e007d51879724b6c2c6d13aa2435"}
Sep 07 12:03:57.147437 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:03:57.147351    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7506-140546698491440" podStartSLOduration=1.0230471780000001 podStartE2EDuration="3.147332149s" totalImagesPullingTime="2.124284971s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 12:03:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 12:03:57.147042492 +0000 UTC m=+1008.665774308" watchObservedRunningTime="2026-09-07 12:03:57.147332149 +0000 UTC m=+1008.666063962"
Sep 07 12:03:57.232280 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:03:57.232249    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7506-140546698491440_f18d86ff-b46e-4449-bedd-38a6673c34d9/test-curl-7506-140546698491440/0.log"
Sep 07 12:03:58.141787 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:03:58.141757    2563 generic.go:356] "Generic (PLEG): container finished" podID="f18d86ff-b46e-4449-bedd-38a6673c34d9" containerID="a728e493ed4ed66bd8d047db3a715d0f5dd7e007d51879724b6c2c6d13aa2435" exitCode=0
Sep 07 12:03:58.142140 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:03:58.141847    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7506-140546698491440" event={"ID":"f18d86ff-b46e-4449-bedd-38a6673c34d9","Type":"ContainerDied","Data":"a728e493ed4ed66bd8d047db3a715d0f5dd7e007d51879724b6c2c6d13aa2435"}
Sep 07 12:03:59.153107 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:03:59.153066    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:03:59.251838 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:03:59.251793    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:03:59.285967 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:03:59.285934    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7506-140546698491440"
Sep 07 12:03:59.315604 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:03:59.315575    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7506-140546698491440"]
Sep 07 12:03:59.319267 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:03:59.319244    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7506-140546698491440"]
Sep 07 12:03:59.387094 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:03:59.387067    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8zd29\" (UniqueName: \"kubernetes.io/projected/f18d86ff-b46e-4449-bedd-38a6673c34d9-kube-api-access-8zd29\") pod \"f18d86ff-b46e-4449-bedd-38a6673c34d9\" (UID: \"f18d86ff-b46e-4449-bedd-38a6673c34d9\") "
Sep 07 12:03:59.389147 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:03:59.389121    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f18d86ff-b46e-4449-bedd-38a6673c34d9-kube-api-access-8zd29" (OuterVolumeSpecName: "kube-api-access-8zd29") pod "f18d86ff-b46e-4449-bedd-38a6673c34d9" (UID: "f18d86ff-b46e-4449-bedd-38a6673c34d9"). InnerVolumeSpecName "kube-api-access-8zd29". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 12:03:59.488612 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:03:59.488581    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8zd29\" (UniqueName: \"kubernetes.io/projected/f18d86ff-b46e-4449-bedd-38a6673c34d9-kube-api-access-8zd29\") on node \"ip-10-0-135-173.ec2.internal\" DevicePath \"\""
Sep 07 12:03:59.839396 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:03:59.839315    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:04:00.146948 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:04:00.146859    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:04:00.153133 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:04:00.153111    2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2bb5bbceed765af133ffe8ba693af8f52079ac5669432475a9f1b19f92a3bfbe"
Sep 07 12:04:00.153133 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:04:00.153116    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7506-140546698491440"
Sep 07 12:04:00.162329 ip-10-0-135-173 kubenswrapper[2563]: E0907 12:04:00.162298    2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7506-140546698491440\" is forbidden: User \"system:node:ip-10-0-135-173.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-135-173.ec2.internal' and this object" podUID="f18d86ff-b46e-4449-bedd-38a6673c34d9" pod="opendatahub/test-curl-7506-140546698491440"
Sep 07 12:04:00.351155 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:04:00.351116    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:04:00.452997 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:04:00.452961    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:04:00.550491 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:04:00.550461    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:04:00.654709 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:04:00.654661    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:04:00.832747 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:04:00.832659    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:04:00.931742 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:04:00.931704    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:04:01.039875 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:04:01.039838    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f18d86ff-b46e-4449-bedd-38a6673c34d9" path="/var/lib/kubelet/pods/f18d86ff-b46e-4449-bedd-38a6673c34d9/volumes"
Sep 07 12:04:03.681443 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:04:03.681397    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:04:03.780357 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:04:03.780324    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:04:04.224038 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:04:04.224004    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-p4l4w"]
Sep 07 12:04:04.816908 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:04:04.816865    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-ebf845-5cd8fdb5bc-nbwrx"]
Sep 07 12:04:04.817528 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:04:04.817505    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f18d86ff-b46e-4449-bedd-38a6673c34d9" containerName="test-curl-7506-140546698491440"
Sep 07 12:04:04.817687 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:04:04.817673    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="f18d86ff-b46e-4449-bedd-38a6673c34d9" containerName="test-curl-7506-140546698491440"
Sep 07 12:04:04.817888 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:04:04.817873    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="f18d86ff-b46e-4449-bedd-38a6673c34d9" containerName="test-curl-7506-140546698491440"
Sep 07 12:04:04.822647 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:04:04.822620    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-autores-ebf845-5cd8fdb5bc-nbwrx"
Sep 07 12:04:04.824750 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:04:04.824511    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 07 12:04:04.824750 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:04:04.824530    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-autores-ebf845-serving-cert\""
Sep 07 12:04:04.824750 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:04:04.824559    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Sep 07 12:04:04.825016 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:04:04.824873    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-p4vz9\""
Sep 07 12:04:04.825016 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:04:04.824897    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Sep 07 12:04:04.834030 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:04:04.833877    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-ebf845-5cd8fdb5bc-nbwrx"]
Sep 07 12:04:04.857614 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:04:04.857585    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:04:04.934535 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:04:04.934481    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/673c6979-7dd6-4815-b8f3-e33f906ed5da-openshift-service-ca\") pod \"maas-api-e2e-autores-ebf845-5cd8fdb5bc-nbwrx\" (UID: \"673c6979-7dd6-4815-b8f3-e33f906ed5da\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-ebf845-5cd8fdb5bc-nbwrx"
Sep 07 12:04:04.934535 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:04:04.934534    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zh4pm\" (UniqueName: \"kubernetes.io/projected/673c6979-7dd6-4815-b8f3-e33f906ed5da-kube-api-access-zh4pm\") pod \"maas-api-e2e-autores-ebf845-5cd8fdb5bc-nbwrx\" (UID: \"673c6979-7dd6-4815-b8f3-e33f906ed5da\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-ebf845-5cd8fdb5bc-nbwrx"
Sep 07 12:04:04.934749 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:04:04.934604    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/673c6979-7dd6-4815-b8f3-e33f906ed5da-maas-api-tls\") pod \"maas-api-e2e-autores-ebf845-5cd8fdb5bc-nbwrx\" (UID: \"673c6979-7dd6-4815-b8f3-e33f906ed5da\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-ebf845-5cd8fdb5bc-nbwrx"
Sep 07 12:04:04.934749 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:04:04.934677    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/673c6979-7dd6-4815-b8f3-e33f906ed5da-metrics-certs\") pod \"maas-api-e2e-autores-ebf845-5cd8fdb5bc-nbwrx\" (UID: \"673c6979-7dd6-4815-b8f3-e33f906ed5da\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-ebf845-5cd8fdb5bc-nbwrx"
Sep 07 12:04:04.958341 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:04:04.958305    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:04:05.036843 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:04:05.036801    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/673c6979-7dd6-4815-b8f3-e33f906ed5da-metrics-certs\") pod \"maas-api-e2e-autores-ebf845-5cd8fdb5bc-nbwrx\" (UID: \"673c6979-7dd6-4815-b8f3-e33f906ed5da\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-ebf845-5cd8fdb5bc-nbwrx"
Sep 07 12:04:05.037171 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:04:05.037152    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/673c6979-7dd6-4815-b8f3-e33f906ed5da-openshift-service-ca\") pod \"maas-api-e2e-autores-ebf845-5cd8fdb5bc-nbwrx\" (UID: \"673c6979-7dd6-4815-b8f3-e33f906ed5da\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-ebf845-5cd8fdb5bc-nbwrx"
Sep 07 12:04:05.037295 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:04:05.037282    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zh4pm\" (UniqueName: \"kubernetes.io/projected/673c6979-7dd6-4815-b8f3-e33f906ed5da-kube-api-access-zh4pm\") pod \"maas-api-e2e-autores-ebf845-5cd8fdb5bc-nbwrx\" (UID: \"673c6979-7dd6-4815-b8f3-e33f906ed5da\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-ebf845-5cd8fdb5bc-nbwrx"
Sep 07 12:04:05.037407 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:04:05.037394    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/673c6979-7dd6-4815-b8f3-e33f906ed5da-maas-api-tls\") pod \"maas-api-e2e-autores-ebf845-5cd8fdb5bc-nbwrx\" (UID: \"673c6979-7dd6-4815-b8f3-e33f906ed5da\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-ebf845-5cd8fdb5bc-nbwrx"
Sep 07 12:04:05.037700 ip-10-0-135-173 kubenswrapper[2563]: E0907 12:04:05.037676    2563 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-autores-ebf845-serving-cert: secret "maas-api-e2e-autores-ebf845-serving-cert" not found
Sep 07 12:04:05.037858 ip-10-0-135-173 kubenswrapper[2563]: E0907 12:04:05.037848    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/673c6979-7dd6-4815-b8f3-e33f906ed5da-maas-api-tls podName:673c6979-7dd6-4815-b8f3-e33f906ed5da nodeName:}" failed. No retries permitted until 2026-09-07 12:04:05.537826923 +0000 UTC m=+1017.056558720 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/673c6979-7dd6-4815-b8f3-e33f906ed5da-maas-api-tls") pod "maas-api-e2e-autores-ebf845-5cd8fdb5bc-nbwrx" (UID: "673c6979-7dd6-4815-b8f3-e33f906ed5da") : secret "maas-api-e2e-autores-ebf845-serving-cert" not found
Sep 07 12:04:05.041476 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:04:05.041398    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/673c6979-7dd6-4815-b8f3-e33f906ed5da-openshift-service-ca\") pod \"maas-api-e2e-autores-ebf845-5cd8fdb5bc-nbwrx\" (UID: \"673c6979-7dd6-4815-b8f3-e33f906ed5da\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-ebf845-5cd8fdb5bc-nbwrx"
Sep 07 12:04:05.044715 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:04:05.044681    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/673c6979-7dd6-4815-b8f3-e33f906ed5da-metrics-certs\") pod \"maas-api-e2e-autores-ebf845-5cd8fdb5bc-nbwrx\" (UID: \"673c6979-7dd6-4815-b8f3-e33f906ed5da\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-ebf845-5cd8fdb5bc-nbwrx"
Sep 07 12:04:05.048012 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:04:05.047989    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zh4pm\" (UniqueName: \"kubernetes.io/projected/673c6979-7dd6-4815-b8f3-e33f906ed5da-kube-api-access-zh4pm\") pod \"maas-api-e2e-autores-ebf845-5cd8fdb5bc-nbwrx\" (UID: \"673c6979-7dd6-4815-b8f3-e33f906ed5da\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-ebf845-5cd8fdb5bc-nbwrx"
Sep 07 12:04:05.278696 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:04:05.278647    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:04:05.463096 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:04:05.463062    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:04:05.542077 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:04:05.541997    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/673c6979-7dd6-4815-b8f3-e33f906ed5da-maas-api-tls\") pod \"maas-api-e2e-autores-ebf845-5cd8fdb5bc-nbwrx\" (UID: \"673c6979-7dd6-4815-b8f3-e33f906ed5da\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-ebf845-5cd8fdb5bc-nbwrx"
Sep 07 12:04:05.544178 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:04:05.544157    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/673c6979-7dd6-4815-b8f3-e33f906ed5da-maas-api-tls\") pod \"maas-api-e2e-autores-ebf845-5cd8fdb5bc-nbwrx\" (UID: \"673c6979-7dd6-4815-b8f3-e33f906ed5da\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-ebf845-5cd8fdb5bc-nbwrx"
Sep 07 12:04:05.569234 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:04:05.569203    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:04:05.738506 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:04:05.738473    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-autores-ebf845-5cd8fdb5bc-nbwrx"
Sep 07 12:04:06.012624 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:04:06.012598    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-ebf845-5cd8fdb5bc-nbwrx"]
Sep 07 12:04:06.015385 ip-10-0-135-173 kubenswrapper[2563]: W0907 12:04:06.015358    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod673c6979_7dd6_4815_b8f3_e33f906ed5da.slice/crio-dc70ea68a2304213a4a260a779662cf84f9d8fe10059e5fc1024050d0e8ab818 WatchSource:0}: Error finding container dc70ea68a2304213a4a260a779662cf84f9d8fe10059e5fc1024050d0e8ab818: Status 404 returned error can't find the container with id dc70ea68a2304213a4a260a779662cf84f9d8fe10059e5fc1024050d0e8ab818
Sep 07 12:04:06.182590 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:04:06.182545    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-autores-ebf845-5cd8fdb5bc-nbwrx" event={"ID":"673c6979-7dd6-4815-b8f3-e33f906ed5da","Type":"ContainerStarted","Data":"dc70ea68a2304213a4a260a779662cf84f9d8fe10059e5fc1024050d0e8ab818"}
Sep 07 12:04:06.871316 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:04:06.871279    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:04:07.022028 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:04:07.021981    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:04:07.126345 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:04:07.126231    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:04:07.223017 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:04:07.222983    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:04:08.194229 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:04:08.194199    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-autores-ebf845-5cd8fdb5bc-nbwrx" event={"ID":"673c6979-7dd6-4815-b8f3-e33f906ed5da","Type":"ContainerStarted","Data":"0fd9c4d19e8ff19ffabd9950c5aa0ec5f16276965026c6d7d953562fd1d87b87"}
Sep 07 12:04:08.194639 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:04:08.194257    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-autores-ebf845-5cd8fdb5bc-nbwrx"
Sep 07 12:04:08.209467 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:04:08.209402    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-autores-ebf845-5cd8fdb5bc-nbwrx" podStartSLOduration=2.819945148 podStartE2EDuration="4.209386115s" totalImagesPullingTime="1.389440967s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 12:04:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 12:04:08.208728262 +0000 UTC m=+1019.727460074" watchObservedRunningTime="2026-09-07 12:04:08.209386115 +0000 UTC m=+1019.728117928"
Sep 07 12:04:09.227873 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:04:09.227843    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:04:10.346432 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:04:10.346392    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:04:10.400253 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:04:10.400220    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:04:10.974441 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:04:10.974396    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:04:11.075977 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:04:11.075944    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:04:11.427275 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:04:11.427182    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:04:13.946874 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:04:13.946838    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:04:14.205058 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:04:14.204975    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-autores-ebf845-5cd8fdb5bc-nbwrx"
Sep 07 12:04:15.893112 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:04:15.893064    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:04:15.951800 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:04:15.951761    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:04:16.026524 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:04:16.026493    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:04:16.121470 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:04:16.121399    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:04:16.221835 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:04:16.221796    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:04:16.585269 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:04:16.585194    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:04:16.891493 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:04:16.891397    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:04:17.160391 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:04:17.160314    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-af50c7-openshift-default-67788bbd-v68lr"]
Sep 07 12:04:17.163376 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:04:17.163356    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-af50c7-openshift-default-67788bbd-v68lr"
Sep 07 12:04:17.165296 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:04:17.165274    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-af50c7-openshift-default-dockercfg-scpgb\""
Sep 07 12:04:17.165522 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:04:17.165501    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Sep 07 12:04:17.175302 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:04:17.175283    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-af50c7-openshift-default-67788bbd-v68lr"]
Sep 07 12:04:17.246123 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:04:17.246098    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bc21582c-d7f9-48e4-9309-ffeefa3f7f03-istio-podinfo\") pod \"e2e-inf-b-af50c7-openshift-default-67788bbd-v68lr\" (UID: \"bc21582c-d7f9-48e4-9309-ffeefa3f7f03\") " pod="openshift-ingress/e2e-inf-b-af50c7-openshift-default-67788bbd-v68lr"
Sep 07 12:04:17.246256 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:04:17.246155    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bc21582c-d7f9-48e4-9309-ffeefa3f7f03-workload-certs\") pod \"e2e-inf-b-af50c7-openshift-default-67788bbd-v68lr\" (UID: \"bc21582c-d7f9-48e4-9309-ffeefa3f7f03\") " pod="openshift-ingress/e2e-inf-b-af50c7-openshift-default-67788bbd-v68lr"
Sep 07 12:04:17.246298 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:04:17.246259    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bc21582c-d7f9-48e4-9309-ffeefa3f7f03-istio-data\") pod \"e2e-inf-b-af50c7-openshift-default-67788bbd-v68lr\" (UID: \"bc21582c-d7f9-48e4-9309-ffeefa3f7f03\") " pod="openshift-ingress/e2e-inf-b-af50c7-openshift-default-67788bbd-v68lr"
Sep 07 12:04:17.246341 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:04:17.246300    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bc21582c-d7f9-48e4-9309-ffeefa3f7f03-credential-socket\") pod \"e2e-inf-b-af50c7-openshift-default-67788bbd-v68lr\" (UID: \"bc21582c-d7f9-48e4-9309-ffeefa3f7f03\") " pod="openshift-ingress/e2e-inf-b-af50c7-openshift-default-67788bbd-v68lr"
Sep 07 12:04:17.246341 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:04:17.246329    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2tgdf\" (UniqueName: \"kubernetes.io/projected/bc21582c-d7f9-48e4-9309-ffeefa3f7f03-kube-api-access-2tgdf\") pod \"e2e-inf-b-af50c7-openshift-default-67788bbd-v68lr\" (UID: \"bc21582c-d7f9-48e4-9309-ffeefa3f7f03\") " pod="openshift-ingress/e2e-inf-b-af50c7-openshift-default-67788bbd-v68lr"
Sep 07 12:04:17.246407 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:04:17.246358    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bc21582c-d7f9-48e4-9309-ffeefa3f7f03-istio-token\") pod \"e2e-inf-b-af50c7-openshift-default-67788bbd-v68lr\" (UID: \"bc21582c-d7f9-48e4-9309-ffeefa3f7f03\") " pod="openshift-ingress/e2e-inf-b-af50c7-openshift-default-67788bbd-v68lr"
Sep 07 12:04:17.246407 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:04:17.246381    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bc21582c-d7f9-48e4-9309-ffeefa3f7f03-istiod-ca-cert\") pod \"e2e-inf-b-af50c7-openshift-default-67788bbd-v68lr\" (UID: \"bc21582c-d7f9-48e4-9309-ffeefa3f7f03\") " pod="openshift-ingress/e2e-inf-b-af50c7-openshift-default-67788bbd-v68lr"
Sep 07 12:04:17.246509 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:04:17.246447    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bc21582c-d7f9-48e4-9309-ffeefa3f7f03-workload-socket\") pod \"e2e-inf-b-af50c7-openshift-default-67788bbd-v68lr\" (UID: \"bc21582c-d7f9-48e4-9309-ffeefa3f7f03\") " pod="openshift-ingress/e2e-inf-b-af50c7-openshift-default-67788bbd-v68lr"
Sep 07 12:04:17.246509 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:04:17.246489    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bc21582c-d7f9-48e4-9309-ffeefa3f7f03-istio-envoy\") pod \"e2e-inf-b-af50c7-openshift-default-67788bbd-v68lr\" (UID: \"bc21582c-d7f9-48e4-9309-ffeefa3f7f03\") " pod="openshift-ingress/e2e-inf-b-af50c7-openshift-default-67788bbd-v68lr"
Sep 07 12:04:17.260048 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:04:17.260018    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:04:17.347721 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:04:17.347687    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bc21582c-d7f9-48e4-9309-ffeefa3f7f03-credential-socket\") pod \"e2e-inf-b-af50c7-openshift-default-67788bbd-v68lr\" (UID: \"bc21582c-d7f9-48e4-9309-ffeefa3f7f03\") " pod="openshift-ingress/e2e-inf-b-af50c7-openshift-default-67788bbd-v68lr"
Sep 07 12:04:17.347863 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:04:17.347731    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2tgdf\" (UniqueName: \"kubernetes.io/projected/bc21582c-d7f9-48e4-9309-ffeefa3f7f03-kube-api-access-2tgdf\") pod \"e2e-inf-b-af50c7-openshift-default-67788bbd-v68lr\" (UID: \"bc21582c-d7f9-48e4-9309-ffeefa3f7f03\") " pod="openshift-ingress/e2e-inf-b-af50c7-openshift-default-67788bbd-v68lr"
Sep 07 12:04:17.347863 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:04:17.347762    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bc21582c-d7f9-48e4-9309-ffeefa3f7f03-istio-token\") pod \"e2e-inf-b-af50c7-openshift-default-67788bbd-v68lr\" (UID: \"bc21582c-d7f9-48e4-9309-ffeefa3f7f03\") " pod="openshift-ingress/e2e-inf-b-af50c7-openshift-default-67788bbd-v68lr"
Sep 07 12:04:17.347863 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:04:17.347786    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bc21582c-d7f9-48e4-9309-ffeefa3f7f03-istiod-ca-cert\") pod \"e2e-inf-b-af50c7-openshift-default-67788bbd-v68lr\" (UID: \"bc21582c-d7f9-48e4-9309-ffeefa3f7f03\") " pod="openshift-ingress/e2e-inf-b-af50c7-openshift-default-67788bbd-v68lr"
Sep 07 12:04:17.348021 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:04:17.347951    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bc21582c-d7f9-48e4-9309-ffeefa3f7f03-workload-socket\") pod \"e2e-inf-b-af50c7-openshift-default-67788bbd-v68lr\" (UID: \"bc21582c-d7f9-48e4-9309-ffeefa3f7f03\") " pod="openshift-ingress/e2e-inf-b-af50c7-openshift-default-67788bbd-v68lr"
Sep 07 12:04:17.348076 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:04:17.348016    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bc21582c-d7f9-48e4-9309-ffeefa3f7f03-istio-envoy\") pod \"e2e-inf-b-af50c7-openshift-default-67788bbd-v68lr\" (UID: \"bc21582c-d7f9-48e4-9309-ffeefa3f7f03\") " pod="openshift-ingress/e2e-inf-b-af50c7-openshift-default-67788bbd-v68lr"
Sep 07 12:04:17.348126 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:04:17.348090    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bc21582c-d7f9-48e4-9309-ffeefa3f7f03-istio-podinfo\") pod \"e2e-inf-b-af50c7-openshift-default-67788bbd-v68lr\" (UID: \"bc21582c-d7f9-48e4-9309-ffeefa3f7f03\") " pod="openshift-ingress/e2e-inf-b-af50c7-openshift-default-67788bbd-v68lr"
Sep 07 12:04:17.348126 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:04:17.348112    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bc21582c-d7f9-48e4-9309-ffeefa3f7f03-credential-socket\") pod \"e2e-inf-b-af50c7-openshift-default-67788bbd-v68lr\" (UID: \"bc21582c-d7f9-48e4-9309-ffeefa3f7f03\") " pod="openshift-ingress/e2e-inf-b-af50c7-openshift-default-67788bbd-v68lr"
Sep 07 12:04:17.348242 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:04:17.348220    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bc21582c-d7f9-48e4-9309-ffeefa3f7f03-workload-certs\") pod \"e2e-inf-b-af50c7-openshift-default-67788bbd-v68lr\" (UID: \"bc21582c-d7f9-48e4-9309-ffeefa3f7f03\") " pod="openshift-ingress/e2e-inf-b-af50c7-openshift-default-67788bbd-v68lr"
Sep 07 12:04:17.348376 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:04:17.348282    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bc21582c-d7f9-48e4-9309-ffeefa3f7f03-workload-socket\") pod \"e2e-inf-b-af50c7-openshift-default-67788bbd-v68lr\" (UID: \"bc21582c-d7f9-48e4-9309-ffeefa3f7f03\") " pod="openshift-ingress/e2e-inf-b-af50c7-openshift-default-67788bbd-v68lr"
Sep 07 12:04:17.348376 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:04:17.348287    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bc21582c-d7f9-48e4-9309-ffeefa3f7f03-istio-data\") pod \"e2e-inf-b-af50c7-openshift-default-67788bbd-v68lr\" (UID: \"bc21582c-d7f9-48e4-9309-ffeefa3f7f03\") " pod="openshift-ingress/e2e-inf-b-af50c7-openshift-default-67788bbd-v68lr"
Sep 07 12:04:17.348593 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:04:17.348571    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bc21582c-d7f9-48e4-9309-ffeefa3f7f03-istio-data\") pod \"e2e-inf-b-af50c7-openshift-default-67788bbd-v68lr\" (UID: \"bc21582c-d7f9-48e4-9309-ffeefa3f7f03\") " pod="openshift-ingress/e2e-inf-b-af50c7-openshift-default-67788bbd-v68lr"
Sep 07 12:04:17.348657 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:04:17.348594    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bc21582c-d7f9-48e4-9309-ffeefa3f7f03-workload-certs\") pod \"e2e-inf-b-af50c7-openshift-default-67788bbd-v68lr\" (UID: \"bc21582c-d7f9-48e4-9309-ffeefa3f7f03\") " pod="openshift-ingress/e2e-inf-b-af50c7-openshift-default-67788bbd-v68lr"
Sep 07 12:04:17.348657 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:04:17.348598    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bc21582c-d7f9-48e4-9309-ffeefa3f7f03-istiod-ca-cert\") pod \"e2e-inf-b-af50c7-openshift-default-67788bbd-v68lr\" (UID: \"bc21582c-d7f9-48e4-9309-ffeefa3f7f03\") " pod="openshift-ingress/e2e-inf-b-af50c7-openshift-default-67788bbd-v68lr"
Sep 07 12:04:17.350213 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:04:17.350194    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bc21582c-d7f9-48e4-9309-ffeefa3f7f03-istio-envoy\") pod \"e2e-inf-b-af50c7-openshift-default-67788bbd-v68lr\" (UID: \"bc21582c-d7f9-48e4-9309-ffeefa3f7f03\") " pod="openshift-ingress/e2e-inf-b-af50c7-openshift-default-67788bbd-v68lr"
Sep 07 12:04:17.350552 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:04:17.350532    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bc21582c-d7f9-48e4-9309-ffeefa3f7f03-istio-podinfo\") pod \"e2e-inf-b-af50c7-openshift-default-67788bbd-v68lr\" (UID: \"bc21582c-d7f9-48e4-9309-ffeefa3f7f03\") " pod="openshift-ingress/e2e-inf-b-af50c7-openshift-default-67788bbd-v68lr"
Sep 07 12:04:17.354491 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:04:17.354472    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bc21582c-d7f9-48e4-9309-ffeefa3f7f03-istio-token\") pod \"e2e-inf-b-af50c7-openshift-default-67788bbd-v68lr\" (UID: \"bc21582c-d7f9-48e4-9309-ffeefa3f7f03\") " pod="openshift-ingress/e2e-inf-b-af50c7-openshift-default-67788bbd-v68lr"
Sep 07 12:04:17.354683 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:04:17.354657    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2tgdf\" (UniqueName: \"kubernetes.io/projected/bc21582c-d7f9-48e4-9309-ffeefa3f7f03-kube-api-access-2tgdf\") pod \"e2e-inf-b-af50c7-openshift-default-67788bbd-v68lr\" (UID: \"bc21582c-d7f9-48e4-9309-ffeefa3f7f03\") " pod="openshift-ingress/e2e-inf-b-af50c7-openshift-default-67788bbd-v68lr"
Sep 07 12:04:17.475196 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:04:17.475172    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-af50c7-openshift-default-67788bbd-v68lr"
Sep 07 12:04:17.758935 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:04:17.758912    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-af50c7-openshift-default-67788bbd-v68lr"]
Sep 07 12:04:17.761395 ip-10-0-135-173 kubenswrapper[2563]: W0907 12:04:17.761368    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbc21582c_d7f9_48e4_9309_ffeefa3f7f03.slice/crio-c95e9f45f77fb1baf94b70a2d887d9065c91823db60e98d94d9a59cc726d620e WatchSource:0}: Error finding container c95e9f45f77fb1baf94b70a2d887d9065c91823db60e98d94d9a59cc726d620e: Status 404 returned error can't find the container with id c95e9f45f77fb1baf94b70a2d887d9065c91823db60e98d94d9a59cc726d620e
Sep 07 12:04:17.882375 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:04:17.882343    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:04:18.246129 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:04:18.246095    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-af50c7-openshift-default-67788bbd-v68lr" event={"ID":"bc21582c-d7f9-48e4-9309-ffeefa3f7f03","Type":"ContainerStarted","Data":"c95e9f45f77fb1baf94b70a2d887d9065c91823db60e98d94d9a59cc726d620e"}
Sep 07 12:04:20.328837 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:04:20.328799    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 07 12:04:20.329206 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:04:20.328888    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 07 12:04:20.329206 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:04:20.328932    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 07 12:04:20.397982 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:04:20.397956    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-p4l4w"]
Sep 07 12:04:21.264502 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:04:21.264462    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-af50c7-openshift-default-67788bbd-v68lr" event={"ID":"bc21582c-d7f9-48e4-9309-ffeefa3f7f03","Type":"ContainerStarted","Data":"0dd8e0198d29d62d2fe341baa5f3f2ab5a405b4c862095ecf99ff2f1791ba8c6"}
Sep 07 12:04:21.282103 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:04:21.282053    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-af50c7-openshift-default-67788bbd-v68lr" podStartSLOduration=1.717129428 podStartE2EDuration="4.282035142s" totalImagesPullingTime="2.564905714s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 12:04:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 12:04:21.280572177 +0000 UTC m=+1032.799303993" watchObservedRunningTime="2026-09-07 12:04:21.282035142 +0000 UTC m=+1032.800766949"
Sep 07 12:04:21.289189 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:04:21.289161    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:04:21.475630 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:04:21.475598    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-af50c7-openshift-default-67788bbd-v68lr"
Sep 07 12:04:21.480467 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:04:21.480445    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-af50c7-openshift-default-67788bbd-v68lr"
Sep 07 12:04:22.269329 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:04:22.269298    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-af50c7-openshift-default-67788bbd-v68lr"
Sep 07 12:04:22.270284 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:04:22.270265    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-af50c7-openshift-default-67788bbd-v68lr"
Sep 07 12:04:22.623371 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:04:22.623284    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:04:22.689602 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:04:22.689561    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:04:23.130626 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:04:23.130594    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:04:23.228335 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:04:23.228295    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:04:24.259409 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:04:24.259374    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-af50c7-96f78668b-6bxt4"]
Sep 07 12:04:24.263645 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:04:24.263623    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-af50c7-96f78668b-6bxt4"
Sep 07 12:04:24.265384 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:04:24.265360    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-b-af50c7-serving-cert\""
Sep 07 12:04:24.271654 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:04:24.271632    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-af50c7-96f78668b-6bxt4"]
Sep 07 12:04:24.296179 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:04:24.296150    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:04:24.365269 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:04:24.365239    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:04:24.416365 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:04:24.416339    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pnbqd\" (UniqueName: \"kubernetes.io/projected/a10d4b3d-8750-4ce2-baa6-68fade76984a-kube-api-access-pnbqd\") pod \"maas-api-e2e-inf-b-af50c7-96f78668b-6bxt4\" (UID: \"a10d4b3d-8750-4ce2-baa6-68fade76984a\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-af50c7-96f78668b-6bxt4"
Sep 07 12:04:24.416508 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:04:24.416408    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a10d4b3d-8750-4ce2-baa6-68fade76984a-maas-api-tls\") pod \"maas-api-e2e-inf-b-af50c7-96f78668b-6bxt4\" (UID: \"a10d4b3d-8750-4ce2-baa6-68fade76984a\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-af50c7-96f78668b-6bxt4"
Sep 07 12:04:24.416508 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:04:24.416468    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a10d4b3d-8750-4ce2-baa6-68fade76984a-metrics-certs\") pod \"maas-api-e2e-inf-b-af50c7-96f78668b-6bxt4\" (UID: \"a10d4b3d-8750-4ce2-baa6-68fade76984a\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-af50c7-96f78668b-6bxt4"
Sep 07 12:04:24.416603 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:04:24.416512    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a10d4b3d-8750-4ce2-baa6-68fade76984a-openshift-service-ca\") pod \"maas-api-e2e-inf-b-af50c7-96f78668b-6bxt4\" (UID: \"a10d4b3d-8750-4ce2-baa6-68fade76984a\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-af50c7-96f78668b-6bxt4"
Sep 07 12:04:24.517580 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:04:24.517496    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pnbqd\" (UniqueName: \"kubernetes.io/projected/a10d4b3d-8750-4ce2-baa6-68fade76984a-kube-api-access-pnbqd\") pod \"maas-api-e2e-inf-b-af50c7-96f78668b-6bxt4\" (UID: \"a10d4b3d-8750-4ce2-baa6-68fade76984a\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-af50c7-96f78668b-6bxt4"
Sep 07 12:04:24.517580 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:04:24.517546    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a10d4b3d-8750-4ce2-baa6-68fade76984a-maas-api-tls\") pod \"maas-api-e2e-inf-b-af50c7-96f78668b-6bxt4\" (UID: \"a10d4b3d-8750-4ce2-baa6-68fade76984a\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-af50c7-96f78668b-6bxt4"
Sep 07 12:04:24.517788 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:04:24.517598    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a10d4b3d-8750-4ce2-baa6-68fade76984a-metrics-certs\") pod \"maas-api-e2e-inf-b-af50c7-96f78668b-6bxt4\" (UID: \"a10d4b3d-8750-4ce2-baa6-68fade76984a\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-af50c7-96f78668b-6bxt4"
Sep 07 12:04:24.517788 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:04:24.517633    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a10d4b3d-8750-4ce2-baa6-68fade76984a-openshift-service-ca\") pod \"maas-api-e2e-inf-b-af50c7-96f78668b-6bxt4\" (UID: \"a10d4b3d-8750-4ce2-baa6-68fade76984a\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-af50c7-96f78668b-6bxt4"
Sep 07 12:04:24.518274 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:04:24.518249    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a10d4b3d-8750-4ce2-baa6-68fade76984a-openshift-service-ca\") pod \"maas-api-e2e-inf-b-af50c7-96f78668b-6bxt4\" (UID: \"a10d4b3d-8750-4ce2-baa6-68fade76984a\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-af50c7-96f78668b-6bxt4"
Sep 07 12:04:24.520054 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:04:24.520030    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a10d4b3d-8750-4ce2-baa6-68fade76984a-metrics-certs\") pod \"maas-api-e2e-inf-b-af50c7-96f78668b-6bxt4\" (UID: \"a10d4b3d-8750-4ce2-baa6-68fade76984a\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-af50c7-96f78668b-6bxt4"
Sep 07 12:04:24.520158 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:04:24.520135    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a10d4b3d-8750-4ce2-baa6-68fade76984a-maas-api-tls\") pod \"maas-api-e2e-inf-b-af50c7-96f78668b-6bxt4\" (UID: \"a10d4b3d-8750-4ce2-baa6-68fade76984a\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-af50c7-96f78668b-6bxt4"
Sep 07 12:04:24.524070 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:04:24.524052    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pnbqd\" (UniqueName: \"kubernetes.io/projected/a10d4b3d-8750-4ce2-baa6-68fade76984a-kube-api-access-pnbqd\") pod \"maas-api-e2e-inf-b-af50c7-96f78668b-6bxt4\" (UID: \"a10d4b3d-8750-4ce2-baa6-68fade76984a\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-af50c7-96f78668b-6bxt4"
Sep 07 12:04:24.578104 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:04:24.578081    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-af50c7-96f78668b-6bxt4"
Sep 07 12:04:24.809407 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:04:24.809318    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:04:24.847936 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:04:24.847912    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-af50c7-96f78668b-6bxt4"]
Sep 07 12:04:24.850598 ip-10-0-135-173 kubenswrapper[2563]: W0907 12:04:24.850565    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda10d4b3d_8750_4ce2_baa6_68fade76984a.slice/crio-abdefd08f60d3cdccdcaf4a8d4fd34f4ed514f4c4b4ede515c75722cd8af233b WatchSource:0}: Error finding container abdefd08f60d3cdccdcaf4a8d4fd34f4ed514f4c4b4ede515c75722cd8af233b: Status 404 returned error can't find the container with id abdefd08f60d3cdccdcaf4a8d4fd34f4ed514f4c4b4ede515c75722cd8af233b
Sep 07 12:04:24.904571 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:04:24.904540    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:04:25.286579 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:04:25.286543    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-af50c7-96f78668b-6bxt4" event={"ID":"a10d4b3d-8750-4ce2-baa6-68fade76984a","Type":"ContainerStarted","Data":"abdefd08f60d3cdccdcaf4a8d4fd34f4ed514f4c4b4ede515c75722cd8af233b"}
Sep 07 12:04:25.356825 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:04:25.356797    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:04:26.131247 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:04:26.131200    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:04:26.293275 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:04:26.293243    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-af50c7-96f78668b-6bxt4" event={"ID":"a10d4b3d-8750-4ce2-baa6-68fade76984a","Type":"ContainerStarted","Data":"93e675e2a6d3ff82eaf351483123e06067193c6641401f3a2d859fea4328700d"}
Sep 07 12:04:26.293742 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:04:26.293370    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-af50c7-96f78668b-6bxt4"
Sep 07 12:04:26.307324 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:04:26.307279    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-af50c7-96f78668b-6bxt4" podStartSLOduration=1.479552194 podStartE2EDuration="2.307263063s" totalImagesPullingTime="827.710869ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 12:04:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 12:04:26.306591145 +0000 UTC m=+1037.825322958" watchObservedRunningTime="2026-09-07 12:04:26.307263063 +0000 UTC m=+1037.825994873"
Sep 07 12:04:26.939872 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:04:26.939841    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:04:28.629946 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:04:28.629907    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:04:31.280003 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:04:31.279955    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:04:32.303873 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:04:32.303845    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-af50c7-96f78668b-6bxt4"
Sep 07 12:04:35.904616 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:04:35.904579    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-p4l4w"]
Sep 07 12:04:38.311570 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:04:38.311531    2563 patch_prober.go:28] interesting pod/e2e-inf-b-af50c7-openshift-default-67788bbd-v68lr container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.36:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Sep 07 12:04:38.311921 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:04:38.311644    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-b-af50c7-openshift-default-67788bbd-v68lr" podUID="bc21582c-d7f9-48e4-9309-ffeefa3f7f03" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.36:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Sep 07 12:04:39.365222 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:04:39.365188    2563 generic.go:356] "Generic (PLEG): container finished" podID="bc21582c-d7f9-48e4-9309-ffeefa3f7f03" containerID="0dd8e0198d29d62d2fe341baa5f3f2ab5a405b4c862095ecf99ff2f1791ba8c6" exitCode=0
Sep 07 12:04:39.365671 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:04:39.365257    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-af50c7-openshift-default-67788bbd-v68lr" event={"ID":"bc21582c-d7f9-48e4-9309-ffeefa3f7f03","Type":"ContainerDied","Data":"0dd8e0198d29d62d2fe341baa5f3f2ab5a405b4c862095ecf99ff2f1791ba8c6"}
Sep 07 12:04:39.365745 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:04:39.365709    2563 scope.go:117] "RemoveContainer" containerID="0dd8e0198d29d62d2fe341baa5f3f2ab5a405b4c862095ecf99ff2f1791ba8c6"
Sep 07 12:04:39.367061 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:04:39.367005    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 07 12:04:39.367133 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:04:39.367102    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 07 12:04:39.367177 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:04:39.367132    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 07 12:04:40.373036 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:04:40.373000    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-af50c7-openshift-default-67788bbd-v68lr" event={"ID":"bc21582c-d7f9-48e4-9309-ffeefa3f7f03","Type":"ContainerStarted","Data":"9b88ef6e84375dd2043a9bb56c56e164029c3d12fdb2faa60380a2075a9bf783"}
Sep 07 12:04:40.475624 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:04:40.475590    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-af50c7-openshift-default-67788bbd-v68lr"
Sep 07 12:04:40.475835 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:04:40.475799    2563 patch_prober.go:28] interesting pod/e2e-inf-b-af50c7-openshift-default-67788bbd-v68lr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.36:15021/healthz/ready\": dial tcp 10.133.0.36:15021: connect: connection refused" start-of-body=
Sep 07 12:04:40.475896 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:04:40.475849    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-af50c7-openshift-default-67788bbd-v68lr" podUID="bc21582c-d7f9-48e4-9309-ffeefa3f7f03" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.36:15021/healthz/ready\": dial tcp 10.133.0.36:15021: connect: connection refused"
Sep 07 12:04:40.610370 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:04:40.610330    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:04:41.342927 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:04:41.342890    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:04:41.476652 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:04:41.476614    2563 patch_prober.go:28] interesting pod/e2e-inf-b-af50c7-openshift-default-67788bbd-v68lr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.36:15021/healthz/ready\": dial tcp 10.133.0.36:15021: connect: connection refused" start-of-body=
Sep 07 12:04:41.477051 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:04:41.476688    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-af50c7-openshift-default-67788bbd-v68lr" podUID="bc21582c-d7f9-48e4-9309-ffeefa3f7f03" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.36:15021/healthz/ready\": dial tcp 10.133.0.36:15021: connect: connection refused"
Sep 07 12:04:42.330949 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:04:42.330908    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:04:42.417734 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:04:42.417694    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:04:42.476331 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:04:42.476295    2563 patch_prober.go:28] interesting pod/e2e-inf-b-af50c7-openshift-default-67788bbd-v68lr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.36:15021/healthz/ready\": dial tcp 10.133.0.36:15021: connect: connection refused" start-of-body=
Sep 07 12:04:42.476542 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:04:42.476353    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-af50c7-openshift-default-67788bbd-v68lr" podUID="bc21582c-d7f9-48e4-9309-ffeefa3f7f03" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.36:15021/healthz/ready\": dial tcp 10.133.0.36:15021: connect: connection refused"
Sep 07 12:04:42.614815 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:04:42.614720    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:04:43.148080 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:04:43.148045    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:04:43.194067 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:04:43.194035    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-conflict-a-6267dfa0-kserve-757756cc4f-6fd6v"]
Sep 07 12:04:43.196521 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:04:43.196502    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-conflict-a-6267dfa0-kserve-757756cc4f-6fd6v"
Sep 07 12:04:43.198505 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:04:43.198489    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-conflict-a-6267dfa0-kserve-self-signed-certs\""
Sep 07 12:04:43.205408 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:04:43.205385    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-conflict-a-6267dfa0-kserve-757756cc4f-6fd6v"]
Sep 07 12:04:43.296804 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:04:43.296767    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4f6rt\" (UniqueName: \"kubernetes.io/projected/9911c0b6-2b94-4b1f-829f-77fb622098ef-kube-api-access-4f6rt\") pod \"e2e-conflict-a-6267dfa0-kserve-757756cc4f-6fd6v\" (UID: \"9911c0b6-2b94-4b1f-829f-77fb622098ef\") " pod="llm/e2e-conflict-a-6267dfa0-kserve-757756cc4f-6fd6v"
Sep 07 12:04:43.296804 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:04:43.296808    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9911c0b6-2b94-4b1f-829f-77fb622098ef-tls-certs\") pod \"e2e-conflict-a-6267dfa0-kserve-757756cc4f-6fd6v\" (UID: \"9911c0b6-2b94-4b1f-829f-77fb622098ef\") " pod="llm/e2e-conflict-a-6267dfa0-kserve-757756cc4f-6fd6v"
Sep 07 12:04:43.297082 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:04:43.296926    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9911c0b6-2b94-4b1f-829f-77fb622098ef-tmp-dir\") pod \"e2e-conflict-a-6267dfa0-kserve-757756cc4f-6fd6v\" (UID: \"9911c0b6-2b94-4b1f-829f-77fb622098ef\") " pod="llm/e2e-conflict-a-6267dfa0-kserve-757756cc4f-6fd6v"
Sep 07 12:04:43.297082 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:04:43.296976    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9911c0b6-2b94-4b1f-829f-77fb622098ef-model-cache\") pod \"e2e-conflict-a-6267dfa0-kserve-757756cc4f-6fd6v\" (UID: \"9911c0b6-2b94-4b1f-829f-77fb622098ef\") " pod="llm/e2e-conflict-a-6267dfa0-kserve-757756cc4f-6fd6v"
Sep 07 12:04:43.297082 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:04:43.297001    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9911c0b6-2b94-4b1f-829f-77fb622098ef-dshm\") pod \"e2e-conflict-a-6267dfa0-kserve-757756cc4f-6fd6v\" (UID: \"9911c0b6-2b94-4b1f-829f-77fb622098ef\") " pod="llm/e2e-conflict-a-6267dfa0-kserve-757756cc4f-6fd6v"
Sep 07 12:04:43.297082 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:04:43.297044    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9911c0b6-2b94-4b1f-829f-77fb622098ef-home\") pod \"e2e-conflict-a-6267dfa0-kserve-757756cc4f-6fd6v\" (UID: \"9911c0b6-2b94-4b1f-829f-77fb622098ef\") " pod="llm/e2e-conflict-a-6267dfa0-kserve-757756cc4f-6fd6v"
Sep 07 12:04:43.398448 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:04:43.398355    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9911c0b6-2b94-4b1f-829f-77fb622098ef-home\") pod \"e2e-conflict-a-6267dfa0-kserve-757756cc4f-6fd6v\" (UID: \"9911c0b6-2b94-4b1f-829f-77fb622098ef\") " pod="llm/e2e-conflict-a-6267dfa0-kserve-757756cc4f-6fd6v"
Sep 07 12:04:43.398680 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:04:43.398664    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4f6rt\" (UniqueName: \"kubernetes.io/projected/9911c0b6-2b94-4b1f-829f-77fb622098ef-kube-api-access-4f6rt\") pod \"e2e-conflict-a-6267dfa0-kserve-757756cc4f-6fd6v\" (UID: \"9911c0b6-2b94-4b1f-829f-77fb622098ef\") " pod="llm/e2e-conflict-a-6267dfa0-kserve-757756cc4f-6fd6v"
Sep 07 12:04:43.398763 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:04:43.398740    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9911c0b6-2b94-4b1f-829f-77fb622098ef-home\") pod \"e2e-conflict-a-6267dfa0-kserve-757756cc4f-6fd6v\" (UID: \"9911c0b6-2b94-4b1f-829f-77fb622098ef\") " pod="llm/e2e-conflict-a-6267dfa0-kserve-757756cc4f-6fd6v"
Sep 07 12:04:43.398837 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:04:43.398826    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9911c0b6-2b94-4b1f-829f-77fb622098ef-tls-certs\") pod \"e2e-conflict-a-6267dfa0-kserve-757756cc4f-6fd6v\" (UID: \"9911c0b6-2b94-4b1f-829f-77fb622098ef\") " pod="llm/e2e-conflict-a-6267dfa0-kserve-757756cc4f-6fd6v"
Sep 07 12:04:43.398962 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:04:43.398952    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9911c0b6-2b94-4b1f-829f-77fb622098ef-tmp-dir\") pod \"e2e-conflict-a-6267dfa0-kserve-757756cc4f-6fd6v\" (UID: \"9911c0b6-2b94-4b1f-829f-77fb622098ef\") " pod="llm/e2e-conflict-a-6267dfa0-kserve-757756cc4f-6fd6v"
Sep 07 12:04:43.399039 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:04:43.399030    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9911c0b6-2b94-4b1f-829f-77fb622098ef-model-cache\") pod \"e2e-conflict-a-6267dfa0-kserve-757756cc4f-6fd6v\" (UID: \"9911c0b6-2b94-4b1f-829f-77fb622098ef\") " pod="llm/e2e-conflict-a-6267dfa0-kserve-757756cc4f-6fd6v"
Sep 07 12:04:43.399393 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:04:43.399177    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9911c0b6-2b94-4b1f-829f-77fb622098ef-dshm\") pod \"e2e-conflict-a-6267dfa0-kserve-757756cc4f-6fd6v\" (UID: \"9911c0b6-2b94-4b1f-829f-77fb622098ef\") " pod="llm/e2e-conflict-a-6267dfa0-kserve-757756cc4f-6fd6v"
Sep 07 12:04:43.399393 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:04:43.399239    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9911c0b6-2b94-4b1f-829f-77fb622098ef-tmp-dir\") pod \"e2e-conflict-a-6267dfa0-kserve-757756cc4f-6fd6v\" (UID: \"9911c0b6-2b94-4b1f-829f-77fb622098ef\") " pod="llm/e2e-conflict-a-6267dfa0-kserve-757756cc4f-6fd6v"
Sep 07 12:04:43.399393 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:04:43.399349    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9911c0b6-2b94-4b1f-829f-77fb622098ef-model-cache\") pod \"e2e-conflict-a-6267dfa0-kserve-757756cc4f-6fd6v\" (UID: \"9911c0b6-2b94-4b1f-829f-77fb622098ef\") " pod="llm/e2e-conflict-a-6267dfa0-kserve-757756cc4f-6fd6v"
Sep 07 12:04:43.401225 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:04:43.401204    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9911c0b6-2b94-4b1f-829f-77fb622098ef-dshm\") pod \"e2e-conflict-a-6267dfa0-kserve-757756cc4f-6fd6v\" (UID: \"9911c0b6-2b94-4b1f-829f-77fb622098ef\") " pod="llm/e2e-conflict-a-6267dfa0-kserve-757756cc4f-6fd6v"
Sep 07 12:04:43.401295 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:04:43.401210    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9911c0b6-2b94-4b1f-829f-77fb622098ef-tls-certs\") pod \"e2e-conflict-a-6267dfa0-kserve-757756cc4f-6fd6v\" (UID: \"9911c0b6-2b94-4b1f-829f-77fb622098ef\") " pod="llm/e2e-conflict-a-6267dfa0-kserve-757756cc4f-6fd6v"
Sep 07 12:04:43.405584 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:04:43.405563    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4f6rt\" (UniqueName: \"kubernetes.io/projected/9911c0b6-2b94-4b1f-829f-77fb622098ef-kube-api-access-4f6rt\") pod \"e2e-conflict-a-6267dfa0-kserve-757756cc4f-6fd6v\" (UID: \"9911c0b6-2b94-4b1f-829f-77fb622098ef\") " pod="llm/e2e-conflict-a-6267dfa0-kserve-757756cc4f-6fd6v"
Sep 07 12:04:43.476128 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:04:43.476094    2563 patch_prober.go:28] interesting pod/e2e-inf-b-af50c7-openshift-default-67788bbd-v68lr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.36:15021/healthz/ready\": dial tcp 10.133.0.36:15021: connect: connection refused" start-of-body=
Sep 07 12:04:43.476292 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:04:43.476195    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-af50c7-openshift-default-67788bbd-v68lr" podUID="bc21582c-d7f9-48e4-9309-ffeefa3f7f03" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.36:15021/healthz/ready\": dial tcp 10.133.0.36:15021: connect: connection refused"
Sep 07 12:04:43.507478 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:04:43.507442    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-conflict-a-6267dfa0-kserve-757756cc4f-6fd6v"
Sep 07 12:04:43.594722 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:04:43.594454    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:04:43.803472 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:04:43.803448    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-conflict-a-6267dfa0-kserve-757756cc4f-6fd6v"]
Sep 07 12:04:43.806309 ip-10-0-135-173 kubenswrapper[2563]: W0907 12:04:43.806283    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9911c0b6_2b94_4b1f_829f_77fb622098ef.slice/crio-7f5d9ebae4b2255377476e9938f95e6cc9536f0c0dd5bfcfb6a29c200fb89db7 WatchSource:0}: Error finding container 7f5d9ebae4b2255377476e9938f95e6cc9536f0c0dd5bfcfb6a29c200fb89db7: Status 404 returned error can't find the container with id 7f5d9ebae4b2255377476e9938f95e6cc9536f0c0dd5bfcfb6a29c200fb89db7
Sep 07 12:04:44.401595 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:04:44.401561    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-conflict-a-6267dfa0-kserve-757756cc4f-6fd6v" event={"ID":"9911c0b6-2b94-4b1f-829f-77fb622098ef","Type":"ContainerStarted","Data":"eb1733e573187105ba653a3512d954b6adfa29229795560108ca02e59906f49e"}
Sep 07 12:04:44.401781 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:04:44.401605    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-conflict-a-6267dfa0-kserve-757756cc4f-6fd6v" event={"ID":"9911c0b6-2b94-4b1f-829f-77fb622098ef","Type":"ContainerStarted","Data":"7f5d9ebae4b2255377476e9938f95e6cc9536f0c0dd5bfcfb6a29c200fb89db7"}
Sep 07 12:04:44.418102 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:04:44.418053    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-conflict-a-6267dfa0-kserve-757756cc4f-6fd6v" podStartSLOduration=1.41803759 podStartE2EDuration="1.41803759s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 12:04:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 12:04:44.416784286 +0000 UTC m=+1055.935516101" watchObservedRunningTime="2026-09-07 12:04:44.41803759 +0000 UTC m=+1055.936769402"
Sep 07 12:04:44.476431 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:04:44.476383    2563 patch_prober.go:28] interesting pod/e2e-inf-b-af50c7-openshift-default-67788bbd-v68lr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.36:15021/healthz/ready\": dial tcp 10.133.0.36:15021: connect: connection refused" start-of-body=
Sep 07 12:04:44.476600 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:04:44.476474    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-af50c7-openshift-default-67788bbd-v68lr" podUID="bc21582c-d7f9-48e4-9309-ffeefa3f7f03" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.36:15021/healthz/ready\": dial tcp 10.133.0.36:15021: connect: connection refused"
Sep 07 12:04:45.227871 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:04:45.227838    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:04:45.476005 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:04:45.475962    2563 patch_prober.go:28] interesting pod/e2e-inf-b-af50c7-openshift-default-67788bbd-v68lr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.36:15021/healthz/ready\": dial tcp 10.133.0.36:15021: connect: connection refused" start-of-body=
Sep 07 12:04:45.476187 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:04:45.476024    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-af50c7-openshift-default-67788bbd-v68lr" podUID="bc21582c-d7f9-48e4-9309-ffeefa3f7f03" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.36:15021/healthz/ready\": dial tcp 10.133.0.36:15021: connect: connection refused"
Sep 07 12:04:46.648930 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:04:46.648882    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-af50c7-openshift-default-67788bbd-v68lr"
Sep 07 12:04:46.649374 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:04:46.649352    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-af50c7-openshift-default-67788bbd-v68lr"
Sep 07 12:04:46.650305 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:04:46.650281    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-af50c7-openshift-default-67788bbd-v68lr"
Sep 07 12:04:47.833802 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:04:47.833766    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:04:47.938204 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:04:47.938165    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:04:48.846114 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:04:48.846084    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:04:48.946563 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:04:48.946518    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:04:49.483477 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:04:49.483443    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:04:49.583823 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:04:49.583788    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:04:51.909698 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:04:51.909660    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-p4l4w"]
Sep 07 12:04:53.507730 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:04:53.507692    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-conflict-a-6267dfa0-kserve-757756cc4f-6fd6v"
Sep 07 12:04:53.508130 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:04:53.507745    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-conflict-a-6267dfa0-kserve-757756cc4f-6fd6v"
Sep 07 12:04:53.521442 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:04:53.521383    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-conflict-a-6267dfa0-kserve-757756cc4f-6fd6v"
Sep 07 12:04:54.465580 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:04:54.465549    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-conflict-a-6267dfa0-kserve-757756cc4f-6fd6v"
Sep 07 12:05:13.590118 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:05:13.590040    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-autores-ebf845/autores-model-ebf845-kserve-56d5f6fb7f-4tjmn"]
Sep 07 12:05:13.592679 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:05:13.592656    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-autores-ebf845/autores-model-ebf845-kserve-56d5f6fb7f-4tjmn"
Sep 07 12:05:13.594588 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:05:13.594565    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-autores-ebf845\"/\"kube-root-ca.crt\""
Sep 07 12:05:13.594709 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:05:13.594566    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-autores-ebf845\"/\"default-dockercfg-r25qc\""
Sep 07 12:05:13.594929 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:05:13.594912    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-autores-ebf845\"/\"autores-model-ebf845-kserve-self-signed-certs\""
Sep 07 12:05:13.595023 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:05:13.594949    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-autores-ebf845\"/\"openshift-service-ca.crt\""
Sep 07 12:05:13.599590 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:05:13.599559    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-autores-ebf845/autores-model-ebf845-kserve-56d5f6fb7f-4tjmn"]
Sep 07 12:05:13.692531 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:05:13.692499    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:05:13.692776 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:05:13.692757    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/669e2ea8-7635-4eaf-88ac-9f755ad14d8c-home\") pod \"autores-model-ebf845-kserve-56d5f6fb7f-4tjmn\" (UID: \"669e2ea8-7635-4eaf-88ac-9f755ad14d8c\") " pod="ai-tenant-e2e-autores-ebf845/autores-model-ebf845-kserve-56d5f6fb7f-4tjmn"
Sep 07 12:05:13.692840 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:05:13.692793    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/669e2ea8-7635-4eaf-88ac-9f755ad14d8c-tmp-dir\") pod \"autores-model-ebf845-kserve-56d5f6fb7f-4tjmn\" (UID: \"669e2ea8-7635-4eaf-88ac-9f755ad14d8c\") " pod="ai-tenant-e2e-autores-ebf845/autores-model-ebf845-kserve-56d5f6fb7f-4tjmn"
Sep 07 12:05:13.692877 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:05:13.692864    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hvcwn\" (UniqueName: \"kubernetes.io/projected/669e2ea8-7635-4eaf-88ac-9f755ad14d8c-kube-api-access-hvcwn\") pod \"autores-model-ebf845-kserve-56d5f6fb7f-4tjmn\" (UID: \"669e2ea8-7635-4eaf-88ac-9f755ad14d8c\") " pod="ai-tenant-e2e-autores-ebf845/autores-model-ebf845-kserve-56d5f6fb7f-4tjmn"
Sep 07 12:05:13.692988 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:05:13.692972    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/669e2ea8-7635-4eaf-88ac-9f755ad14d8c-model-cache\") pod \"autores-model-ebf845-kserve-56d5f6fb7f-4tjmn\" (UID: \"669e2ea8-7635-4eaf-88ac-9f755ad14d8c\") " pod="ai-tenant-e2e-autores-ebf845/autores-model-ebf845-kserve-56d5f6fb7f-4tjmn"
Sep 07 12:05:13.693035 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:05:13.693014    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/669e2ea8-7635-4eaf-88ac-9f755ad14d8c-tls-certs\") pod \"autores-model-ebf845-kserve-56d5f6fb7f-4tjmn\" (UID: \"669e2ea8-7635-4eaf-88ac-9f755ad14d8c\") " pod="ai-tenant-e2e-autores-ebf845/autores-model-ebf845-kserve-56d5f6fb7f-4tjmn"
Sep 07 12:05:13.693115 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:05:13.693095    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/669e2ea8-7635-4eaf-88ac-9f755ad14d8c-dshm\") pod \"autores-model-ebf845-kserve-56d5f6fb7f-4tjmn\" (UID: \"669e2ea8-7635-4eaf-88ac-9f755ad14d8c\") " pod="ai-tenant-e2e-autores-ebf845/autores-model-ebf845-kserve-56d5f6fb7f-4tjmn"
Sep 07 12:05:13.793765 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:05:13.793735    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hvcwn\" (UniqueName: \"kubernetes.io/projected/669e2ea8-7635-4eaf-88ac-9f755ad14d8c-kube-api-access-hvcwn\") pod \"autores-model-ebf845-kserve-56d5f6fb7f-4tjmn\" (UID: \"669e2ea8-7635-4eaf-88ac-9f755ad14d8c\") " pod="ai-tenant-e2e-autores-ebf845/autores-model-ebf845-kserve-56d5f6fb7f-4tjmn"
Sep 07 12:05:13.793915 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:05:13.793791    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/669e2ea8-7635-4eaf-88ac-9f755ad14d8c-model-cache\") pod \"autores-model-ebf845-kserve-56d5f6fb7f-4tjmn\" (UID: \"669e2ea8-7635-4eaf-88ac-9f755ad14d8c\") " pod="ai-tenant-e2e-autores-ebf845/autores-model-ebf845-kserve-56d5f6fb7f-4tjmn"
Sep 07 12:05:13.793915 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:05:13.793816    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/669e2ea8-7635-4eaf-88ac-9f755ad14d8c-tls-certs\") pod \"autores-model-ebf845-kserve-56d5f6fb7f-4tjmn\" (UID: \"669e2ea8-7635-4eaf-88ac-9f755ad14d8c\") " pod="ai-tenant-e2e-autores-ebf845/autores-model-ebf845-kserve-56d5f6fb7f-4tjmn"
Sep 07 12:05:13.793915 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:05:13.793844    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/669e2ea8-7635-4eaf-88ac-9f755ad14d8c-dshm\") pod \"autores-model-ebf845-kserve-56d5f6fb7f-4tjmn\" (UID: \"669e2ea8-7635-4eaf-88ac-9f755ad14d8c\") " pod="ai-tenant-e2e-autores-ebf845/autores-model-ebf845-kserve-56d5f6fb7f-4tjmn"
Sep 07 12:05:13.793915 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:05:13.793863    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/669e2ea8-7635-4eaf-88ac-9f755ad14d8c-home\") pod \"autores-model-ebf845-kserve-56d5f6fb7f-4tjmn\" (UID: \"669e2ea8-7635-4eaf-88ac-9f755ad14d8c\") " pod="ai-tenant-e2e-autores-ebf845/autores-model-ebf845-kserve-56d5f6fb7f-4tjmn"
Sep 07 12:05:13.793915 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:05:13.793887    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/669e2ea8-7635-4eaf-88ac-9f755ad14d8c-tmp-dir\") pod \"autores-model-ebf845-kserve-56d5f6fb7f-4tjmn\" (UID: \"669e2ea8-7635-4eaf-88ac-9f755ad14d8c\") " pod="ai-tenant-e2e-autores-ebf845/autores-model-ebf845-kserve-56d5f6fb7f-4tjmn"
Sep 07 12:05:13.794267 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:05:13.794246    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/669e2ea8-7635-4eaf-88ac-9f755ad14d8c-model-cache\") pod \"autores-model-ebf845-kserve-56d5f6fb7f-4tjmn\" (UID: \"669e2ea8-7635-4eaf-88ac-9f755ad14d8c\") " pod="ai-tenant-e2e-autores-ebf845/autores-model-ebf845-kserve-56d5f6fb7f-4tjmn"
Sep 07 12:05:13.794335 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:05:13.794274    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/669e2ea8-7635-4eaf-88ac-9f755ad14d8c-tmp-dir\") pod \"autores-model-ebf845-kserve-56d5f6fb7f-4tjmn\" (UID: \"669e2ea8-7635-4eaf-88ac-9f755ad14d8c\") " pod="ai-tenant-e2e-autores-ebf845/autores-model-ebf845-kserve-56d5f6fb7f-4tjmn"
Sep 07 12:05:13.794335 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:05:13.794322    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/669e2ea8-7635-4eaf-88ac-9f755ad14d8c-home\") pod \"autores-model-ebf845-kserve-56d5f6fb7f-4tjmn\" (UID: \"669e2ea8-7635-4eaf-88ac-9f755ad14d8c\") " pod="ai-tenant-e2e-autores-ebf845/autores-model-ebf845-kserve-56d5f6fb7f-4tjmn"
Sep 07 12:05:13.796150 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:05:13.796129    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/669e2ea8-7635-4eaf-88ac-9f755ad14d8c-dshm\") pod \"autores-model-ebf845-kserve-56d5f6fb7f-4tjmn\" (UID: \"669e2ea8-7635-4eaf-88ac-9f755ad14d8c\") " pod="ai-tenant-e2e-autores-ebf845/autores-model-ebf845-kserve-56d5f6fb7f-4tjmn"
Sep 07 12:05:13.796579 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:05:13.796556    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/669e2ea8-7635-4eaf-88ac-9f755ad14d8c-tls-certs\") pod \"autores-model-ebf845-kserve-56d5f6fb7f-4tjmn\" (UID: \"669e2ea8-7635-4eaf-88ac-9f755ad14d8c\") " pod="ai-tenant-e2e-autores-ebf845/autores-model-ebf845-kserve-56d5f6fb7f-4tjmn"
Sep 07 12:05:13.800289 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:05:13.800264    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hvcwn\" (UniqueName: \"kubernetes.io/projected/669e2ea8-7635-4eaf-88ac-9f755ad14d8c-kube-api-access-hvcwn\") pod \"autores-model-ebf845-kserve-56d5f6fb7f-4tjmn\" (UID: \"669e2ea8-7635-4eaf-88ac-9f755ad14d8c\") " pod="ai-tenant-e2e-autores-ebf845/autores-model-ebf845-kserve-56d5f6fb7f-4tjmn"
Sep 07 12:05:13.903898 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:05:13.903829    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-autores-ebf845/autores-model-ebf845-kserve-56d5f6fb7f-4tjmn"
Sep 07 12:05:14.169746 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:05:14.169721    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-autores-ebf845/autores-model-ebf845-kserve-56d5f6fb7f-4tjmn"]
Sep 07 12:05:14.172441 ip-10-0-135-173 kubenswrapper[2563]: W0907 12:05:14.172386    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod669e2ea8_7635_4eaf_88ac_9f755ad14d8c.slice/crio-f5248018b82c3868cf27b63fe7d3f03e275e601570549a2ae402b60f9bbe9b76 WatchSource:0}: Error finding container f5248018b82c3868cf27b63fe7d3f03e275e601570549a2ae402b60f9bbe9b76: Status 404 returned error can't find the container with id f5248018b82c3868cf27b63fe7d3f03e275e601570549a2ae402b60f9bbe9b76
Sep 07 12:05:14.566497 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:05:14.566465    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-autores-ebf845/autores-model-ebf845-kserve-56d5f6fb7f-4tjmn" event={"ID":"669e2ea8-7635-4eaf-88ac-9f755ad14d8c","Type":"ContainerStarted","Data":"6785c5ab72b80fe1be3c52de9bbe86b91d511cd817b0366e70994eb4b640cb88"}
Sep 07 12:05:14.566497 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:05:14.566507    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-autores-ebf845/autores-model-ebf845-kserve-56d5f6fb7f-4tjmn" event={"ID":"669e2ea8-7635-4eaf-88ac-9f755ad14d8c","Type":"ContainerStarted","Data":"f5248018b82c3868cf27b63fe7d3f03e275e601570549a2ae402b60f9bbe9b76"}
Sep 07 12:05:14.580855 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:05:14.580808    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-autores-ebf845/autores-model-ebf845-kserve-56d5f6fb7f-4tjmn" podStartSLOduration=1.580793297 podStartE2EDuration="1.580793297s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 12:05:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 12:05:14.57808193 +0000 UTC m=+1086.096813742" watchObservedRunningTime="2026-09-07 12:05:14.580793297 +0000 UTC m=+1086.099525109"
Sep 07 12:05:19.484517 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:05:19.484467    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:05:19.584808 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:05:19.584777    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:05:19.677137 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:05:19.677095    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:05:19.731274 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:05:19.731244    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:05:19.812877 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:05:19.812770    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:05:20.216940 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:05:20.216908    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:05:20.497605 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:05:20.497518    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:05:20.657244 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:05:20.657206    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:05:20.806150 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:05:20.806066    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-p4l4w"]
Sep 07 12:05:22.741790 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:05:22.741741    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:05:22.828914 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:05:22.828878    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:05:23.856860 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:05:23.856824    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:05:23.904942 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:05:23.904893    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-autores-ebf845/autores-model-ebf845-kserve-56d5f6fb7f-4tjmn"
Sep 07 12:05:23.904942 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:05:23.904949    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-autores-ebf845/autores-model-ebf845-kserve-56d5f6fb7f-4tjmn"
Sep 07 12:05:23.923231 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:05:23.923206    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-autores-ebf845/autores-model-ebf845-kserve-56d5f6fb7f-4tjmn"
Sep 07 12:05:23.944331 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:05:23.944301    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:05:24.073184 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:05:24.073143    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:05:24.444238 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:05:24.444199    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:05:24.544995 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:05:24.544962    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:05:24.627618 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:05:24.627577    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-autores-ebf845/autores-model-ebf845-kserve-56d5f6fb7f-4tjmn"
Sep 07 12:05:34.974410 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:05:34.974369    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:05:37.786475 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:05:37.786442    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-a-c7d751/test-model-c7d751-kserve-67b985587-d97xz"]
Sep 07 12:05:37.789258 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:05:37.789241    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-c7d751/test-model-c7d751-kserve-67b985587-d97xz"
Sep 07 12:05:37.791407 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:05:37.791377    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-c7d751\"/\"test-model-c7d751-kserve-self-signed-certs\""
Sep 07 12:05:37.791407 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:05:37.791379    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-c7d751\"/\"openshift-service-ca.crt\""
Sep 07 12:05:37.791629 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:05:37.791380    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-c7d751\"/\"kube-root-ca.crt\""
Sep 07 12:05:37.791629 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:05:37.791380    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-c7d751\"/\"default-dockercfg-4z77l\""
Sep 07 12:05:37.798268 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:05:37.798243    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-c7d751/test-model-c7d751-kserve-67b985587-d97xz"]
Sep 07 12:05:37.938285 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:05:37.938249    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cd6bd43f-698a-491f-90b1-0a0d0519939d-tmp-dir\") pod \"test-model-c7d751-kserve-67b985587-d97xz\" (UID: \"cd6bd43f-698a-491f-90b1-0a0d0519939d\") " pod="ai-tenant-e2e-inf-a-c7d751/test-model-c7d751-kserve-67b985587-d97xz"
Sep 07 12:05:37.938474 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:05:37.938291    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cd6bd43f-698a-491f-90b1-0a0d0519939d-tls-certs\") pod \"test-model-c7d751-kserve-67b985587-d97xz\" (UID: \"cd6bd43f-698a-491f-90b1-0a0d0519939d\") " pod="ai-tenant-e2e-inf-a-c7d751/test-model-c7d751-kserve-67b985587-d97xz"
Sep 07 12:05:37.938474 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:05:37.938343    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cd6bd43f-698a-491f-90b1-0a0d0519939d-home\") pod \"test-model-c7d751-kserve-67b985587-d97xz\" (UID: \"cd6bd43f-698a-491f-90b1-0a0d0519939d\") " pod="ai-tenant-e2e-inf-a-c7d751/test-model-c7d751-kserve-67b985587-d97xz"
Sep 07 12:05:37.938474 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:05:37.938382    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cd6bd43f-698a-491f-90b1-0a0d0519939d-dshm\") pod \"test-model-c7d751-kserve-67b985587-d97xz\" (UID: \"cd6bd43f-698a-491f-90b1-0a0d0519939d\") " pod="ai-tenant-e2e-inf-a-c7d751/test-model-c7d751-kserve-67b985587-d97xz"
Sep 07 12:05:37.938474 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:05:37.938398    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cd6bd43f-698a-491f-90b1-0a0d0519939d-model-cache\") pod \"test-model-c7d751-kserve-67b985587-d97xz\" (UID: \"cd6bd43f-698a-491f-90b1-0a0d0519939d\") " pod="ai-tenant-e2e-inf-a-c7d751/test-model-c7d751-kserve-67b985587-d97xz"
Sep 07 12:05:37.938634 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:05:37.938506    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-72t4d\" (UniqueName: \"kubernetes.io/projected/cd6bd43f-698a-491f-90b1-0a0d0519939d-kube-api-access-72t4d\") pod \"test-model-c7d751-kserve-67b985587-d97xz\" (UID: \"cd6bd43f-698a-491f-90b1-0a0d0519939d\") " pod="ai-tenant-e2e-inf-a-c7d751/test-model-c7d751-kserve-67b985587-d97xz"
Sep 07 12:05:38.039150 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:05:38.039065    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cd6bd43f-698a-491f-90b1-0a0d0519939d-tmp-dir\") pod \"test-model-c7d751-kserve-67b985587-d97xz\" (UID: \"cd6bd43f-698a-491f-90b1-0a0d0519939d\") " pod="ai-tenant-e2e-inf-a-c7d751/test-model-c7d751-kserve-67b985587-d97xz"
Sep 07 12:05:38.039150 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:05:38.039102    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cd6bd43f-698a-491f-90b1-0a0d0519939d-tls-certs\") pod \"test-model-c7d751-kserve-67b985587-d97xz\" (UID: \"cd6bd43f-698a-491f-90b1-0a0d0519939d\") " pod="ai-tenant-e2e-inf-a-c7d751/test-model-c7d751-kserve-67b985587-d97xz"
Sep 07 12:05:38.039150 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:05:38.039133    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cd6bd43f-698a-491f-90b1-0a0d0519939d-home\") pod \"test-model-c7d751-kserve-67b985587-d97xz\" (UID: \"cd6bd43f-698a-491f-90b1-0a0d0519939d\") " pod="ai-tenant-e2e-inf-a-c7d751/test-model-c7d751-kserve-67b985587-d97xz"
Sep 07 12:05:38.039436 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:05:38.039183    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cd6bd43f-698a-491f-90b1-0a0d0519939d-dshm\") pod \"test-model-c7d751-kserve-67b985587-d97xz\" (UID: \"cd6bd43f-698a-491f-90b1-0a0d0519939d\") " pod="ai-tenant-e2e-inf-a-c7d751/test-model-c7d751-kserve-67b985587-d97xz"
Sep 07 12:05:38.039436 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:05:38.039221    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cd6bd43f-698a-491f-90b1-0a0d0519939d-model-cache\") pod \"test-model-c7d751-kserve-67b985587-d97xz\" (UID: \"cd6bd43f-698a-491f-90b1-0a0d0519939d\") " pod="ai-tenant-e2e-inf-a-c7d751/test-model-c7d751-kserve-67b985587-d97xz"
Sep 07 12:05:38.039436 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:05:38.039284    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-72t4d\" (UniqueName: \"kubernetes.io/projected/cd6bd43f-698a-491f-90b1-0a0d0519939d-kube-api-access-72t4d\") pod \"test-model-c7d751-kserve-67b985587-d97xz\" (UID: \"cd6bd43f-698a-491f-90b1-0a0d0519939d\") " pod="ai-tenant-e2e-inf-a-c7d751/test-model-c7d751-kserve-67b985587-d97xz"
Sep 07 12:05:38.039607 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:05:38.039542    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cd6bd43f-698a-491f-90b1-0a0d0519939d-tmp-dir\") pod \"test-model-c7d751-kserve-67b985587-d97xz\" (UID: \"cd6bd43f-698a-491f-90b1-0a0d0519939d\") " pod="ai-tenant-e2e-inf-a-c7d751/test-model-c7d751-kserve-67b985587-d97xz"
Sep 07 12:05:38.039607 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:05:38.039582    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cd6bd43f-698a-491f-90b1-0a0d0519939d-home\") pod \"test-model-c7d751-kserve-67b985587-d97xz\" (UID: \"cd6bd43f-698a-491f-90b1-0a0d0519939d\") " pod="ai-tenant-e2e-inf-a-c7d751/test-model-c7d751-kserve-67b985587-d97xz"
Sep 07 12:05:38.039713 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:05:38.039637    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cd6bd43f-698a-491f-90b1-0a0d0519939d-model-cache\") pod \"test-model-c7d751-kserve-67b985587-d97xz\" (UID: \"cd6bd43f-698a-491f-90b1-0a0d0519939d\") " pod="ai-tenant-e2e-inf-a-c7d751/test-model-c7d751-kserve-67b985587-d97xz"
Sep 07 12:05:38.041620 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:05:38.041598    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cd6bd43f-698a-491f-90b1-0a0d0519939d-dshm\") pod \"test-model-c7d751-kserve-67b985587-d97xz\" (UID: \"cd6bd43f-698a-491f-90b1-0a0d0519939d\") " pod="ai-tenant-e2e-inf-a-c7d751/test-model-c7d751-kserve-67b985587-d97xz"
Sep 07 12:05:38.041857 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:05:38.041838    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cd6bd43f-698a-491f-90b1-0a0d0519939d-tls-certs\") pod \"test-model-c7d751-kserve-67b985587-d97xz\" (UID: \"cd6bd43f-698a-491f-90b1-0a0d0519939d\") " pod="ai-tenant-e2e-inf-a-c7d751/test-model-c7d751-kserve-67b985587-d97xz"
Sep 07 12:05:38.046268 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:05:38.046245    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-72t4d\" (UniqueName: \"kubernetes.io/projected/cd6bd43f-698a-491f-90b1-0a0d0519939d-kube-api-access-72t4d\") pod \"test-model-c7d751-kserve-67b985587-d97xz\" (UID: \"cd6bd43f-698a-491f-90b1-0a0d0519939d\") " pod="ai-tenant-e2e-inf-a-c7d751/test-model-c7d751-kserve-67b985587-d97xz"
Sep 07 12:05:38.087337 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:05:38.087305    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:05:38.100098 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:05:38.100077    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-c7d751/test-model-c7d751-kserve-67b985587-d97xz"
Sep 07 12:05:38.371258 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:05:38.371236    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-c7d751/test-model-c7d751-kserve-67b985587-d97xz"]
Sep 07 12:05:38.373665 ip-10-0-135-173 kubenswrapper[2563]: W0907 12:05:38.373640    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podcd6bd43f_698a_491f_90b1_0a0d0519939d.slice/crio-e07a5b6ad18162364a953d241ad6f1d0f6706d008e82dd639385d57066fe8a13 WatchSource:0}: Error finding container e07a5b6ad18162364a953d241ad6f1d0f6706d008e82dd639385d57066fe8a13: Status 404 returned error can't find the container with id e07a5b6ad18162364a953d241ad6f1d0f6706d008e82dd639385d57066fe8a13
Sep 07 12:05:38.686442 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:05:38.686337    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-c7d751/test-model-c7d751-kserve-67b985587-d97xz" event={"ID":"cd6bd43f-698a-491f-90b1-0a0d0519939d","Type":"ContainerStarted","Data":"f7ff381125aae03c0070a2031bae62bfdae6aaf8d13f6f057824057df678727d"}
Sep 07 12:05:38.686442 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:05:38.686377    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-c7d751/test-model-c7d751-kserve-67b985587-d97xz" event={"ID":"cd6bd43f-698a-491f-90b1-0a0d0519939d","Type":"ContainerStarted","Data":"e07a5b6ad18162364a953d241ad6f1d0f6706d008e82dd639385d57066fe8a13"}
Sep 07 12:05:38.703120 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:05:38.703070    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-a-c7d751/test-model-c7d751-kserve-67b985587-d97xz" podStartSLOduration=1.703054631 podStartE2EDuration="1.703054631s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 12:05:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 12:05:38.701331082 +0000 UTC m=+1110.220062888" watchObservedRunningTime="2026-09-07 12:05:38.703054631 +0000 UTC m=+1110.221786443"
Sep 07 12:05:42.293934 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:05:42.293898    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-conflict-b-6267dfa0-kserve-695f5bc479-b6fv7"]
Sep 07 12:05:42.297161 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:05:42.297140    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-conflict-b-6267dfa0-kserve-695f5bc479-b6fv7"
Sep 07 12:05:42.298808 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:05:42.298780    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-conflict-b-6267dfa0-kserve-self-signed-certs\""
Sep 07 12:05:42.313340 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:05:42.313185    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-conflict-b-6267dfa0-kserve-695f5bc479-b6fv7"]
Sep 07 12:05:42.354099 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:05:42.354071    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:05:42.485847 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:05:42.485809    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a5fd07ed-3cfe-45b4-a507-c6e58b57e361-tls-certs\") pod \"e2e-conflict-b-6267dfa0-kserve-695f5bc479-b6fv7\" (UID: \"a5fd07ed-3cfe-45b4-a507-c6e58b57e361\") " pod="llm/e2e-conflict-b-6267dfa0-kserve-695f5bc479-b6fv7"
Sep 07 12:05:42.485847 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:05:42.485850    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a5fd07ed-3cfe-45b4-a507-c6e58b57e361-home\") pod \"e2e-conflict-b-6267dfa0-kserve-695f5bc479-b6fv7\" (UID: \"a5fd07ed-3cfe-45b4-a507-c6e58b57e361\") " pod="llm/e2e-conflict-b-6267dfa0-kserve-695f5bc479-b6fv7"
Sep 07 12:05:42.486065 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:05:42.485871    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a5fd07ed-3cfe-45b4-a507-c6e58b57e361-model-cache\") pod \"e2e-conflict-b-6267dfa0-kserve-695f5bc479-b6fv7\" (UID: \"a5fd07ed-3cfe-45b4-a507-c6e58b57e361\") " pod="llm/e2e-conflict-b-6267dfa0-kserve-695f5bc479-b6fv7"
Sep 07 12:05:42.486065 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:05:42.485951    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ck5wd\" (UniqueName: \"kubernetes.io/projected/a5fd07ed-3cfe-45b4-a507-c6e58b57e361-kube-api-access-ck5wd\") pod \"e2e-conflict-b-6267dfa0-kserve-695f5bc479-b6fv7\" (UID: \"a5fd07ed-3cfe-45b4-a507-c6e58b57e361\") " pod="llm/e2e-conflict-b-6267dfa0-kserve-695f5bc479-b6fv7"
Sep 07 12:05:42.486065 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:05:42.485991    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a5fd07ed-3cfe-45b4-a507-c6e58b57e361-dshm\") pod \"e2e-conflict-b-6267dfa0-kserve-695f5bc479-b6fv7\" (UID: \"a5fd07ed-3cfe-45b4-a507-c6e58b57e361\") " pod="llm/e2e-conflict-b-6267dfa0-kserve-695f5bc479-b6fv7"
Sep 07 12:05:42.486183 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:05:42.486120    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a5fd07ed-3cfe-45b4-a507-c6e58b57e361-tmp-dir\") pod \"e2e-conflict-b-6267dfa0-kserve-695f5bc479-b6fv7\" (UID: \"a5fd07ed-3cfe-45b4-a507-c6e58b57e361\") " pod="llm/e2e-conflict-b-6267dfa0-kserve-695f5bc479-b6fv7"
Sep 07 12:05:42.587402 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:05:42.587318    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a5fd07ed-3cfe-45b4-a507-c6e58b57e361-tmp-dir\") pod \"e2e-conflict-b-6267dfa0-kserve-695f5bc479-b6fv7\" (UID: \"a5fd07ed-3cfe-45b4-a507-c6e58b57e361\") " pod="llm/e2e-conflict-b-6267dfa0-kserve-695f5bc479-b6fv7"
Sep 07 12:05:42.587402 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:05:42.587375    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a5fd07ed-3cfe-45b4-a507-c6e58b57e361-tls-certs\") pod \"e2e-conflict-b-6267dfa0-kserve-695f5bc479-b6fv7\" (UID: \"a5fd07ed-3cfe-45b4-a507-c6e58b57e361\") " pod="llm/e2e-conflict-b-6267dfa0-kserve-695f5bc479-b6fv7"
Sep 07 12:05:42.587402 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:05:42.587394    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a5fd07ed-3cfe-45b4-a507-c6e58b57e361-home\") pod \"e2e-conflict-b-6267dfa0-kserve-695f5bc479-b6fv7\" (UID: \"a5fd07ed-3cfe-45b4-a507-c6e58b57e361\") " pod="llm/e2e-conflict-b-6267dfa0-kserve-695f5bc479-b6fv7"
Sep 07 12:05:42.587688 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:05:42.587410    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a5fd07ed-3cfe-45b4-a507-c6e58b57e361-model-cache\") pod \"e2e-conflict-b-6267dfa0-kserve-695f5bc479-b6fv7\" (UID: \"a5fd07ed-3cfe-45b4-a507-c6e58b57e361\") " pod="llm/e2e-conflict-b-6267dfa0-kserve-695f5bc479-b6fv7"
Sep 07 12:05:42.587688 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:05:42.587448    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ck5wd\" (UniqueName: \"kubernetes.io/projected/a5fd07ed-3cfe-45b4-a507-c6e58b57e361-kube-api-access-ck5wd\") pod \"e2e-conflict-b-6267dfa0-kserve-695f5bc479-b6fv7\" (UID: \"a5fd07ed-3cfe-45b4-a507-c6e58b57e361\") " pod="llm/e2e-conflict-b-6267dfa0-kserve-695f5bc479-b6fv7"
Sep 07 12:05:42.587688 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:05:42.587473    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a5fd07ed-3cfe-45b4-a507-c6e58b57e361-dshm\") pod \"e2e-conflict-b-6267dfa0-kserve-695f5bc479-b6fv7\" (UID: \"a5fd07ed-3cfe-45b4-a507-c6e58b57e361\") " pod="llm/e2e-conflict-b-6267dfa0-kserve-695f5bc479-b6fv7"
Sep 07 12:05:42.587851 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:05:42.587745    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a5fd07ed-3cfe-45b4-a507-c6e58b57e361-tmp-dir\") pod \"e2e-conflict-b-6267dfa0-kserve-695f5bc479-b6fv7\" (UID: \"a5fd07ed-3cfe-45b4-a507-c6e58b57e361\") " pod="llm/e2e-conflict-b-6267dfa0-kserve-695f5bc479-b6fv7"
Sep 07 12:05:42.587851 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:05:42.587790    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a5fd07ed-3cfe-45b4-a507-c6e58b57e361-home\") pod \"e2e-conflict-b-6267dfa0-kserve-695f5bc479-b6fv7\" (UID: \"a5fd07ed-3cfe-45b4-a507-c6e58b57e361\") " pod="llm/e2e-conflict-b-6267dfa0-kserve-695f5bc479-b6fv7"
Sep 07 12:05:42.587851 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:05:42.587839    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a5fd07ed-3cfe-45b4-a507-c6e58b57e361-model-cache\") pod \"e2e-conflict-b-6267dfa0-kserve-695f5bc479-b6fv7\" (UID: \"a5fd07ed-3cfe-45b4-a507-c6e58b57e361\") " pod="llm/e2e-conflict-b-6267dfa0-kserve-695f5bc479-b6fv7"
Sep 07 12:05:42.589691 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:05:42.589670    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a5fd07ed-3cfe-45b4-a507-c6e58b57e361-dshm\") pod \"e2e-conflict-b-6267dfa0-kserve-695f5bc479-b6fv7\" (UID: \"a5fd07ed-3cfe-45b4-a507-c6e58b57e361\") " pod="llm/e2e-conflict-b-6267dfa0-kserve-695f5bc479-b6fv7"
Sep 07 12:05:42.589933 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:05:42.589913    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a5fd07ed-3cfe-45b4-a507-c6e58b57e361-tls-certs\") pod \"e2e-conflict-b-6267dfa0-kserve-695f5bc479-b6fv7\" (UID: \"a5fd07ed-3cfe-45b4-a507-c6e58b57e361\") " pod="llm/e2e-conflict-b-6267dfa0-kserve-695f5bc479-b6fv7"
Sep 07 12:05:42.593893 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:05:42.593875    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ck5wd\" (UniqueName: \"kubernetes.io/projected/a5fd07ed-3cfe-45b4-a507-c6e58b57e361-kube-api-access-ck5wd\") pod \"e2e-conflict-b-6267dfa0-kserve-695f5bc479-b6fv7\" (UID: \"a5fd07ed-3cfe-45b4-a507-c6e58b57e361\") " pod="llm/e2e-conflict-b-6267dfa0-kserve-695f5bc479-b6fv7"
Sep 07 12:05:42.615928 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:05:42.615908    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-conflict-b-6267dfa0-kserve-695f5bc479-b6fv7"
Sep 07 12:05:42.883465 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:05:42.883436    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-conflict-b-6267dfa0-kserve-695f5bc479-b6fv7"]
Sep 07 12:05:42.885425 ip-10-0-135-173 kubenswrapper[2563]: W0907 12:05:42.885380    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda5fd07ed_3cfe_45b4_a507_c6e58b57e361.slice/crio-75eb84abbd1b1bbf5a52efc456de3a1690dfb624d70fc63e3a1eebe2ee6c86a3 WatchSource:0}: Error finding container 75eb84abbd1b1bbf5a52efc456de3a1690dfb624d70fc63e3a1eebe2ee6c86a3: Status 404 returned error can't find the container with id 75eb84abbd1b1bbf5a52efc456de3a1690dfb624d70fc63e3a1eebe2ee6c86a3
Sep 07 12:05:43.601973 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:05:43.601946    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-p4l4w"]
Sep 07 12:05:43.714910 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:05:43.714872    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-conflict-b-6267dfa0-kserve-695f5bc479-b6fv7" event={"ID":"a5fd07ed-3cfe-45b4-a507-c6e58b57e361","Type":"ContainerStarted","Data":"2fcb71cdb24e8561a224f7137c1afac4e0d1a0705bf32e40624639c20c954ed7"}
Sep 07 12:05:43.714910 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:05:43.714921    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-conflict-b-6267dfa0-kserve-695f5bc479-b6fv7" event={"ID":"a5fd07ed-3cfe-45b4-a507-c6e58b57e361","Type":"ContainerStarted","Data":"75eb84abbd1b1bbf5a52efc456de3a1690dfb624d70fc63e3a1eebe2ee6c86a3"}
Sep 07 12:05:43.729802 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:05:43.729734    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-conflict-b-6267dfa0-kserve-695f5bc479-b6fv7" podStartSLOduration=1.729720492 podStartE2EDuration="1.729720492s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 12:05:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 12:05:43.727946996 +0000 UTC m=+1115.246678809" watchObservedRunningTime="2026-09-07 12:05:43.729720492 +0000 UTC m=+1115.248452304"
Sep 07 12:05:45.285194 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:05:45.285160    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-autores-ebf845/autores-model-ebf845-kserve-56d5f6fb7f-4tjmn"]
Sep 07 12:05:45.307739 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:05:45.307705    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:05:45.990267 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:05:45.990233    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:05:46.675163 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:05:46.675130    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:05:48.101257 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:05:48.101221    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-a-c7d751/test-model-c7d751-kserve-67b985587-d97xz"
Sep 07 12:05:48.101257 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:05:48.101264    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-a-c7d751/test-model-c7d751-kserve-67b985587-d97xz"
Sep 07 12:05:48.113691 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:05:48.113668    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-a-c7d751/test-model-c7d751-kserve-67b985587-d97xz"
Sep 07 12:05:48.752156 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:05:48.752123    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-a-c7d751/test-model-c7d751-kserve-67b985587-d97xz"
Sep 07 12:05:52.616065 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:05:52.616030    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-conflict-b-6267dfa0-kserve-695f5bc479-b6fv7"
Sep 07 12:05:52.616455 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:05:52.616090    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-conflict-b-6267dfa0-kserve-695f5bc479-b6fv7"
Sep 07 12:05:52.628557 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:05:52.628531    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-conflict-b-6267dfa0-kserve-695f5bc479-b6fv7"
Sep 07 12:05:52.776389 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:05:52.776361    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-conflict-b-6267dfa0-kserve-695f5bc479-b6fv7"
Sep 07 12:05:54.367058 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:05:54.367015    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:05:54.467632 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:05:54.467595    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:05:54.558516 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:05:54.558481    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:05:54.611013 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:05:54.610982    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:05:55.631823 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:05:55.631787    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:05:55.688255 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:05:55.688223    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:05:56.097039 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:05:56.096997    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:05:56.253032 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:05:56.253001    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-ebf845-5cd8fdb5bc-nbwrx"]
Sep 07 12:05:56.253224 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:05:56.253200    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-autores-ebf845-5cd8fdb5bc-nbwrx" podUID="673c6979-7dd6-4815-b8f3-e33f906ed5da" containerName="maas-api" containerID="cri-o://0fd9c4d19e8ff19ffabd9950c5aa0ec5f16276965026c6d7d953562fd1d87b87" gracePeriod=30
Sep 07 12:05:56.286036 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:05:56.286001    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:05:56.365744 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:05:56.365642    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:05:56.468299 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:05:56.468266    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:05:56.495559 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:05:56.495534    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-autores-ebf845-5cd8fdb5bc-nbwrx"
Sep 07 12:05:56.517686 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:05:56.517661    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/673c6979-7dd6-4815-b8f3-e33f906ed5da-metrics-certs\") pod \"673c6979-7dd6-4815-b8f3-e33f906ed5da\" (UID: \"673c6979-7dd6-4815-b8f3-e33f906ed5da\") "
Sep 07 12:05:56.517827 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:05:56.517733    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/673c6979-7dd6-4815-b8f3-e33f906ed5da-maas-api-tls\") pod \"673c6979-7dd6-4815-b8f3-e33f906ed5da\" (UID: \"673c6979-7dd6-4815-b8f3-e33f906ed5da\") "
Sep 07 12:05:56.517950 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:05:56.517840    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/673c6979-7dd6-4815-b8f3-e33f906ed5da-openshift-service-ca\") pod \"673c6979-7dd6-4815-b8f3-e33f906ed5da\" (UID: \"673c6979-7dd6-4815-b8f3-e33f906ed5da\") "
Sep 07 12:05:56.518009 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:05:56.517948    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zh4pm\" (UniqueName: \"kubernetes.io/projected/673c6979-7dd6-4815-b8f3-e33f906ed5da-kube-api-access-zh4pm\") pod \"673c6979-7dd6-4815-b8f3-e33f906ed5da\" (UID: \"673c6979-7dd6-4815-b8f3-e33f906ed5da\") "
Sep 07 12:05:56.518203 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:05:56.518179    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/673c6979-7dd6-4815-b8f3-e33f906ed5da-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "673c6979-7dd6-4815-b8f3-e33f906ed5da" (UID: "673c6979-7dd6-4815-b8f3-e33f906ed5da"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 07 12:05:56.518408 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:05:56.518375    2563 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/673c6979-7dd6-4815-b8f3-e33f906ed5da-openshift-service-ca\") on node \"ip-10-0-135-173.ec2.internal\" DevicePath \"\""
Sep 07 12:05:56.520084 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:05:56.520060    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/673c6979-7dd6-4815-b8f3-e33f906ed5da-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "673c6979-7dd6-4815-b8f3-e33f906ed5da" (UID: "673c6979-7dd6-4815-b8f3-e33f906ed5da"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 07 12:05:56.520175 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:05:56.520075    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/673c6979-7dd6-4815-b8f3-e33f906ed5da-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "673c6979-7dd6-4815-b8f3-e33f906ed5da" (UID: "673c6979-7dd6-4815-b8f3-e33f906ed5da"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 07 12:05:56.520396 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:05:56.520363    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/673c6979-7dd6-4815-b8f3-e33f906ed5da-kube-api-access-zh4pm" (OuterVolumeSpecName: "kube-api-access-zh4pm") pod "673c6979-7dd6-4815-b8f3-e33f906ed5da" (UID: "673c6979-7dd6-4815-b8f3-e33f906ed5da"). InnerVolumeSpecName "kube-api-access-zh4pm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 12:05:56.528903 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:05:56.528861    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:05:56.589959 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:05:56.589929    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:05:56.619818 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:05:56.619754    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zh4pm\" (UniqueName: \"kubernetes.io/projected/673c6979-7dd6-4815-b8f3-e33f906ed5da-kube-api-access-zh4pm\") on node \"ip-10-0-135-173.ec2.internal\" DevicePath \"\""
Sep 07 12:05:56.619818 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:05:56.619777    2563 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/673c6979-7dd6-4815-b8f3-e33f906ed5da-metrics-certs\") on node \"ip-10-0-135-173.ec2.internal\" DevicePath \"\""
Sep 07 12:05:56.619818 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:05:56.619787    2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/673c6979-7dd6-4815-b8f3-e33f906ed5da-maas-api-tls\") on node \"ip-10-0-135-173.ec2.internal\" DevicePath \"\""
Sep 07 12:05:56.785680 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:05:56.785646    2563 generic.go:356] "Generic (PLEG): container finished" podID="673c6979-7dd6-4815-b8f3-e33f906ed5da" containerID="0fd9c4d19e8ff19ffabd9950c5aa0ec5f16276965026c6d7d953562fd1d87b87" exitCode=0
Sep 07 12:05:56.786159 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:05:56.785701    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-autores-ebf845-5cd8fdb5bc-nbwrx"
Sep 07 12:05:56.786159 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:05:56.785729    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-autores-ebf845-5cd8fdb5bc-nbwrx" event={"ID":"673c6979-7dd6-4815-b8f3-e33f906ed5da","Type":"ContainerDied","Data":"0fd9c4d19e8ff19ffabd9950c5aa0ec5f16276965026c6d7d953562fd1d87b87"}
Sep 07 12:05:56.786159 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:05:56.785794    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-autores-ebf845-5cd8fdb5bc-nbwrx" event={"ID":"673c6979-7dd6-4815-b8f3-e33f906ed5da","Type":"ContainerDied","Data":"dc70ea68a2304213a4a260a779662cf84f9d8fe10059e5fc1024050d0e8ab818"}
Sep 07 12:05:56.786159 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:05:56.785826    2563 scope.go:117] "RemoveContainer" containerID="0fd9c4d19e8ff19ffabd9950c5aa0ec5f16276965026c6d7d953562fd1d87b87"
Sep 07 12:05:56.795560 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:05:56.795543    2563 scope.go:117] "RemoveContainer" containerID="0fd9c4d19e8ff19ffabd9950c5aa0ec5f16276965026c6d7d953562fd1d87b87"
Sep 07 12:05:56.795830 ip-10-0-135-173 kubenswrapper[2563]: E0907 12:05:56.795810    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0fd9c4d19e8ff19ffabd9950c5aa0ec5f16276965026c6d7d953562fd1d87b87\": container with ID starting with 0fd9c4d19e8ff19ffabd9950c5aa0ec5f16276965026c6d7d953562fd1d87b87 not found: ID does not exist" containerID="0fd9c4d19e8ff19ffabd9950c5aa0ec5f16276965026c6d7d953562fd1d87b87"
Sep 07 12:05:56.795930 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:05:56.795835    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0fd9c4d19e8ff19ffabd9950c5aa0ec5f16276965026c6d7d953562fd1d87b87"} err="failed to get container status \"0fd9c4d19e8ff19ffabd9950c5aa0ec5f16276965026c6d7d953562fd1d87b87\": rpc error: code = NotFound desc = could not find container \"0fd9c4d19e8ff19ffabd9950c5aa0ec5f16276965026c6d7d953562fd1d87b87\": container with ID starting with 0fd9c4d19e8ff19ffabd9950c5aa0ec5f16276965026c6d7d953562fd1d87b87 not found: ID does not exist"
Sep 07 12:05:56.807816 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:05:56.807792    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-ebf845-5cd8fdb5bc-nbwrx"]
Sep 07 12:05:56.810783 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:05:56.810764    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-ebf845-5cd8fdb5bc-nbwrx"]
Sep 07 12:05:56.858548 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:05:56.858519    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:05:56.977492 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:05:56.977455    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:05:57.040607 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:05:57.040578    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="673c6979-7dd6-4815-b8f3-e33f906ed5da" path="/var/lib/kubelet/pods/673c6979-7dd6-4815-b8f3-e33f906ed5da/volumes"
Sep 07 12:05:58.883630 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:05:58.883597    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:05:58.973524 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:05:58.973493    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:06:00.108790 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:06:00.108755    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:06:00.209140 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:06:00.209100    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:06:00.306856 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:06:00.306814    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-autores-ebf845/autores-model-ebf845-kserve-56d5f6fb7f-4tjmn" podUID="669e2ea8-7635-4eaf-88ac-9f755ad14d8c" containerName="main" containerID="cri-o://6785c5ab72b80fe1be3c52de9bbe86b91d511cd817b0366e70994eb4b640cb88" gracePeriod=45
Sep 07 12:06:00.308970 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:06:00.308940    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:06:00.567483 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:06:00.567456    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-autores-ebf845/autores-model-ebf845-kserve-56d5f6fb7f-4tjmn"
Sep 07 12:06:00.638921 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:06:00.638891    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:06:00.658918 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:06:00.658891    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/669e2ea8-7635-4eaf-88ac-9f755ad14d8c-tls-certs\") pod \"669e2ea8-7635-4eaf-88ac-9f755ad14d8c\" (UID: \"669e2ea8-7635-4eaf-88ac-9f755ad14d8c\") "
Sep 07 12:06:00.659051 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:06:00.658927    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hvcwn\" (UniqueName: \"kubernetes.io/projected/669e2ea8-7635-4eaf-88ac-9f755ad14d8c-kube-api-access-hvcwn\") pod \"669e2ea8-7635-4eaf-88ac-9f755ad14d8c\" (UID: \"669e2ea8-7635-4eaf-88ac-9f755ad14d8c\") "
Sep 07 12:06:00.659051 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:06:00.658959    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/669e2ea8-7635-4eaf-88ac-9f755ad14d8c-tmp-dir\") pod \"669e2ea8-7635-4eaf-88ac-9f755ad14d8c\" (UID: \"669e2ea8-7635-4eaf-88ac-9f755ad14d8c\") "
Sep 07 12:06:00.659051 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:06:00.659003    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/669e2ea8-7635-4eaf-88ac-9f755ad14d8c-home\") pod \"669e2ea8-7635-4eaf-88ac-9f755ad14d8c\" (UID: \"669e2ea8-7635-4eaf-88ac-9f755ad14d8c\") "
Sep 07 12:06:00.659220 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:06:00.659072    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/669e2ea8-7635-4eaf-88ac-9f755ad14d8c-model-cache\") pod \"669e2ea8-7635-4eaf-88ac-9f755ad14d8c\" (UID: \"669e2ea8-7635-4eaf-88ac-9f755ad14d8c\") "
Sep 07 12:06:00.659220 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:06:00.659089    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/669e2ea8-7635-4eaf-88ac-9f755ad14d8c-dshm\") pod \"669e2ea8-7635-4eaf-88ac-9f755ad14d8c\" (UID: \"669e2ea8-7635-4eaf-88ac-9f755ad14d8c\") "
Sep 07 12:06:00.659331 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:06:00.659301    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/669e2ea8-7635-4eaf-88ac-9f755ad14d8c-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "669e2ea8-7635-4eaf-88ac-9f755ad14d8c" (UID: "669e2ea8-7635-4eaf-88ac-9f755ad14d8c"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 12:06:00.659376 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:06:00.659345    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/669e2ea8-7635-4eaf-88ac-9f755ad14d8c-home" (OuterVolumeSpecName: "home") pod "669e2ea8-7635-4eaf-88ac-9f755ad14d8c" (UID: "669e2ea8-7635-4eaf-88ac-9f755ad14d8c"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 12:06:00.659376 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:06:00.659361    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/669e2ea8-7635-4eaf-88ac-9f755ad14d8c-model-cache" (OuterVolumeSpecName: "model-cache") pod "669e2ea8-7635-4eaf-88ac-9f755ad14d8c" (UID: "669e2ea8-7635-4eaf-88ac-9f755ad14d8c"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 12:06:00.659518 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:06:00.659477    2563 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/669e2ea8-7635-4eaf-88ac-9f755ad14d8c-model-cache\") on node \"ip-10-0-135-173.ec2.internal\" DevicePath \"\""
Sep 07 12:06:00.659518 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:06:00.659495    2563 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/669e2ea8-7635-4eaf-88ac-9f755ad14d8c-tmp-dir\") on node \"ip-10-0-135-173.ec2.internal\" DevicePath \"\""
Sep 07 12:06:00.659518 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:06:00.659507    2563 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/669e2ea8-7635-4eaf-88ac-9f755ad14d8c-home\") on node \"ip-10-0-135-173.ec2.internal\" DevicePath \"\""
Sep 07 12:06:00.661540 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:06:00.661506    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/669e2ea8-7635-4eaf-88ac-9f755ad14d8c-dshm" (OuterVolumeSpecName: "dshm") pod "669e2ea8-7635-4eaf-88ac-9f755ad14d8c" (UID: "669e2ea8-7635-4eaf-88ac-9f755ad14d8c"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 12:06:00.661540 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:06:00.661524    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/669e2ea8-7635-4eaf-88ac-9f755ad14d8c-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "669e2ea8-7635-4eaf-88ac-9f755ad14d8c" (UID: "669e2ea8-7635-4eaf-88ac-9f755ad14d8c"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 07 12:06:00.661679 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:06:00.661625    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/669e2ea8-7635-4eaf-88ac-9f755ad14d8c-kube-api-access-hvcwn" (OuterVolumeSpecName: "kube-api-access-hvcwn") pod "669e2ea8-7635-4eaf-88ac-9f755ad14d8c" (UID: "669e2ea8-7635-4eaf-88ac-9f755ad14d8c"). InnerVolumeSpecName "kube-api-access-hvcwn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 12:06:00.760576 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:06:00.760546    2563 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/669e2ea8-7635-4eaf-88ac-9f755ad14d8c-tls-certs\") on node \"ip-10-0-135-173.ec2.internal\" DevicePath \"\""
Sep 07 12:06:00.760576 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:06:00.760574    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hvcwn\" (UniqueName: \"kubernetes.io/projected/669e2ea8-7635-4eaf-88ac-9f755ad14d8c-kube-api-access-hvcwn\") on node \"ip-10-0-135-173.ec2.internal\" DevicePath \"\""
Sep 07 12:06:00.760576 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:06:00.760583    2563 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/669e2ea8-7635-4eaf-88ac-9f755ad14d8c-dshm\") on node \"ip-10-0-135-173.ec2.internal\" DevicePath \"\""
Sep 07 12:06:00.774478 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:06:00.774449    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:06:00.808232 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:06:00.808202    2563 generic.go:356] "Generic (PLEG): container finished" podID="669e2ea8-7635-4eaf-88ac-9f755ad14d8c" containerID="6785c5ab72b80fe1be3c52de9bbe86b91d511cd817b0366e70994eb4b640cb88" exitCode=0
Sep 07 12:06:00.808368 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:06:00.808247    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-autores-ebf845/autores-model-ebf845-kserve-56d5f6fb7f-4tjmn"
Sep 07 12:06:00.808368 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:06:00.808282    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-autores-ebf845/autores-model-ebf845-kserve-56d5f6fb7f-4tjmn" event={"ID":"669e2ea8-7635-4eaf-88ac-9f755ad14d8c","Type":"ContainerDied","Data":"6785c5ab72b80fe1be3c52de9bbe86b91d511cd817b0366e70994eb4b640cb88"}
Sep 07 12:06:00.808368 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:06:00.808326    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-autores-ebf845/autores-model-ebf845-kserve-56d5f6fb7f-4tjmn" event={"ID":"669e2ea8-7635-4eaf-88ac-9f755ad14d8c","Type":"ContainerDied","Data":"f5248018b82c3868cf27b63fe7d3f03e275e601570549a2ae402b60f9bbe9b76"}
Sep 07 12:06:00.808368 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:06:00.808345    2563 scope.go:117] "RemoveContainer" containerID="6785c5ab72b80fe1be3c52de9bbe86b91d511cd817b0366e70994eb4b640cb88"
Sep 07 12:06:00.817593 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:06:00.817576    2563 scope.go:117] "RemoveContainer" containerID="6785c5ab72b80fe1be3c52de9bbe86b91d511cd817b0366e70994eb4b640cb88"
Sep 07 12:06:00.817847 ip-10-0-135-173 kubenswrapper[2563]: E0907 12:06:00.817828    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6785c5ab72b80fe1be3c52de9bbe86b91d511cd817b0366e70994eb4b640cb88\": container with ID starting with 6785c5ab72b80fe1be3c52de9bbe86b91d511cd817b0366e70994eb4b640cb88 not found: ID does not exist" containerID="6785c5ab72b80fe1be3c52de9bbe86b91d511cd817b0366e70994eb4b640cb88"
Sep 07 12:06:00.817909 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:06:00.817859    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6785c5ab72b80fe1be3c52de9bbe86b91d511cd817b0366e70994eb4b640cb88"} err="failed to get container status \"6785c5ab72b80fe1be3c52de9bbe86b91d511cd817b0366e70994eb4b640cb88\": rpc error: code = NotFound desc = could not find container \"6785c5ab72b80fe1be3c52de9bbe86b91d511cd817b0366e70994eb4b640cb88\": container with ID starting with 6785c5ab72b80fe1be3c52de9bbe86b91d511cd817b0366e70994eb4b640cb88 not found: ID does not exist"
Sep 07 12:06:00.830603 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:06:00.830531    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-autores-ebf845/autores-model-ebf845-kserve-56d5f6fb7f-4tjmn"]
Sep 07 12:06:00.832729 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:06:00.832709    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-autores-ebf845/autores-model-ebf845-kserve-56d5f6fb7f-4tjmn"]
Sep 07 12:06:01.041277 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:06:01.041198    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="669e2ea8-7635-4eaf-88ac-9f755ad14d8c" path="/var/lib/kubelet/pods/669e2ea8-7635-4eaf-88ac-9f755ad14d8c/volumes"
Sep 07 12:06:02.853637 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:06:02.853604    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:06:02.913518 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:06:02.913481    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:06:03.109501 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:06:03.109391    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:06:03.574088 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:06:03.574051    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:06:03.772196 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:06:03.772166    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-explicit-3fa352-openshift-default-6b44f857c6-pl9wf"]
Sep 07 12:06:03.773203 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:06:03.773176    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="673c6979-7dd6-4815-b8f3-e33f906ed5da" containerName="maas-api"
Sep 07 12:06:03.773203 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:06:03.773204    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="673c6979-7dd6-4815-b8f3-e33f906ed5da" containerName="maas-api"
Sep 07 12:06:03.773371 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:06:03.773280    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="669e2ea8-7635-4eaf-88ac-9f755ad14d8c" containerName="main"
Sep 07 12:06:03.773371 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:06:03.773290    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="669e2ea8-7635-4eaf-88ac-9f755ad14d8c" containerName="main"
Sep 07 12:06:03.773705 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:06:03.773680    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="673c6979-7dd6-4815-b8f3-e33f906ed5da" containerName="maas-api"
Sep 07 12:06:03.773795 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:06:03.773717    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="669e2ea8-7635-4eaf-88ac-9f755ad14d8c" containerName="main"
Sep 07 12:06:03.786729 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:06:03.786140    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-explicit-3fa352-openshift-default-6b44f857c6-pl9wf"
Sep 07 12:06:03.787653 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:06:03.787630    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-explicit-3fa352-openshift-default-6b44f857c6-pl9wf"]
Sep 07 12:06:03.788079 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:06:03.788052    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-explicit-3fa352-openshift-default-dockercfg-hhq48\""
Sep 07 12:06:03.800575 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:06:03.800551    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:06:03.887863 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:06:03.887784    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sp9fm\" (UniqueName: \"kubernetes.io/projected/3eb6cef9-a0e7-43ba-a582-7a4329013a5b-kube-api-access-sp9fm\") pod \"e2e-explicit-3fa352-openshift-default-6b44f857c6-pl9wf\" (UID: \"3eb6cef9-a0e7-43ba-a582-7a4329013a5b\") " pod="openshift-ingress/e2e-explicit-3fa352-openshift-default-6b44f857c6-pl9wf"
Sep 07 12:06:03.887863 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:06:03.887828    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3eb6cef9-a0e7-43ba-a582-7a4329013a5b-workload-certs\") pod \"e2e-explicit-3fa352-openshift-default-6b44f857c6-pl9wf\" (UID: \"3eb6cef9-a0e7-43ba-a582-7a4329013a5b\") " pod="openshift-ingress/e2e-explicit-3fa352-openshift-default-6b44f857c6-pl9wf"
Sep 07 12:06:03.887863 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:06:03.887853    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3eb6cef9-a0e7-43ba-a582-7a4329013a5b-istiod-ca-cert\") pod \"e2e-explicit-3fa352-openshift-default-6b44f857c6-pl9wf\" (UID: \"3eb6cef9-a0e7-43ba-a582-7a4329013a5b\") " pod="openshift-ingress/e2e-explicit-3fa352-openshift-default-6b44f857c6-pl9wf"
Sep 07 12:06:03.888277 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:06:03.887904    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3eb6cef9-a0e7-43ba-a582-7a4329013a5b-credential-socket\") pod \"e2e-explicit-3fa352-openshift-default-6b44f857c6-pl9wf\" (UID: \"3eb6cef9-a0e7-43ba-a582-7a4329013a5b\") " pod="openshift-ingress/e2e-explicit-3fa352-openshift-default-6b44f857c6-pl9wf"
Sep 07 12:06:03.888277 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:06:03.887943    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3eb6cef9-a0e7-43ba-a582-7a4329013a5b-istio-token\") pod \"e2e-explicit-3fa352-openshift-default-6b44f857c6-pl9wf\" (UID: \"3eb6cef9-a0e7-43ba-a582-7a4329013a5b\") " pod="openshift-ingress/e2e-explicit-3fa352-openshift-default-6b44f857c6-pl9wf"
Sep 07 12:06:03.888277 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:06:03.887971    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3eb6cef9-a0e7-43ba-a582-7a4329013a5b-workload-socket\") pod \"e2e-explicit-3fa352-openshift-default-6b44f857c6-pl9wf\" (UID: \"3eb6cef9-a0e7-43ba-a582-7a4329013a5b\") " pod="openshift-ingress/e2e-explicit-3fa352-openshift-default-6b44f857c6-pl9wf"
Sep 07 12:06:03.888277 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:06:03.888036    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3eb6cef9-a0e7-43ba-a582-7a4329013a5b-istio-podinfo\") pod \"e2e-explicit-3fa352-openshift-default-6b44f857c6-pl9wf\" (UID: \"3eb6cef9-a0e7-43ba-a582-7a4329013a5b\") " pod="openshift-ingress/e2e-explicit-3fa352-openshift-default-6b44f857c6-pl9wf"
Sep 07 12:06:03.888277 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:06:03.888058    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3eb6cef9-a0e7-43ba-a582-7a4329013a5b-istio-envoy\") pod \"e2e-explicit-3fa352-openshift-default-6b44f857c6-pl9wf\" (UID: \"3eb6cef9-a0e7-43ba-a582-7a4329013a5b\") " pod="openshift-ingress/e2e-explicit-3fa352-openshift-default-6b44f857c6-pl9wf"
Sep 07 12:06:03.888277 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:06:03.888110    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3eb6cef9-a0e7-43ba-a582-7a4329013a5b-istio-data\") pod \"e2e-explicit-3fa352-openshift-default-6b44f857c6-pl9wf\" (UID: \"3eb6cef9-a0e7-43ba-a582-7a4329013a5b\") " pod="openshift-ingress/e2e-explicit-3fa352-openshift-default-6b44f857c6-pl9wf"
Sep 07 12:06:03.901917 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:06:03.901825    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:06:03.989607 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:06:03.989576    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3eb6cef9-a0e7-43ba-a582-7a4329013a5b-istio-podinfo\") pod \"e2e-explicit-3fa352-openshift-default-6b44f857c6-pl9wf\" (UID: \"3eb6cef9-a0e7-43ba-a582-7a4329013a5b\") " pod="openshift-ingress/e2e-explicit-3fa352-openshift-default-6b44f857c6-pl9wf"
Sep 07 12:06:03.989607 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:06:03.989610    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3eb6cef9-a0e7-43ba-a582-7a4329013a5b-istio-envoy\") pod \"e2e-explicit-3fa352-openshift-default-6b44f857c6-pl9wf\" (UID: \"3eb6cef9-a0e7-43ba-a582-7a4329013a5b\") " pod="openshift-ingress/e2e-explicit-3fa352-openshift-default-6b44f857c6-pl9wf"
Sep 07 12:06:03.989787 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:06:03.989702    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3eb6cef9-a0e7-43ba-a582-7a4329013a5b-istio-data\") pod \"e2e-explicit-3fa352-openshift-default-6b44f857c6-pl9wf\" (UID: \"3eb6cef9-a0e7-43ba-a582-7a4329013a5b\") " pod="openshift-ingress/e2e-explicit-3fa352-openshift-default-6b44f857c6-pl9wf"
Sep 07 12:06:03.989787 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:06:03.989749    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sp9fm\" (UniqueName: \"kubernetes.io/projected/3eb6cef9-a0e7-43ba-a582-7a4329013a5b-kube-api-access-sp9fm\") pod \"e2e-explicit-3fa352-openshift-default-6b44f857c6-pl9wf\" (UID: \"3eb6cef9-a0e7-43ba-a582-7a4329013a5b\") " pod="openshift-ingress/e2e-explicit-3fa352-openshift-default-6b44f857c6-pl9wf"
Sep 07 12:06:03.989892 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:06:03.989805    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3eb6cef9-a0e7-43ba-a582-7a4329013a5b-workload-certs\") pod \"e2e-explicit-3fa352-openshift-default-6b44f857c6-pl9wf\" (UID: \"3eb6cef9-a0e7-43ba-a582-7a4329013a5b\") " pod="openshift-ingress/e2e-explicit-3fa352-openshift-default-6b44f857c6-pl9wf"
Sep 07 12:06:03.989892 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:06:03.989834    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3eb6cef9-a0e7-43ba-a582-7a4329013a5b-istiod-ca-cert\") pod \"e2e-explicit-3fa352-openshift-default-6b44f857c6-pl9wf\" (UID: \"3eb6cef9-a0e7-43ba-a582-7a4329013a5b\") " pod="openshift-ingress/e2e-explicit-3fa352-openshift-default-6b44f857c6-pl9wf"
Sep 07 12:06:03.990020 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:06:03.989976    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3eb6cef9-a0e7-43ba-a582-7a4329013a5b-credential-socket\") pod \"e2e-explicit-3fa352-openshift-default-6b44f857c6-pl9wf\" (UID: \"3eb6cef9-a0e7-43ba-a582-7a4329013a5b\") " pod="openshift-ingress/e2e-explicit-3fa352-openshift-default-6b44f857c6-pl9wf"
Sep 07 12:06:03.990085 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:06:03.990048    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3eb6cef9-a0e7-43ba-a582-7a4329013a5b-istio-token\") pod \"e2e-explicit-3fa352-openshift-default-6b44f857c6-pl9wf\" (UID: \"3eb6cef9-a0e7-43ba-a582-7a4329013a5b\") " pod="openshift-ingress/e2e-explicit-3fa352-openshift-default-6b44f857c6-pl9wf"
Sep 07 12:06:03.990143 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:06:03.990096    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3eb6cef9-a0e7-43ba-a582-7a4329013a5b-workload-socket\") pod \"e2e-explicit-3fa352-openshift-default-6b44f857c6-pl9wf\" (UID: \"3eb6cef9-a0e7-43ba-a582-7a4329013a5b\") " pod="openshift-ingress/e2e-explicit-3fa352-openshift-default-6b44f857c6-pl9wf"
Sep 07 12:06:03.990201 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:06:03.990150    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3eb6cef9-a0e7-43ba-a582-7a4329013a5b-istio-data\") pod \"e2e-explicit-3fa352-openshift-default-6b44f857c6-pl9wf\" (UID: \"3eb6cef9-a0e7-43ba-a582-7a4329013a5b\") " pod="openshift-ingress/e2e-explicit-3fa352-openshift-default-6b44f857c6-pl9wf"
Sep 07 12:06:03.990376 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:06:03.990355    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3eb6cef9-a0e7-43ba-a582-7a4329013a5b-workload-certs\") pod \"e2e-explicit-3fa352-openshift-default-6b44f857c6-pl9wf\" (UID: \"3eb6cef9-a0e7-43ba-a582-7a4329013a5b\") " pod="openshift-ingress/e2e-explicit-3fa352-openshift-default-6b44f857c6-pl9wf"
Sep 07 12:06:03.990486 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:06:03.990455    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3eb6cef9-a0e7-43ba-a582-7a4329013a5b-credential-socket\") pod \"e2e-explicit-3fa352-openshift-default-6b44f857c6-pl9wf\" (UID: \"3eb6cef9-a0e7-43ba-a582-7a4329013a5b\") " pod="openshift-ingress/e2e-explicit-3fa352-openshift-default-6b44f857c6-pl9wf"
Sep 07 12:06:03.990637 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:06:03.990617    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3eb6cef9-a0e7-43ba-a582-7a4329013a5b-workload-socket\") pod \"e2e-explicit-3fa352-openshift-default-6b44f857c6-pl9wf\" (UID: \"3eb6cef9-a0e7-43ba-a582-7a4329013a5b\") " pod="openshift-ingress/e2e-explicit-3fa352-openshift-default-6b44f857c6-pl9wf"
Sep 07 12:06:03.990794 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:06:03.990771    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3eb6cef9-a0e7-43ba-a582-7a4329013a5b-istiod-ca-cert\") pod \"e2e-explicit-3fa352-openshift-default-6b44f857c6-pl9wf\" (UID: \"3eb6cef9-a0e7-43ba-a582-7a4329013a5b\") " pod="openshift-ingress/e2e-explicit-3fa352-openshift-default-6b44f857c6-pl9wf"
Sep 07 12:06:03.992000 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:06:03.991979    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3eb6cef9-a0e7-43ba-a582-7a4329013a5b-istio-envoy\") pod \"e2e-explicit-3fa352-openshift-default-6b44f857c6-pl9wf\" (UID: \"3eb6cef9-a0e7-43ba-a582-7a4329013a5b\") " pod="openshift-ingress/e2e-explicit-3fa352-openshift-default-6b44f857c6-pl9wf"
Sep 07 12:06:03.992211 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:06:03.992195    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3eb6cef9-a0e7-43ba-a582-7a4329013a5b-istio-podinfo\") pod \"e2e-explicit-3fa352-openshift-default-6b44f857c6-pl9wf\" (UID: \"3eb6cef9-a0e7-43ba-a582-7a4329013a5b\") " pod="openshift-ingress/e2e-explicit-3fa352-openshift-default-6b44f857c6-pl9wf"
Sep 07 12:06:03.997028 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:06:03.996994    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3eb6cef9-a0e7-43ba-a582-7a4329013a5b-istio-token\") pod \"e2e-explicit-3fa352-openshift-default-6b44f857c6-pl9wf\" (UID: \"3eb6cef9-a0e7-43ba-a582-7a4329013a5b\") " pod="openshift-ingress/e2e-explicit-3fa352-openshift-default-6b44f857c6-pl9wf"
Sep 07 12:06:03.997212 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:06:03.997196    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sp9fm\" (UniqueName: \"kubernetes.io/projected/3eb6cef9-a0e7-43ba-a582-7a4329013a5b-kube-api-access-sp9fm\") pod \"e2e-explicit-3fa352-openshift-default-6b44f857c6-pl9wf\" (UID: \"3eb6cef9-a0e7-43ba-a582-7a4329013a5b\") " pod="openshift-ingress/e2e-explicit-3fa352-openshift-default-6b44f857c6-pl9wf"
Sep 07 12:06:04.097324 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:06:04.097298    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-explicit-3fa352-openshift-default-6b44f857c6-pl9wf"
Sep 07 12:06:04.371131 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:06:04.371104    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-explicit-3fa352-openshift-default-6b44f857c6-pl9wf"]
Sep 07 12:06:04.373488 ip-10-0-135-173 kubenswrapper[2563]: W0907 12:06:04.373458    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3eb6cef9_a0e7_43ba_a582_7a4329013a5b.slice/crio-6b80ad41849e61219a2a71a46dd40fc700c982157dc6780796cbb4b0c7fed895 WatchSource:0}: Error finding container 6b80ad41849e61219a2a71a46dd40fc700c982157dc6780796cbb4b0c7fed895: Status 404 returned error can't find the container with id 6b80ad41849e61219a2a71a46dd40fc700c982157dc6780796cbb4b0c7fed895
Sep 07 12:06:04.376058 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:06:04.376013    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 07 12:06:04.376143 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:06:04.376107    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 07 12:06:04.376184 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:06:04.376152    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 07 12:06:04.411309 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:06:04.411279    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-p4l4w"]
Sep 07 12:06:04.833108 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:06:04.833069    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-3fa352-openshift-default-6b44f857c6-pl9wf" event={"ID":"3eb6cef9-a0e7-43ba-a582-7a4329013a5b","Type":"ContainerStarted","Data":"d1109feac35941c502bdd4cda60fe9058262040367a102b968da2dbbe6b90790"}
Sep 07 12:06:04.833278 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:06:04.833140    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-3fa352-openshift-default-6b44f857c6-pl9wf" event={"ID":"3eb6cef9-a0e7-43ba-a582-7a4329013a5b","Type":"ContainerStarted","Data":"6b80ad41849e61219a2a71a46dd40fc700c982157dc6780796cbb4b0c7fed895"}
Sep 07 12:06:04.854103 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:06:04.854050    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-explicit-3fa352-openshift-default-6b44f857c6-pl9wf" podStartSLOduration=1.8540279549999998 podStartE2EDuration="1.854027955s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 12:06:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 12:06:04.851476071 +0000 UTC m=+1136.370207887" watchObservedRunningTime="2026-09-07 12:06:04.854027955 +0000 UTC m=+1136.372759769"
Sep 07 12:06:05.098097 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:06:05.098026    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-explicit-3fa352-openshift-default-6b44f857c6-pl9wf"
Sep 07 12:06:05.103086 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:06:05.103062    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-explicit-3fa352-openshift-default-6b44f857c6-pl9wf"
Sep 07 12:06:05.405147 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:06:05.405061    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:06:05.504891 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:06:05.504860    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:06:05.838169 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:06:05.838139    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-explicit-3fa352-openshift-default-6b44f857c6-pl9wf"
Sep 07 12:06:05.839069 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:06:05.839048    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-explicit-3fa352-openshift-default-6b44f857c6-pl9wf"
Sep 07 12:06:06.697383 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:06:06.697344    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:06:06.788158 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:06:06.788121    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:06:07.945992 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:06:07.945949    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:06:08.002053 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:06:08.002018    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:06:08.431772 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:06:08.431686    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:06:08.514618 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:06:08.514585    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:06:20.100497 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:06:20.100457    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-p4l4w"]
Sep 07 12:06:21.196159 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:06:21.196117    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:06:22.471747 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:06:22.471695    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:06:23.120369 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:06:23.120327    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:06:24.934794 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:06:24.934681    2563 generic.go:356] "Generic (PLEG): container finished" podID="3eb6cef9-a0e7-43ba-a582-7a4329013a5b" containerID="d1109feac35941c502bdd4cda60fe9058262040367a102b968da2dbbe6b90790" exitCode=0
Sep 07 12:06:24.934794 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:06:24.934760    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-3fa352-openshift-default-6b44f857c6-pl9wf" event={"ID":"3eb6cef9-a0e7-43ba-a582-7a4329013a5b","Type":"ContainerDied","Data":"d1109feac35941c502bdd4cda60fe9058262040367a102b968da2dbbe6b90790"}
Sep 07 12:06:24.935334 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:06:24.935286    2563 scope.go:117] "RemoveContainer" containerID="d1109feac35941c502bdd4cda60fe9058262040367a102b968da2dbbe6b90790"
Sep 07 12:06:24.936406 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:06:24.936379    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 07 12:06:24.936527 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:06:24.936453    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 07 12:06:24.936527 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:06:24.936484    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 07 12:06:25.941554 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:06:25.941514    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-3fa352-openshift-default-6b44f857c6-pl9wf" event={"ID":"3eb6cef9-a0e7-43ba-a582-7a4329013a5b","Type":"ContainerStarted","Data":"29acfec6e51dc56b53b3e8a979d40cc7d5cbaed16f9a4574fd0ff4b0ace08052"}
Sep 07 12:06:26.097984 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:06:26.097943    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-explicit-3fa352-openshift-default-6b44f857c6-pl9wf"
Sep 07 12:06:26.098317 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:06:26.098293    2563 patch_prober.go:28] interesting pod/e2e-explicit-3fa352-openshift-default-6b44f857c6-pl9wf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused" start-of-body=
Sep 07 12:06:26.098368 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:06:26.098344    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-3fa352-openshift-default-6b44f857c6-pl9wf" podUID="3eb6cef9-a0e7-43ba-a582-7a4329013a5b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused"
Sep 07 12:06:27.097818 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:06:27.097771    2563 patch_prober.go:28] interesting pod/e2e-explicit-3fa352-openshift-default-6b44f857c6-pl9wf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused" start-of-body=
Sep 07 12:06:27.098255 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:06:27.097834    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-3fa352-openshift-default-6b44f857c6-pl9wf" podUID="3eb6cef9-a0e7-43ba-a582-7a4329013a5b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused"
Sep 07 12:06:28.097768 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:06:28.097732    2563 patch_prober.go:28] interesting pod/e2e-explicit-3fa352-openshift-default-6b44f857c6-pl9wf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused" start-of-body=
Sep 07 12:06:28.097949 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:06:28.097799    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-3fa352-openshift-default-6b44f857c6-pl9wf" podUID="3eb6cef9-a0e7-43ba-a582-7a4329013a5b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused"
Sep 07 12:06:29.097922 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:06:29.097886    2563 patch_prober.go:28] interesting pod/e2e-explicit-3fa352-openshift-default-6b44f857c6-pl9wf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused" start-of-body=
Sep 07 12:06:29.098089 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:06:29.097943    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-3fa352-openshift-default-6b44f857c6-pl9wf" podUID="3eb6cef9-a0e7-43ba-a582-7a4329013a5b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused"
Sep 07 12:06:29.840910 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:06:29.840865    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:06:30.098701 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:06:30.098602    2563 patch_prober.go:28] interesting pod/e2e-explicit-3fa352-openshift-default-6b44f857c6-pl9wf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused" start-of-body=
Sep 07 12:06:30.099141 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:06:30.098704    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-3fa352-openshift-default-6b44f857c6-pl9wf" podUID="3eb6cef9-a0e7-43ba-a582-7a4329013a5b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused"
Sep 07 12:06:30.402360 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:06:30.402263    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:06:30.502063 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:06:30.502023    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:06:30.590594 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:06:30.590555    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:06:30.654167 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:06:30.654061    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:06:30.748350 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:06:30.748321    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:06:31.142507 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:06:31.142430    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-explicit-3fa352-openshift-default-6b44f857c6-pl9wf"
Sep 07 12:06:31.143079 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:06:31.142795    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-explicit-3fa352-openshift-default-6b44f857c6-pl9wf"
Sep 07 12:06:31.143790 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:06:31.143773    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-explicit-3fa352-openshift-default-6b44f857c6-pl9wf"
Sep 07 12:06:31.377156 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:06:31.377113    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:06:31.472921 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:06:31.472890    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:06:31.575483 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:06:31.575446    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:06:31.673156 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:06:31.673121    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:06:32.856278 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:06:32.856241    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:06:33.986784 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:06:33.986749    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-conflict-b-6267dfa0-kserve-695f5bc479-b6fv7"]
Sep 07 12:06:34.014954 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:06:34.014918    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:06:36.100839 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:06:36.100806    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-p4l4w"]
Sep 07 12:06:36.497771 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:06:36.497734    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-conflict-a-6267dfa0-kserve-757756cc4f-6fd6v"]
Sep 07 12:06:36.515914 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:06:36.515852    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:06:38.828959 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:06:38.828921    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:06:38.930313 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:06:38.930280    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:06:39.030593 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:06:39.030564    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:06:39.130025 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:06:39.129909    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:06:41.417437 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:06:41.417384    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:06:49.008534 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:06:49.008470    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-conflict-b-6267dfa0-kserve-695f5bc479-b6fv7" podUID="a5fd07ed-3cfe-45b4-a507-c6e58b57e361" containerName="main" containerID="cri-o://2fcb71cdb24e8561a224f7137c1afac4e0d1a0705bf32e40624639c20c954ed7" gracePeriod=45
Sep 07 12:06:49.250031 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:06:49.250004    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-conflict-b-6267dfa0-kserve-695f5bc479-b6fv7"
Sep 07 12:06:49.360270 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:06:49.360196    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a5fd07ed-3cfe-45b4-a507-c6e58b57e361-tmp-dir\") pod \"a5fd07ed-3cfe-45b4-a507-c6e58b57e361\" (UID: \"a5fd07ed-3cfe-45b4-a507-c6e58b57e361\") "
Sep 07 12:06:49.360270 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:06:49.360233    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a5fd07ed-3cfe-45b4-a507-c6e58b57e361-home\") pod \"a5fd07ed-3cfe-45b4-a507-c6e58b57e361\" (UID: \"a5fd07ed-3cfe-45b4-a507-c6e58b57e361\") "
Sep 07 12:06:49.360493 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:06:49.360274    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a5fd07ed-3cfe-45b4-a507-c6e58b57e361-dshm\") pod \"a5fd07ed-3cfe-45b4-a507-c6e58b57e361\" (UID: \"a5fd07ed-3cfe-45b4-a507-c6e58b57e361\") "
Sep 07 12:06:49.360493 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:06:49.360310    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a5fd07ed-3cfe-45b4-a507-c6e58b57e361-tls-certs\") pod \"a5fd07ed-3cfe-45b4-a507-c6e58b57e361\" (UID: \"a5fd07ed-3cfe-45b4-a507-c6e58b57e361\") "
Sep 07 12:06:49.360493 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:06:49.360384    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a5fd07ed-3cfe-45b4-a507-c6e58b57e361-model-cache\") pod \"a5fd07ed-3cfe-45b4-a507-c6e58b57e361\" (UID: \"a5fd07ed-3cfe-45b4-a507-c6e58b57e361\") "
Sep 07 12:06:49.360493 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:06:49.360434    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ck5wd\" (UniqueName: \"kubernetes.io/projected/a5fd07ed-3cfe-45b4-a507-c6e58b57e361-kube-api-access-ck5wd\") pod \"a5fd07ed-3cfe-45b4-a507-c6e58b57e361\" (UID: \"a5fd07ed-3cfe-45b4-a507-c6e58b57e361\") "
Sep 07 12:06:49.360493 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:06:49.360447    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a5fd07ed-3cfe-45b4-a507-c6e58b57e361-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "a5fd07ed-3cfe-45b4-a507-c6e58b57e361" (UID: "a5fd07ed-3cfe-45b4-a507-c6e58b57e361"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 12:06:49.360759 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:06:49.360552    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a5fd07ed-3cfe-45b4-a507-c6e58b57e361-home" (OuterVolumeSpecName: "home") pod "a5fd07ed-3cfe-45b4-a507-c6e58b57e361" (UID: "a5fd07ed-3cfe-45b4-a507-c6e58b57e361"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 12:06:49.360759 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:06:49.360646    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a5fd07ed-3cfe-45b4-a507-c6e58b57e361-model-cache" (OuterVolumeSpecName: "model-cache") pod "a5fd07ed-3cfe-45b4-a507-c6e58b57e361" (UID: "a5fd07ed-3cfe-45b4-a507-c6e58b57e361"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 12:06:49.360849 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:06:49.360831    2563 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a5fd07ed-3cfe-45b4-a507-c6e58b57e361-home\") on node \"ip-10-0-135-173.ec2.internal\" DevicePath \"\""
Sep 07 12:06:49.360888 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:06:49.360854    2563 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a5fd07ed-3cfe-45b4-a507-c6e58b57e361-model-cache\") on node \"ip-10-0-135-173.ec2.internal\" DevicePath \"\""
Sep 07 12:06:49.360929 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:06:49.360891    2563 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a5fd07ed-3cfe-45b4-a507-c6e58b57e361-tmp-dir\") on node \"ip-10-0-135-173.ec2.internal\" DevicePath \"\""
Sep 07 12:06:49.362547 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:06:49.362520    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a5fd07ed-3cfe-45b4-a507-c6e58b57e361-dshm" (OuterVolumeSpecName: "dshm") pod "a5fd07ed-3cfe-45b4-a507-c6e58b57e361" (UID: "a5fd07ed-3cfe-45b4-a507-c6e58b57e361"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 12:06:49.362649 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:06:49.362544    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a5fd07ed-3cfe-45b4-a507-c6e58b57e361-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "a5fd07ed-3cfe-45b4-a507-c6e58b57e361" (UID: "a5fd07ed-3cfe-45b4-a507-c6e58b57e361"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 07 12:06:49.362649 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:06:49.362574    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a5fd07ed-3cfe-45b4-a507-c6e58b57e361-kube-api-access-ck5wd" (OuterVolumeSpecName: "kube-api-access-ck5wd") pod "a5fd07ed-3cfe-45b4-a507-c6e58b57e361" (UID: "a5fd07ed-3cfe-45b4-a507-c6e58b57e361"). InnerVolumeSpecName "kube-api-access-ck5wd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 12:06:49.462187 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:06:49.462162    2563 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a5fd07ed-3cfe-45b4-a507-c6e58b57e361-dshm\") on node \"ip-10-0-135-173.ec2.internal\" DevicePath \"\""
Sep 07 12:06:49.462187 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:06:49.462183    2563 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a5fd07ed-3cfe-45b4-a507-c6e58b57e361-tls-certs\") on node \"ip-10-0-135-173.ec2.internal\" DevicePath \"\""
Sep 07 12:06:49.462187 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:06:49.462193    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ck5wd\" (UniqueName: \"kubernetes.io/projected/a5fd07ed-3cfe-45b4-a507-c6e58b57e361-kube-api-access-ck5wd\") on node \"ip-10-0-135-173.ec2.internal\" DevicePath \"\""
Sep 07 12:06:50.072201 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:06:50.072163    2563 generic.go:356] "Generic (PLEG): container finished" podID="a5fd07ed-3cfe-45b4-a507-c6e58b57e361" containerID="2fcb71cdb24e8561a224f7137c1afac4e0d1a0705bf32e40624639c20c954ed7" exitCode=0
Sep 07 12:06:50.072611 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:06:50.072224    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-conflict-b-6267dfa0-kserve-695f5bc479-b6fv7"
Sep 07 12:06:50.072611 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:06:50.072247    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-conflict-b-6267dfa0-kserve-695f5bc479-b6fv7" event={"ID":"a5fd07ed-3cfe-45b4-a507-c6e58b57e361","Type":"ContainerDied","Data":"2fcb71cdb24e8561a224f7137c1afac4e0d1a0705bf32e40624639c20c954ed7"}
Sep 07 12:06:50.072611 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:06:50.072291    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-conflict-b-6267dfa0-kserve-695f5bc479-b6fv7" event={"ID":"a5fd07ed-3cfe-45b4-a507-c6e58b57e361","Type":"ContainerDied","Data":"75eb84abbd1b1bbf5a52efc456de3a1690dfb624d70fc63e3a1eebe2ee6c86a3"}
Sep 07 12:06:50.072611 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:06:50.072309    2563 scope.go:117] "RemoveContainer" containerID="2fcb71cdb24e8561a224f7137c1afac4e0d1a0705bf32e40624639c20c954ed7"
Sep 07 12:06:50.081577 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:06:50.081558    2563 scope.go:117] "RemoveContainer" containerID="2fcb71cdb24e8561a224f7137c1afac4e0d1a0705bf32e40624639c20c954ed7"
Sep 07 12:06:50.081853 ip-10-0-135-173 kubenswrapper[2563]: E0907 12:06:50.081835    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2fcb71cdb24e8561a224f7137c1afac4e0d1a0705bf32e40624639c20c954ed7\": container with ID starting with 2fcb71cdb24e8561a224f7137c1afac4e0d1a0705bf32e40624639c20c954ed7 not found: ID does not exist" containerID="2fcb71cdb24e8561a224f7137c1afac4e0d1a0705bf32e40624639c20c954ed7"
Sep 07 12:06:50.081913 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:06:50.081861    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2fcb71cdb24e8561a224f7137c1afac4e0d1a0705bf32e40624639c20c954ed7"} err="failed to get container status \"2fcb71cdb24e8561a224f7137c1afac4e0d1a0705bf32e40624639c20c954ed7\": rpc error: code = NotFound desc = could not find container \"2fcb71cdb24e8561a224f7137c1afac4e0d1a0705bf32e40624639c20c954ed7\": container with ID starting with 2fcb71cdb24e8561a224f7137c1afac4e0d1a0705bf32e40624639c20c954ed7 not found: ID does not exist"
Sep 07 12:06:50.092763 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:06:50.092722    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-conflict-b-6267dfa0-kserve-695f5bc479-b6fv7"]
Sep 07 12:06:50.094647 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:06:50.094628    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-conflict-b-6267dfa0-kserve-695f5bc479-b6fv7"]
Sep 07 12:06:51.041460 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:06:51.041411    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a5fd07ed-3cfe-45b4-a507-c6e58b57e361" path="/var/lib/kubelet/pods/a5fd07ed-3cfe-45b4-a507-c6e58b57e361/volumes"
Sep 07 12:06:51.523646 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:06:51.523582    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-conflict-a-6267dfa0-kserve-757756cc4f-6fd6v" podUID="9911c0b6-2b94-4b1f-829f-77fb622098ef" containerName="main" containerID="cri-o://eb1733e573187105ba653a3512d954b6adfa29229795560108ca02e59906f49e" gracePeriod=45
Sep 07 12:06:51.774882 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:06:51.774825    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-conflict-a-6267dfa0-kserve-757756cc4f-6fd6v"
Sep 07 12:06:51.886319 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:06:51.886286    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9911c0b6-2b94-4b1f-829f-77fb622098ef-home\") pod \"9911c0b6-2b94-4b1f-829f-77fb622098ef\" (UID: \"9911c0b6-2b94-4b1f-829f-77fb622098ef\") "
Sep 07 12:06:51.886531 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:06:51.886349    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9911c0b6-2b94-4b1f-829f-77fb622098ef-dshm\") pod \"9911c0b6-2b94-4b1f-829f-77fb622098ef\" (UID: \"9911c0b6-2b94-4b1f-829f-77fb622098ef\") "
Sep 07 12:06:51.886531 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:06:51.886387    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9911c0b6-2b94-4b1f-829f-77fb622098ef-tmp-dir\") pod \"9911c0b6-2b94-4b1f-829f-77fb622098ef\" (UID: \"9911c0b6-2b94-4b1f-829f-77fb622098ef\") "
Sep 07 12:06:51.886531 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:06:51.886470    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9911c0b6-2b94-4b1f-829f-77fb622098ef-tls-certs\") pod \"9911c0b6-2b94-4b1f-829f-77fb622098ef\" (UID: \"9911c0b6-2b94-4b1f-829f-77fb622098ef\") "
Sep 07 12:06:51.886709 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:06:51.886538    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9911c0b6-2b94-4b1f-829f-77fb622098ef-model-cache\") pod \"9911c0b6-2b94-4b1f-829f-77fb622098ef\" (UID: \"9911c0b6-2b94-4b1f-829f-77fb622098ef\") "
Sep 07 12:06:51.886709 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:06:51.886589    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4f6rt\" (UniqueName: \"kubernetes.io/projected/9911c0b6-2b94-4b1f-829f-77fb622098ef-kube-api-access-4f6rt\") pod \"9911c0b6-2b94-4b1f-829f-77fb622098ef\" (UID: \"9911c0b6-2b94-4b1f-829f-77fb622098ef\") "
Sep 07 12:06:51.886709 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:06:51.886654    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9911c0b6-2b94-4b1f-829f-77fb622098ef-home" (OuterVolumeSpecName: "home") pod "9911c0b6-2b94-4b1f-829f-77fb622098ef" (UID: "9911c0b6-2b94-4b1f-829f-77fb622098ef"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 12:06:51.886868 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:06:51.886733    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9911c0b6-2b94-4b1f-829f-77fb622098ef-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "9911c0b6-2b94-4b1f-829f-77fb622098ef" (UID: "9911c0b6-2b94-4b1f-829f-77fb622098ef"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 12:06:51.886868 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:06:51.886830    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9911c0b6-2b94-4b1f-829f-77fb622098ef-model-cache" (OuterVolumeSpecName: "model-cache") pod "9911c0b6-2b94-4b1f-829f-77fb622098ef" (UID: "9911c0b6-2b94-4b1f-829f-77fb622098ef"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 12:06:51.886998 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:06:51.886981    2563 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9911c0b6-2b94-4b1f-829f-77fb622098ef-home\") on node \"ip-10-0-135-173.ec2.internal\" DevicePath \"\""
Sep 07 12:06:51.887034 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:06:51.887003    2563 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9911c0b6-2b94-4b1f-829f-77fb622098ef-tmp-dir\") on node \"ip-10-0-135-173.ec2.internal\" DevicePath \"\""
Sep 07 12:06:51.887034 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:06:51.887018    2563 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9911c0b6-2b94-4b1f-829f-77fb622098ef-model-cache\") on node \"ip-10-0-135-173.ec2.internal\" DevicePath \"\""
Sep 07 12:06:51.888463 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:06:51.888430    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9911c0b6-2b94-4b1f-829f-77fb622098ef-dshm" (OuterVolumeSpecName: "dshm") pod "9911c0b6-2b94-4b1f-829f-77fb622098ef" (UID: "9911c0b6-2b94-4b1f-829f-77fb622098ef"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 12:06:51.888591 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:06:51.888577    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9911c0b6-2b94-4b1f-829f-77fb622098ef-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "9911c0b6-2b94-4b1f-829f-77fb622098ef" (UID: "9911c0b6-2b94-4b1f-829f-77fb622098ef"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 07 12:06:51.888634 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:06:51.888602    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9911c0b6-2b94-4b1f-829f-77fb622098ef-kube-api-access-4f6rt" (OuterVolumeSpecName: "kube-api-access-4f6rt") pod "9911c0b6-2b94-4b1f-829f-77fb622098ef" (UID: "9911c0b6-2b94-4b1f-829f-77fb622098ef"). InnerVolumeSpecName "kube-api-access-4f6rt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 12:06:51.988083 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:06:51.988056    2563 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9911c0b6-2b94-4b1f-829f-77fb622098ef-tls-certs\") on node \"ip-10-0-135-173.ec2.internal\" DevicePath \"\""
Sep 07 12:06:51.988083 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:06:51.988080    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4f6rt\" (UniqueName: \"kubernetes.io/projected/9911c0b6-2b94-4b1f-829f-77fb622098ef-kube-api-access-4f6rt\") on node \"ip-10-0-135-173.ec2.internal\" DevicePath \"\""
Sep 07 12:06:51.988260 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:06:51.988092    2563 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9911c0b6-2b94-4b1f-829f-77fb622098ef-dshm\") on node \"ip-10-0-135-173.ec2.internal\" DevicePath \"\""
Sep 07 12:06:52.087022 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:06:52.086942    2563 generic.go:356] "Generic (PLEG): container finished" podID="9911c0b6-2b94-4b1f-829f-77fb622098ef" containerID="eb1733e573187105ba653a3512d954b6adfa29229795560108ca02e59906f49e" exitCode=0
Sep 07 12:06:52.087022 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:06:52.086985    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-conflict-a-6267dfa0-kserve-757756cc4f-6fd6v"
Sep 07 12:06:52.087022 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:06:52.087017    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-conflict-a-6267dfa0-kserve-757756cc4f-6fd6v" event={"ID":"9911c0b6-2b94-4b1f-829f-77fb622098ef","Type":"ContainerDied","Data":"eb1733e573187105ba653a3512d954b6adfa29229795560108ca02e59906f49e"}
Sep 07 12:06:52.087281 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:06:52.087045    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-conflict-a-6267dfa0-kserve-757756cc4f-6fd6v" event={"ID":"9911c0b6-2b94-4b1f-829f-77fb622098ef","Type":"ContainerDied","Data":"7f5d9ebae4b2255377476e9938f95e6cc9536f0c0dd5bfcfb6a29c200fb89db7"}
Sep 07 12:06:52.087281 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:06:52.087060    2563 scope.go:117] "RemoveContainer" containerID="eb1733e573187105ba653a3512d954b6adfa29229795560108ca02e59906f49e"
Sep 07 12:06:52.096936 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:06:52.096915    2563 scope.go:117] "RemoveContainer" containerID="eb1733e573187105ba653a3512d954b6adfa29229795560108ca02e59906f49e"
Sep 07 12:06:52.097188 ip-10-0-135-173 kubenswrapper[2563]: E0907 12:06:52.097168    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"eb1733e573187105ba653a3512d954b6adfa29229795560108ca02e59906f49e\": container with ID starting with eb1733e573187105ba653a3512d954b6adfa29229795560108ca02e59906f49e not found: ID does not exist" containerID="eb1733e573187105ba653a3512d954b6adfa29229795560108ca02e59906f49e"
Sep 07 12:06:52.097265 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:06:52.097195    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"eb1733e573187105ba653a3512d954b6adfa29229795560108ca02e59906f49e"} err="failed to get container status \"eb1733e573187105ba653a3512d954b6adfa29229795560108ca02e59906f49e\": rpc error: code = NotFound desc = could not find container \"eb1733e573187105ba653a3512d954b6adfa29229795560108ca02e59906f49e\": container with ID starting with eb1733e573187105ba653a3512d954b6adfa29229795560108ca02e59906f49e not found: ID does not exist"
Sep 07 12:06:52.111331 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:06:52.111310    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-conflict-a-6267dfa0-kserve-757756cc4f-6fd6v"]
Sep 07 12:06:52.115092 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:06:52.115071    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-conflict-a-6267dfa0-kserve-757756cc4f-6fd6v"]
Sep 07 12:06:52.115932 ip-10-0-135-173 kubenswrapper[2563]: E0907 12:06:52.115906    2563 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9911c0b6_2b94_4b1f_829f_77fb622098ef.slice/crio-7f5d9ebae4b2255377476e9938f95e6cc9536f0c0dd5bfcfb6a29c200fb89db7\": RecentStats: unable to find data in memory cache]"
Sep 07 12:06:52.996934 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:06:52.996902    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-p4l4w"]
Sep 07 12:06:53.044635 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:06:53.044597    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9911c0b6-2b94-4b1f-829f-77fb622098ef" path="/var/lib/kubelet/pods/9911c0b6-2b94-4b1f-829f-77fb622098ef/volumes"
Sep 07 12:06:56.601916 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:06:56.601887    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-explicit-3fa352/explicit-model-3fa352-kserve-6cc9dd6b8f-bzwmf"]
Sep 07 12:06:56.602270 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:06:56.602256    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a5fd07ed-3cfe-45b4-a507-c6e58b57e361" containerName="main"
Sep 07 12:06:56.602313 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:06:56.602271    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="a5fd07ed-3cfe-45b4-a507-c6e58b57e361" containerName="main"
Sep 07 12:06:56.602313 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:06:56.602293    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9911c0b6-2b94-4b1f-829f-77fb622098ef" containerName="main"
Sep 07 12:06:56.602313 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:06:56.602300    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="9911c0b6-2b94-4b1f-829f-77fb622098ef" containerName="main"
Sep 07 12:06:56.602405 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:06:56.602357    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="9911c0b6-2b94-4b1f-829f-77fb622098ef" containerName="main"
Sep 07 12:06:56.602405 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:06:56.602368    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="a5fd07ed-3cfe-45b4-a507-c6e58b57e361" containerName="main"
Sep 07 12:06:56.605394 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:06:56.605374    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-explicit-3fa352/explicit-model-3fa352-kserve-6cc9dd6b8f-bzwmf"
Sep 07 12:06:56.608190 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:06:56.608168    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-explicit-3fa352\"/\"kube-root-ca.crt\""
Sep 07 12:06:56.608316 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:06:56.608187    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-explicit-3fa352\"/\"explicit-model-3fa352-kserve-self-signed-certs\""
Sep 07 12:06:56.608316 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:06:56.608169    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-explicit-3fa352\"/\"default-dockercfg-frc7j\""
Sep 07 12:06:56.608316 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:06:56.608169    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-explicit-3fa352\"/\"openshift-service-ca.crt\""
Sep 07 12:06:56.613739 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:06:56.613718    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-explicit-3fa352/explicit-model-3fa352-kserve-6cc9dd6b8f-bzwmf"]
Sep 07 12:06:56.627179 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:06:56.627153    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jd4hw\" (UniqueName: \"kubernetes.io/projected/fe65fc72-f5b9-4b7a-b1a8-372511f3dddc-kube-api-access-jd4hw\") pod \"explicit-model-3fa352-kserve-6cc9dd6b8f-bzwmf\" (UID: \"fe65fc72-f5b9-4b7a-b1a8-372511f3dddc\") " pod="ai-tenant-e2e-explicit-3fa352/explicit-model-3fa352-kserve-6cc9dd6b8f-bzwmf"
Sep 07 12:06:56.627286 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:06:56.627224    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fe65fc72-f5b9-4b7a-b1a8-372511f3dddc-dshm\") pod \"explicit-model-3fa352-kserve-6cc9dd6b8f-bzwmf\" (UID: \"fe65fc72-f5b9-4b7a-b1a8-372511f3dddc\") " pod="ai-tenant-e2e-explicit-3fa352/explicit-model-3fa352-kserve-6cc9dd6b8f-bzwmf"
Sep 07 12:06:56.627286 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:06:56.627258    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fe65fc72-f5b9-4b7a-b1a8-372511f3dddc-home\") pod \"explicit-model-3fa352-kserve-6cc9dd6b8f-bzwmf\" (UID: \"fe65fc72-f5b9-4b7a-b1a8-372511f3dddc\") " pod="ai-tenant-e2e-explicit-3fa352/explicit-model-3fa352-kserve-6cc9dd6b8f-bzwmf"
Sep 07 12:06:56.627403 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:06:56.627303    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fe65fc72-f5b9-4b7a-b1a8-372511f3dddc-tls-certs\") pod \"explicit-model-3fa352-kserve-6cc9dd6b8f-bzwmf\" (UID: \"fe65fc72-f5b9-4b7a-b1a8-372511f3dddc\") " pod="ai-tenant-e2e-explicit-3fa352/explicit-model-3fa352-kserve-6cc9dd6b8f-bzwmf"
Sep 07 12:06:56.627403 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:06:56.627332    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fe65fc72-f5b9-4b7a-b1a8-372511f3dddc-model-cache\") pod \"explicit-model-3fa352-kserve-6cc9dd6b8f-bzwmf\" (UID: \"fe65fc72-f5b9-4b7a-b1a8-372511f3dddc\") " pod="ai-tenant-e2e-explicit-3fa352/explicit-model-3fa352-kserve-6cc9dd6b8f-bzwmf"
Sep 07 12:06:56.627403 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:06:56.627348    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fe65fc72-f5b9-4b7a-b1a8-372511f3dddc-tmp-dir\") pod \"explicit-model-3fa352-kserve-6cc9dd6b8f-bzwmf\" (UID: \"fe65fc72-f5b9-4b7a-b1a8-372511f3dddc\") " pod="ai-tenant-e2e-explicit-3fa352/explicit-model-3fa352-kserve-6cc9dd6b8f-bzwmf"
Sep 07 12:06:56.633960 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:06:56.633938    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:06:56.728696 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:06:56.728668    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jd4hw\" (UniqueName: \"kubernetes.io/projected/fe65fc72-f5b9-4b7a-b1a8-372511f3dddc-kube-api-access-jd4hw\") pod \"explicit-model-3fa352-kserve-6cc9dd6b8f-bzwmf\" (UID: \"fe65fc72-f5b9-4b7a-b1a8-372511f3dddc\") " pod="ai-tenant-e2e-explicit-3fa352/explicit-model-3fa352-kserve-6cc9dd6b8f-bzwmf"
Sep 07 12:06:56.728811 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:06:56.728717    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fe65fc72-f5b9-4b7a-b1a8-372511f3dddc-dshm\") pod \"explicit-model-3fa352-kserve-6cc9dd6b8f-bzwmf\" (UID: \"fe65fc72-f5b9-4b7a-b1a8-372511f3dddc\") " pod="ai-tenant-e2e-explicit-3fa352/explicit-model-3fa352-kserve-6cc9dd6b8f-bzwmf"
Sep 07 12:06:56.728917 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:06:56.728899    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fe65fc72-f5b9-4b7a-b1a8-372511f3dddc-home\") pod \"explicit-model-3fa352-kserve-6cc9dd6b8f-bzwmf\" (UID: \"fe65fc72-f5b9-4b7a-b1a8-372511f3dddc\") " pod="ai-tenant-e2e-explicit-3fa352/explicit-model-3fa352-kserve-6cc9dd6b8f-bzwmf"
Sep 07 12:06:56.728981 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:06:56.728967    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fe65fc72-f5b9-4b7a-b1a8-372511f3dddc-tls-certs\") pod \"explicit-model-3fa352-kserve-6cc9dd6b8f-bzwmf\" (UID: \"fe65fc72-f5b9-4b7a-b1a8-372511f3dddc\") " pod="ai-tenant-e2e-explicit-3fa352/explicit-model-3fa352-kserve-6cc9dd6b8f-bzwmf"
Sep 07 12:06:56.729023 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:06:56.729006    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fe65fc72-f5b9-4b7a-b1a8-372511f3dddc-model-cache\") pod \"explicit-model-3fa352-kserve-6cc9dd6b8f-bzwmf\" (UID: \"fe65fc72-f5b9-4b7a-b1a8-372511f3dddc\") " pod="ai-tenant-e2e-explicit-3fa352/explicit-model-3fa352-kserve-6cc9dd6b8f-bzwmf"
Sep 07 12:06:56.729066 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:06:56.729031    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fe65fc72-f5b9-4b7a-b1a8-372511f3dddc-tmp-dir\") pod \"explicit-model-3fa352-kserve-6cc9dd6b8f-bzwmf\" (UID: \"fe65fc72-f5b9-4b7a-b1a8-372511f3dddc\") " pod="ai-tenant-e2e-explicit-3fa352/explicit-model-3fa352-kserve-6cc9dd6b8f-bzwmf"
Sep 07 12:06:56.729290 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:06:56.729263    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fe65fc72-f5b9-4b7a-b1a8-372511f3dddc-home\") pod \"explicit-model-3fa352-kserve-6cc9dd6b8f-bzwmf\" (UID: \"fe65fc72-f5b9-4b7a-b1a8-372511f3dddc\") " pod="ai-tenant-e2e-explicit-3fa352/explicit-model-3fa352-kserve-6cc9dd6b8f-bzwmf"
Sep 07 12:06:56.729442 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:06:56.729309    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fe65fc72-f5b9-4b7a-b1a8-372511f3dddc-model-cache\") pod \"explicit-model-3fa352-kserve-6cc9dd6b8f-bzwmf\" (UID: \"fe65fc72-f5b9-4b7a-b1a8-372511f3dddc\") " pod="ai-tenant-e2e-explicit-3fa352/explicit-model-3fa352-kserve-6cc9dd6b8f-bzwmf"
Sep 07 12:06:56.729442 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:06:56.729405    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fe65fc72-f5b9-4b7a-b1a8-372511f3dddc-tmp-dir\") pod \"explicit-model-3fa352-kserve-6cc9dd6b8f-bzwmf\" (UID: \"fe65fc72-f5b9-4b7a-b1a8-372511f3dddc\") " pod="ai-tenant-e2e-explicit-3fa352/explicit-model-3fa352-kserve-6cc9dd6b8f-bzwmf"
Sep 07 12:06:56.730877 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:06:56.730849    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fe65fc72-f5b9-4b7a-b1a8-372511f3dddc-dshm\") pod \"explicit-model-3fa352-kserve-6cc9dd6b8f-bzwmf\" (UID: \"fe65fc72-f5b9-4b7a-b1a8-372511f3dddc\") " pod="ai-tenant-e2e-explicit-3fa352/explicit-model-3fa352-kserve-6cc9dd6b8f-bzwmf"
Sep 07 12:06:56.731193 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:06:56.731178    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fe65fc72-f5b9-4b7a-b1a8-372511f3dddc-tls-certs\") pod \"explicit-model-3fa352-kserve-6cc9dd6b8f-bzwmf\" (UID: \"fe65fc72-f5b9-4b7a-b1a8-372511f3dddc\") " pod="ai-tenant-e2e-explicit-3fa352/explicit-model-3fa352-kserve-6cc9dd6b8f-bzwmf"
Sep 07 12:06:56.736360 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:06:56.736326    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jd4hw\" (UniqueName: \"kubernetes.io/projected/fe65fc72-f5b9-4b7a-b1a8-372511f3dddc-kube-api-access-jd4hw\") pod \"explicit-model-3fa352-kserve-6cc9dd6b8f-bzwmf\" (UID: \"fe65fc72-f5b9-4b7a-b1a8-372511f3dddc\") " pod="ai-tenant-e2e-explicit-3fa352/explicit-model-3fa352-kserve-6cc9dd6b8f-bzwmf"
Sep 07 12:06:56.916844 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:06:56.916753    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-explicit-3fa352/explicit-model-3fa352-kserve-6cc9dd6b8f-bzwmf"
Sep 07 12:06:57.186709 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:06:57.186672    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-explicit-3fa352/explicit-model-3fa352-kserve-6cc9dd6b8f-bzwmf"]
Sep 07 12:06:57.188822 ip-10-0-135-173 kubenswrapper[2563]: W0907 12:06:57.188797    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podfe65fc72_f5b9_4b7a_b1a8_372511f3dddc.slice/crio-f5ad7f6794ece6cc8076ea713f162da2c937e6e43797ac35c9da58d26efbfac4 WatchSource:0}: Error finding container f5ad7f6794ece6cc8076ea713f162da2c937e6e43797ac35c9da58d26efbfac4: Status 404 returned error can't find the container with id f5ad7f6794ece6cc8076ea713f162da2c937e6e43797ac35c9da58d26efbfac4
Sep 07 12:06:58.117787 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:06:58.117751    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-explicit-3fa352/explicit-model-3fa352-kserve-6cc9dd6b8f-bzwmf" event={"ID":"fe65fc72-f5b9-4b7a-b1a8-372511f3dddc","Type":"ContainerStarted","Data":"59fc58da243a653dbb79f8f48426dd8cd7e1c455c680e45a85237a0af94e6c71"}
Sep 07 12:06:58.117787 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:06:58.117794    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-explicit-3fa352/explicit-model-3fa352-kserve-6cc9dd6b8f-bzwmf" event={"ID":"fe65fc72-f5b9-4b7a-b1a8-372511f3dddc","Type":"ContainerStarted","Data":"f5ad7f6794ece6cc8076ea713f162da2c937e6e43797ac35c9da58d26efbfac4"}
Sep 07 12:06:58.131501 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:06:58.131444    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-explicit-3fa352/explicit-model-3fa352-kserve-6cc9dd6b8f-bzwmf" podStartSLOduration=2.131396597 podStartE2EDuration="2.131396597s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 12:06:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 12:06:58.131303743 +0000 UTC m=+1189.650035560" watchObservedRunningTime="2026-09-07 12:06:58.131396597 +0000 UTC m=+1189.650128411"
Sep 07 12:07:05.405632 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:07:05.405595    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:07:06.917386 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:07:06.917348    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-explicit-3fa352/explicit-model-3fa352-kserve-6cc9dd6b8f-bzwmf"
Sep 07 12:07:06.917783 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:07:06.917409    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-explicit-3fa352/explicit-model-3fa352-kserve-6cc9dd6b8f-bzwmf"
Sep 07 12:07:06.930199 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:07:06.930177    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-explicit-3fa352/explicit-model-3fa352-kserve-6cc9dd6b8f-bzwmf"
Sep 07 12:07:07.178193 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:07:07.178109    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-explicit-3fa352/explicit-model-3fa352-kserve-6cc9dd6b8f-bzwmf"
Sep 07 12:07:09.166652 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:07:09.166628    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-4nfbc_b34ca85d-ac51-4724-a508-1026106b3f3a/console-operator/2.log"
Sep 07 12:07:09.167724 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:07:09.167704    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-4nfbc_b34ca85d-ac51-4724-a508-1026106b3f3a/console-operator/2.log"
Sep 07 12:07:11.308752 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:07:11.308710    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-p4l4w"]
Sep 07 12:07:11.445547 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:07:11.445516    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:07:11.504344 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:07:11.504305    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:07:11.575515 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:07:11.575439    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:07:11.672551 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:07:11.672519    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:07:20.204224 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:07:20.204189    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-b-af50c7/test-model-af50c7-kserve-6895bd997-8c4w5"]
Sep 07 12:07:20.207444 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:07:20.207407    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-af50c7/test-model-af50c7-kserve-6895bd997-8c4w5"
Sep 07 12:07:20.209638 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:07:20.209611    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-af50c7\"/\"default-dockercfg-trhk9\""
Sep 07 12:07:20.209638 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:07:20.209633    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-af50c7\"/\"kube-root-ca.crt\""
Sep 07 12:07:20.209888 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:07:20.209633    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-af50c7\"/\"test-model-af50c7-kserve-self-signed-certs\""
Sep 07 12:07:20.209888 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:07:20.209656    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-af50c7\"/\"openshift-service-ca.crt\""
Sep 07 12:07:20.213771 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:07:20.213751    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-af50c7/test-model-af50c7-kserve-6895bd997-8c4w5"]
Sep 07 12:07:20.311407 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:07:20.311378    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:07:20.341893 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:07:20.341863    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5141e977-a84d-46f6-adbe-da95fef14eb0-model-cache\") pod \"test-model-af50c7-kserve-6895bd997-8c4w5\" (UID: \"5141e977-a84d-46f6-adbe-da95fef14eb0\") " pod="ai-tenant-e2e-inf-b-af50c7/test-model-af50c7-kserve-6895bd997-8c4w5"
Sep 07 12:07:20.342028 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:07:20.341897    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5141e977-a84d-46f6-adbe-da95fef14eb0-dshm\") pod \"test-model-af50c7-kserve-6895bd997-8c4w5\" (UID: \"5141e977-a84d-46f6-adbe-da95fef14eb0\") " pod="ai-tenant-e2e-inf-b-af50c7/test-model-af50c7-kserve-6895bd997-8c4w5"
Sep 07 12:07:20.342028 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:07:20.341914    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-24vmv\" (UniqueName: \"kubernetes.io/projected/5141e977-a84d-46f6-adbe-da95fef14eb0-kube-api-access-24vmv\") pod \"test-model-af50c7-kserve-6895bd997-8c4w5\" (UID: \"5141e977-a84d-46f6-adbe-da95fef14eb0\") " pod="ai-tenant-e2e-inf-b-af50c7/test-model-af50c7-kserve-6895bd997-8c4w5"
Sep 07 12:07:20.342028 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:07:20.341945    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5141e977-a84d-46f6-adbe-da95fef14eb0-tls-certs\") pod \"test-model-af50c7-kserve-6895bd997-8c4w5\" (UID: \"5141e977-a84d-46f6-adbe-da95fef14eb0\") " pod="ai-tenant-e2e-inf-b-af50c7/test-model-af50c7-kserve-6895bd997-8c4w5"
Sep 07 12:07:20.342134 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:07:20.342033    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5141e977-a84d-46f6-adbe-da95fef14eb0-tmp-dir\") pod \"test-model-af50c7-kserve-6895bd997-8c4w5\" (UID: \"5141e977-a84d-46f6-adbe-da95fef14eb0\") " pod="ai-tenant-e2e-inf-b-af50c7/test-model-af50c7-kserve-6895bd997-8c4w5"
Sep 07 12:07:20.342134 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:07:20.342076    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5141e977-a84d-46f6-adbe-da95fef14eb0-home\") pod \"test-model-af50c7-kserve-6895bd997-8c4w5\" (UID: \"5141e977-a84d-46f6-adbe-da95fef14eb0\") " pod="ai-tenant-e2e-inf-b-af50c7/test-model-af50c7-kserve-6895bd997-8c4w5"
Sep 07 12:07:20.442937 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:07:20.442900    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5141e977-a84d-46f6-adbe-da95fef14eb0-tmp-dir\") pod \"test-model-af50c7-kserve-6895bd997-8c4w5\" (UID: \"5141e977-a84d-46f6-adbe-da95fef14eb0\") " pod="ai-tenant-e2e-inf-b-af50c7/test-model-af50c7-kserve-6895bd997-8c4w5"
Sep 07 12:07:20.443101 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:07:20.442964    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5141e977-a84d-46f6-adbe-da95fef14eb0-home\") pod \"test-model-af50c7-kserve-6895bd997-8c4w5\" (UID: \"5141e977-a84d-46f6-adbe-da95fef14eb0\") " pod="ai-tenant-e2e-inf-b-af50c7/test-model-af50c7-kserve-6895bd997-8c4w5"
Sep 07 12:07:20.443101 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:07:20.443018    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5141e977-a84d-46f6-adbe-da95fef14eb0-model-cache\") pod \"test-model-af50c7-kserve-6895bd997-8c4w5\" (UID: \"5141e977-a84d-46f6-adbe-da95fef14eb0\") " pod="ai-tenant-e2e-inf-b-af50c7/test-model-af50c7-kserve-6895bd997-8c4w5"
Sep 07 12:07:20.443101 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:07:20.443051    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5141e977-a84d-46f6-adbe-da95fef14eb0-dshm\") pod \"test-model-af50c7-kserve-6895bd997-8c4w5\" (UID: \"5141e977-a84d-46f6-adbe-da95fef14eb0\") " pod="ai-tenant-e2e-inf-b-af50c7/test-model-af50c7-kserve-6895bd997-8c4w5"
Sep 07 12:07:20.443101 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:07:20.443072    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-24vmv\" (UniqueName: \"kubernetes.io/projected/5141e977-a84d-46f6-adbe-da95fef14eb0-kube-api-access-24vmv\") pod \"test-model-af50c7-kserve-6895bd997-8c4w5\" (UID: \"5141e977-a84d-46f6-adbe-da95fef14eb0\") " pod="ai-tenant-e2e-inf-b-af50c7/test-model-af50c7-kserve-6895bd997-8c4w5"
Sep 07 12:07:20.443317 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:07:20.443282    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5141e977-a84d-46f6-adbe-da95fef14eb0-tls-certs\") pod \"test-model-af50c7-kserve-6895bd997-8c4w5\" (UID: \"5141e977-a84d-46f6-adbe-da95fef14eb0\") " pod="ai-tenant-e2e-inf-b-af50c7/test-model-af50c7-kserve-6895bd997-8c4w5"
Sep 07 12:07:20.443374 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:07:20.443312    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5141e977-a84d-46f6-adbe-da95fef14eb0-tmp-dir\") pod \"test-model-af50c7-kserve-6895bd997-8c4w5\" (UID: \"5141e977-a84d-46f6-adbe-da95fef14eb0\") " pod="ai-tenant-e2e-inf-b-af50c7/test-model-af50c7-kserve-6895bd997-8c4w5"
Sep 07 12:07:20.443436 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:07:20.443377    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5141e977-a84d-46f6-adbe-da95fef14eb0-home\") pod \"test-model-af50c7-kserve-6895bd997-8c4w5\" (UID: \"5141e977-a84d-46f6-adbe-da95fef14eb0\") " pod="ai-tenant-e2e-inf-b-af50c7/test-model-af50c7-kserve-6895bd997-8c4w5"
Sep 07 12:07:20.443506 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:07:20.443489    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5141e977-a84d-46f6-adbe-da95fef14eb0-model-cache\") pod \"test-model-af50c7-kserve-6895bd997-8c4w5\" (UID: \"5141e977-a84d-46f6-adbe-da95fef14eb0\") " pod="ai-tenant-e2e-inf-b-af50c7/test-model-af50c7-kserve-6895bd997-8c4w5"
Sep 07 12:07:20.445239 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:07:20.445212    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5141e977-a84d-46f6-adbe-da95fef14eb0-dshm\") pod \"test-model-af50c7-kserve-6895bd997-8c4w5\" (UID: \"5141e977-a84d-46f6-adbe-da95fef14eb0\") " pod="ai-tenant-e2e-inf-b-af50c7/test-model-af50c7-kserve-6895bd997-8c4w5"
Sep 07 12:07:20.445545 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:07:20.445529    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5141e977-a84d-46f6-adbe-da95fef14eb0-tls-certs\") pod \"test-model-af50c7-kserve-6895bd997-8c4w5\" (UID: \"5141e977-a84d-46f6-adbe-da95fef14eb0\") " pod="ai-tenant-e2e-inf-b-af50c7/test-model-af50c7-kserve-6895bd997-8c4w5"
Sep 07 12:07:20.449788 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:07:20.449765    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-24vmv\" (UniqueName: \"kubernetes.io/projected/5141e977-a84d-46f6-adbe-da95fef14eb0-kube-api-access-24vmv\") pod \"test-model-af50c7-kserve-6895bd997-8c4w5\" (UID: \"5141e977-a84d-46f6-adbe-da95fef14eb0\") " pod="ai-tenant-e2e-inf-b-af50c7/test-model-af50c7-kserve-6895bd997-8c4w5"
Sep 07 12:07:20.517977 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:07:20.517917    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-af50c7/test-model-af50c7-kserve-6895bd997-8c4w5"
Sep 07 12:07:20.786155 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:07:20.786077    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-af50c7/test-model-af50c7-kserve-6895bd997-8c4w5"]
Sep 07 12:07:20.789554 ip-10-0-135-173 kubenswrapper[2563]: W0907 12:07:20.789528    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5141e977_a84d_46f6_adbe_da95fef14eb0.slice/crio-d34cceafed640f567e0ce604cf662c7d095f1dbf1262b9cea3e01188faf35a83 WatchSource:0}: Error finding container d34cceafed640f567e0ce604cf662c7d095f1dbf1262b9cea3e01188faf35a83: Status 404 returned error can't find the container with id d34cceafed640f567e0ce604cf662c7d095f1dbf1262b9cea3e01188faf35a83
Sep 07 12:07:21.251877 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:07:21.251841    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-af50c7/test-model-af50c7-kserve-6895bd997-8c4w5" event={"ID":"5141e977-a84d-46f6-adbe-da95fef14eb0","Type":"ContainerStarted","Data":"5dd5c23fdd24bb2d28d9c76ff40ad8007469c495ff6c57c3be5e2be88d46eb5a"}
Sep 07 12:07:21.252277 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:07:21.251884    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-af50c7/test-model-af50c7-kserve-6895bd997-8c4w5" event={"ID":"5141e977-a84d-46f6-adbe-da95fef14eb0","Type":"ContainerStarted","Data":"d34cceafed640f567e0ce604cf662c7d095f1dbf1262b9cea3e01188faf35a83"}
Sep 07 12:07:21.268002 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:07:21.267953    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-b-af50c7/test-model-af50c7-kserve-6895bd997-8c4w5" podStartSLOduration=1.267938622 podStartE2EDuration="1.267938622s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 12:07:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 12:07:21.266647884 +0000 UTC m=+1212.785379697" watchObservedRunningTime="2026-09-07 12:07:21.267938622 +0000 UTC m=+1212.786670435"
Sep 07 12:07:23.627892 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:07:23.627857    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-explicit-3fa352/explicit-model-3fa352-kserve-6cc9dd6b8f-bzwmf"]
Sep 07 12:07:23.665669 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:07:23.665624    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:07:26.103369 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:07:26.103335    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-p4l4w"]
Sep 07 12:07:27.394598 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:07:27.394549    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:07:30.519000 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:07:30.518965    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-b-af50c7/test-model-af50c7-kserve-6895bd997-8c4w5"
Sep 07 12:07:30.519384 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:07:30.519016    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-b-af50c7/test-model-af50c7-kserve-6895bd997-8c4w5"
Sep 07 12:07:30.531217 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:07:30.531190    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-b-af50c7/test-model-af50c7-kserve-6895bd997-8c4w5"
Sep 07 12:07:31.312951 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:07:31.312913    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-b-af50c7/test-model-af50c7-kserve-6895bd997-8c4w5"
Sep 07 12:07:33.166832 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:07:33.166797    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:07:33.276275 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:07:33.276227    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:07:33.366814 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:07:33.366783    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:07:34.264433 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:07:34.264388    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:07:34.338318 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:07:34.338282    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:07:34.403789 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:07:34.403756    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:07:34.477211 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:07:34.477178    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:07:34.530509 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:07:34.530399    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:07:34.815481 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:07:34.815381    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:07:38.648588 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:07:38.648518    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-explicit-3fa352/explicit-model-3fa352-kserve-6cc9dd6b8f-bzwmf" podUID="fe65fc72-f5b9-4b7a-b1a8-372511f3dddc" containerName="main" containerID="cri-o://59fc58da243a653dbb79f8f48426dd8cd7e1c455c680e45a85237a0af94e6c71" gracePeriod=45
Sep 07 12:07:38.889762 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:07:38.889734    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-explicit-3fa352/explicit-model-3fa352-kserve-6cc9dd6b8f-bzwmf"
Sep 07 12:07:38.925661 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:07:38.925594    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fe65fc72-f5b9-4b7a-b1a8-372511f3dddc-tmp-dir\") pod \"fe65fc72-f5b9-4b7a-b1a8-372511f3dddc\" (UID: \"fe65fc72-f5b9-4b7a-b1a8-372511f3dddc\") "
Sep 07 12:07:38.925661 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:07:38.925632    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fe65fc72-f5b9-4b7a-b1a8-372511f3dddc-dshm\") pod \"fe65fc72-f5b9-4b7a-b1a8-372511f3dddc\" (UID: \"fe65fc72-f5b9-4b7a-b1a8-372511f3dddc\") "
Sep 07 12:07:38.925832 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:07:38.925671    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fe65fc72-f5b9-4b7a-b1a8-372511f3dddc-home\") pod \"fe65fc72-f5b9-4b7a-b1a8-372511f3dddc\" (UID: \"fe65fc72-f5b9-4b7a-b1a8-372511f3dddc\") "
Sep 07 12:07:38.925832 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:07:38.925708    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fe65fc72-f5b9-4b7a-b1a8-372511f3dddc-model-cache\") pod \"fe65fc72-f5b9-4b7a-b1a8-372511f3dddc\" (UID: \"fe65fc72-f5b9-4b7a-b1a8-372511f3dddc\") "
Sep 07 12:07:38.925832 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:07:38.925746    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jd4hw\" (UniqueName: \"kubernetes.io/projected/fe65fc72-f5b9-4b7a-b1a8-372511f3dddc-kube-api-access-jd4hw\") pod \"fe65fc72-f5b9-4b7a-b1a8-372511f3dddc\" (UID: \"fe65fc72-f5b9-4b7a-b1a8-372511f3dddc\") "
Sep 07 12:07:38.925832 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:07:38.925775    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fe65fc72-f5b9-4b7a-b1a8-372511f3dddc-tls-certs\") pod \"fe65fc72-f5b9-4b7a-b1a8-372511f3dddc\" (UID: \"fe65fc72-f5b9-4b7a-b1a8-372511f3dddc\") "
Sep 07 12:07:38.926042 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:07:38.925857    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fe65fc72-f5b9-4b7a-b1a8-372511f3dddc-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "fe65fc72-f5b9-4b7a-b1a8-372511f3dddc" (UID: "fe65fc72-f5b9-4b7a-b1a8-372511f3dddc"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 12:07:38.926042 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:07:38.925947    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fe65fc72-f5b9-4b7a-b1a8-372511f3dddc-home" (OuterVolumeSpecName: "home") pod "fe65fc72-f5b9-4b7a-b1a8-372511f3dddc" (UID: "fe65fc72-f5b9-4b7a-b1a8-372511f3dddc"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 12:07:38.926042 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:07:38.926004    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fe65fc72-f5b9-4b7a-b1a8-372511f3dddc-model-cache" (OuterVolumeSpecName: "model-cache") pod "fe65fc72-f5b9-4b7a-b1a8-372511f3dddc" (UID: "fe65fc72-f5b9-4b7a-b1a8-372511f3dddc"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 12:07:38.926212 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:07:38.926191    2563 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fe65fc72-f5b9-4b7a-b1a8-372511f3dddc-tmp-dir\") on node \"ip-10-0-135-173.ec2.internal\" DevicePath \"\""
Sep 07 12:07:38.926269 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:07:38.926218    2563 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fe65fc72-f5b9-4b7a-b1a8-372511f3dddc-home\") on node \"ip-10-0-135-173.ec2.internal\" DevicePath \"\""
Sep 07 12:07:38.926269 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:07:38.926232    2563 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fe65fc72-f5b9-4b7a-b1a8-372511f3dddc-model-cache\") on node \"ip-10-0-135-173.ec2.internal\" DevicePath \"\""
Sep 07 12:07:38.927990 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:07:38.927969    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fe65fc72-f5b9-4b7a-b1a8-372511f3dddc-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "fe65fc72-f5b9-4b7a-b1a8-372511f3dddc" (UID: "fe65fc72-f5b9-4b7a-b1a8-372511f3dddc"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 07 12:07:38.928079 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:07:38.927995    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fe65fc72-f5b9-4b7a-b1a8-372511f3dddc-kube-api-access-jd4hw" (OuterVolumeSpecName: "kube-api-access-jd4hw") pod "fe65fc72-f5b9-4b7a-b1a8-372511f3dddc" (UID: "fe65fc72-f5b9-4b7a-b1a8-372511f3dddc"). InnerVolumeSpecName "kube-api-access-jd4hw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 12:07:38.928079 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:07:38.928014    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fe65fc72-f5b9-4b7a-b1a8-372511f3dddc-dshm" (OuterVolumeSpecName: "dshm") pod "fe65fc72-f5b9-4b7a-b1a8-372511f3dddc" (UID: "fe65fc72-f5b9-4b7a-b1a8-372511f3dddc"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 12:07:39.027254 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:07:39.027232    2563 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fe65fc72-f5b9-4b7a-b1a8-372511f3dddc-dshm\") on node \"ip-10-0-135-173.ec2.internal\" DevicePath \"\""
Sep 07 12:07:39.027254 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:07:39.027256    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jd4hw\" (UniqueName: \"kubernetes.io/projected/fe65fc72-f5b9-4b7a-b1a8-372511f3dddc-kube-api-access-jd4hw\") on node \"ip-10-0-135-173.ec2.internal\" DevicePath \"\""
Sep 07 12:07:39.027395 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:07:39.027267    2563 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fe65fc72-f5b9-4b7a-b1a8-372511f3dddc-tls-certs\") on node \"ip-10-0-135-173.ec2.internal\" DevicePath \"\""
Sep 07 12:07:39.345463 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:07:39.345397    2563 generic.go:356] "Generic (PLEG): container finished" podID="fe65fc72-f5b9-4b7a-b1a8-372511f3dddc" containerID="59fc58da243a653dbb79f8f48426dd8cd7e1c455c680e45a85237a0af94e6c71" exitCode=0
Sep 07 12:07:39.345463 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:07:39.345441    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-explicit-3fa352/explicit-model-3fa352-kserve-6cc9dd6b8f-bzwmf" event={"ID":"fe65fc72-f5b9-4b7a-b1a8-372511f3dddc","Type":"ContainerDied","Data":"59fc58da243a653dbb79f8f48426dd8cd7e1c455c680e45a85237a0af94e6c71"}
Sep 07 12:07:39.345463 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:07:39.345463    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-explicit-3fa352/explicit-model-3fa352-kserve-6cc9dd6b8f-bzwmf"
Sep 07 12:07:39.345719 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:07:39.345485    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-explicit-3fa352/explicit-model-3fa352-kserve-6cc9dd6b8f-bzwmf" event={"ID":"fe65fc72-f5b9-4b7a-b1a8-372511f3dddc","Type":"ContainerDied","Data":"f5ad7f6794ece6cc8076ea713f162da2c937e6e43797ac35c9da58d26efbfac4"}
Sep 07 12:07:39.345719 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:07:39.345501    2563 scope.go:117] "RemoveContainer" containerID="59fc58da243a653dbb79f8f48426dd8cd7e1c455c680e45a85237a0af94e6c71"
Sep 07 12:07:39.354897 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:07:39.354881    2563 scope.go:117] "RemoveContainer" containerID="59fc58da243a653dbb79f8f48426dd8cd7e1c455c680e45a85237a0af94e6c71"
Sep 07 12:07:39.355145 ip-10-0-135-173 kubenswrapper[2563]: E0907 12:07:39.355126    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"59fc58da243a653dbb79f8f48426dd8cd7e1c455c680e45a85237a0af94e6c71\": container with ID starting with 59fc58da243a653dbb79f8f48426dd8cd7e1c455c680e45a85237a0af94e6c71 not found: ID does not exist" containerID="59fc58da243a653dbb79f8f48426dd8cd7e1c455c680e45a85237a0af94e6c71"
Sep 07 12:07:39.355188 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:07:39.355158    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"59fc58da243a653dbb79f8f48426dd8cd7e1c455c680e45a85237a0af94e6c71"} err="failed to get container status \"59fc58da243a653dbb79f8f48426dd8cd7e1c455c680e45a85237a0af94e6c71\": rpc error: code = NotFound desc = could not find container \"59fc58da243a653dbb79f8f48426dd8cd7e1c455c680e45a85237a0af94e6c71\": container with ID starting with 59fc58da243a653dbb79f8f48426dd8cd7e1c455c680e45a85237a0af94e6c71 not found: ID does not exist"
Sep 07 12:07:39.362905 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:07:39.362874    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-explicit-3fa352/explicit-model-3fa352-kserve-6cc9dd6b8f-bzwmf"]
Sep 07 12:07:39.364874 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:07:39.364854    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-explicit-3fa352/explicit-model-3fa352-kserve-6cc9dd6b8f-bzwmf"]
Sep 07 12:07:39.698322 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:07:39.698293    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-p4l4w"]
Sep 07 12:07:40.970214 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:07:40.970175    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:07:41.026493 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:07:41.026458    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:07:41.040039 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:07:41.040018    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fe65fc72-f5b9-4b7a-b1a8-372511f3dddc" path="/var/lib/kubelet/pods/fe65fc72-f5b9-4b7a-b1a8-372511f3dddc/volumes"
Sep 07 12:07:41.208723 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:07:41.208690    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-explicit-3fa352-openshift-default-6b44f857c6-pl9wf"]
Sep 07 12:07:41.208975 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:07:41.208951    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-explicit-3fa352-openshift-default-6b44f857c6-pl9wf" podUID="3eb6cef9-a0e7-43ba-a582-7a4329013a5b" containerName="istio-proxy" containerID="cri-o://29acfec6e51dc56b53b3e8a979d40cc7d5cbaed16f9a4574fd0ff4b0ace08052" gracePeriod=30
Sep 07 12:07:41.230390 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:07:41.230330    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:07:41.678219 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:07:41.678138    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:07:41.970386 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:07:41.970352    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:07:42.101741 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:07:42.101704    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:07:43.385668 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:07:43.385632    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:07:43.485494 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:07:43.485463    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:07:43.587337 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:07:43.587308    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:07:43.802148 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:07:43.802117    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:07:44.540025 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:07:44.539985    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:07:46.144871 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:07:46.144835    2563 patch_prober.go:28] interesting pod/e2e-explicit-3fa352-openshift-default-6b44f857c6-pl9wf container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 07 12:07:46.145240 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:07:46.144886    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-explicit-3fa352-openshift-default-6b44f857c6-pl9wf" podUID="3eb6cef9-a0e7-43ba-a582-7a4329013a5b" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 07 12:07:46.389901 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:07:46.389869    2563 generic.go:356] "Generic (PLEG): container finished" podID="3eb6cef9-a0e7-43ba-a582-7a4329013a5b" containerID="29acfec6e51dc56b53b3e8a979d40cc7d5cbaed16f9a4574fd0ff4b0ace08052" exitCode=0
Sep 07 12:07:46.390052 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:07:46.389940    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-3fa352-openshift-default-6b44f857c6-pl9wf" event={"ID":"3eb6cef9-a0e7-43ba-a582-7a4329013a5b","Type":"ContainerDied","Data":"29acfec6e51dc56b53b3e8a979d40cc7d5cbaed16f9a4574fd0ff4b0ace08052"}
Sep 07 12:07:46.390052 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:07:46.390014    2563 scope.go:117] "RemoveContainer" containerID="d1109feac35941c502bdd4cda60fe9058262040367a102b968da2dbbe6b90790"
Sep 07 12:07:46.455856 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:07:46.455834    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-explicit-3fa352-openshift-default-6b44f857c6-pl9wf"
Sep 07 12:07:46.598188 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:07:46.598145    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3eb6cef9-a0e7-43ba-a582-7a4329013a5b-istio-podinfo\") pod \"3eb6cef9-a0e7-43ba-a582-7a4329013a5b\" (UID: \"3eb6cef9-a0e7-43ba-a582-7a4329013a5b\") "
Sep 07 12:07:46.598354 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:07:46.598213    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3eb6cef9-a0e7-43ba-a582-7a4329013a5b-workload-certs\") pod \"3eb6cef9-a0e7-43ba-a582-7a4329013a5b\" (UID: \"3eb6cef9-a0e7-43ba-a582-7a4329013a5b\") "
Sep 07 12:07:46.598354 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:07:46.598246    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3eb6cef9-a0e7-43ba-a582-7a4329013a5b-istio-data\") pod \"3eb6cef9-a0e7-43ba-a582-7a4329013a5b\" (UID: \"3eb6cef9-a0e7-43ba-a582-7a4329013a5b\") "
Sep 07 12:07:46.598354 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:07:46.598292    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3eb6cef9-a0e7-43ba-a582-7a4329013a5b-istiod-ca-cert\") pod \"3eb6cef9-a0e7-43ba-a582-7a4329013a5b\" (UID: \"3eb6cef9-a0e7-43ba-a582-7a4329013a5b\") "
Sep 07 12:07:46.598354 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:07:46.598309    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3eb6cef9-a0e7-43ba-a582-7a4329013a5b-credential-socket\") pod \"3eb6cef9-a0e7-43ba-a582-7a4329013a5b\" (UID: \"3eb6cef9-a0e7-43ba-a582-7a4329013a5b\") "
Sep 07 12:07:46.598354 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:07:46.598347    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3eb6cef9-a0e7-43ba-a582-7a4329013a5b-istio-envoy\") pod \"3eb6cef9-a0e7-43ba-a582-7a4329013a5b\" (UID: \"3eb6cef9-a0e7-43ba-a582-7a4329013a5b\") "
Sep 07 12:07:46.598659 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:07:46.598378    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sp9fm\" (UniqueName: \"kubernetes.io/projected/3eb6cef9-a0e7-43ba-a582-7a4329013a5b-kube-api-access-sp9fm\") pod \"3eb6cef9-a0e7-43ba-a582-7a4329013a5b\" (UID: \"3eb6cef9-a0e7-43ba-a582-7a4329013a5b\") "
Sep 07 12:07:46.598659 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:07:46.598448    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3eb6cef9-a0e7-43ba-a582-7a4329013a5b-istio-token\") pod \"3eb6cef9-a0e7-43ba-a582-7a4329013a5b\" (UID: \"3eb6cef9-a0e7-43ba-a582-7a4329013a5b\") "
Sep 07 12:07:46.598659 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:07:46.598483    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3eb6cef9-a0e7-43ba-a582-7a4329013a5b-workload-socket\") pod \"3eb6cef9-a0e7-43ba-a582-7a4329013a5b\" (UID: \"3eb6cef9-a0e7-43ba-a582-7a4329013a5b\") "
Sep 07 12:07:46.598659 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:07:46.598545    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3eb6cef9-a0e7-43ba-a582-7a4329013a5b-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "3eb6cef9-a0e7-43ba-a582-7a4329013a5b" (UID: "3eb6cef9-a0e7-43ba-a582-7a4329013a5b"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 12:07:46.598659 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:07:46.598564    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3eb6cef9-a0e7-43ba-a582-7a4329013a5b-istio-data" (OuterVolumeSpecName: "istio-data") pod "3eb6cef9-a0e7-43ba-a582-7a4329013a5b" (UID: "3eb6cef9-a0e7-43ba-a582-7a4329013a5b"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 12:07:46.598919 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:07:46.598817    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3eb6cef9-a0e7-43ba-a582-7a4329013a5b-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "3eb6cef9-a0e7-43ba-a582-7a4329013a5b" (UID: "3eb6cef9-a0e7-43ba-a582-7a4329013a5b"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 12:07:46.598919 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:07:46.598857    2563 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3eb6cef9-a0e7-43ba-a582-7a4329013a5b-workload-certs\") on node \"ip-10-0-135-173.ec2.internal\" DevicePath \"\""
Sep 07 12:07:46.598919 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:07:46.598860    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3eb6cef9-a0e7-43ba-a582-7a4329013a5b-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "3eb6cef9-a0e7-43ba-a582-7a4329013a5b" (UID: "3eb6cef9-a0e7-43ba-a582-7a4329013a5b"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 07 12:07:46.598919 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:07:46.598879    2563 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3eb6cef9-a0e7-43ba-a582-7a4329013a5b-istio-data\") on node \"ip-10-0-135-173.ec2.internal\" DevicePath \"\""
Sep 07 12:07:46.599123 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:07:46.598979    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3eb6cef9-a0e7-43ba-a582-7a4329013a5b-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "3eb6cef9-a0e7-43ba-a582-7a4329013a5b" (UID: "3eb6cef9-a0e7-43ba-a582-7a4329013a5b"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 12:07:46.600748 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:07:46.600722    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/3eb6cef9-a0e7-43ba-a582-7a4329013a5b-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "3eb6cef9-a0e7-43ba-a582-7a4329013a5b" (UID: "3eb6cef9-a0e7-43ba-a582-7a4329013a5b"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 07 12:07:46.600748 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:07:46.600730    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3eb6cef9-a0e7-43ba-a582-7a4329013a5b-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "3eb6cef9-a0e7-43ba-a582-7a4329013a5b" (UID: "3eb6cef9-a0e7-43ba-a582-7a4329013a5b"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 12:07:46.600886 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:07:46.600746    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3eb6cef9-a0e7-43ba-a582-7a4329013a5b-kube-api-access-sp9fm" (OuterVolumeSpecName: "kube-api-access-sp9fm") pod "3eb6cef9-a0e7-43ba-a582-7a4329013a5b" (UID: "3eb6cef9-a0e7-43ba-a582-7a4329013a5b"). InnerVolumeSpecName "kube-api-access-sp9fm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 12:07:46.600886 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:07:46.600755    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3eb6cef9-a0e7-43ba-a582-7a4329013a5b-istio-token" (OuterVolumeSpecName: "istio-token") pod "3eb6cef9-a0e7-43ba-a582-7a4329013a5b" (UID: "3eb6cef9-a0e7-43ba-a582-7a4329013a5b"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 12:07:46.699589 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:07:46.699568    2563 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3eb6cef9-a0e7-43ba-a582-7a4329013a5b-istio-token\") on node \"ip-10-0-135-173.ec2.internal\" DevicePath \"\""
Sep 07 12:07:46.699718 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:07:46.699592    2563 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3eb6cef9-a0e7-43ba-a582-7a4329013a5b-workload-socket\") on node \"ip-10-0-135-173.ec2.internal\" DevicePath \"\""
Sep 07 12:07:46.699718 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:07:46.699602    2563 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3eb6cef9-a0e7-43ba-a582-7a4329013a5b-istio-podinfo\") on node \"ip-10-0-135-173.ec2.internal\" DevicePath \"\""
Sep 07 12:07:46.699718 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:07:46.699612    2563 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3eb6cef9-a0e7-43ba-a582-7a4329013a5b-istiod-ca-cert\") on node \"ip-10-0-135-173.ec2.internal\" DevicePath \"\""
Sep 07 12:07:46.699718 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:07:46.699620    2563 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3eb6cef9-a0e7-43ba-a582-7a4329013a5b-credential-socket\") on node \"ip-10-0-135-173.ec2.internal\" DevicePath \"\""
Sep 07 12:07:46.699718 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:07:46.699629    2563 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3eb6cef9-a0e7-43ba-a582-7a4329013a5b-istio-envoy\") on node \"ip-10-0-135-173.ec2.internal\" DevicePath \"\""
Sep 07 12:07:46.699718 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:07:46.699637    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sp9fm\" (UniqueName: \"kubernetes.io/projected/3eb6cef9-a0e7-43ba-a582-7a4329013a5b-kube-api-access-sp9fm\") on node \"ip-10-0-135-173.ec2.internal\" DevicePath \"\""
Sep 07 12:07:47.397023 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:07:47.396940    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-explicit-3fa352-openshift-default-6b44f857c6-pl9wf"
Sep 07 12:07:47.397023 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:07:47.396985    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-3fa352-openshift-default-6b44f857c6-pl9wf" event={"ID":"3eb6cef9-a0e7-43ba-a582-7a4329013a5b","Type":"ContainerDied","Data":"6b80ad41849e61219a2a71a46dd40fc700c982157dc6780796cbb4b0c7fed895"}
Sep 07 12:07:47.397524 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:07:47.397052    2563 scope.go:117] "RemoveContainer" containerID="29acfec6e51dc56b53b3e8a979d40cc7d5cbaed16f9a4574fd0ff4b0ace08052"
Sep 07 12:07:47.412227 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:07:47.412205    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-explicit-3fa352-openshift-default-6b44f857c6-pl9wf"]
Sep 07 12:07:47.417403 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:07:47.417382    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-explicit-3fa352-openshift-default-6b44f857c6-pl9wf"]
Sep 07 12:07:49.040448 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:07:49.040399    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3eb6cef9-a0e7-43ba-a582-7a4329013a5b" path="/var/lib/kubelet/pods/3eb6cef9-a0e7-43ba-a582-7a4329013a5b/volumes"
Sep 07 12:07:50.477344 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:07:50.477310    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-noten-cae4d5/noten-model-cae4d5-kserve-74f4ffc967-2jfpf"]
Sep 07 12:07:50.477747 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:07:50.477734    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3eb6cef9-a0e7-43ba-a582-7a4329013a5b" containerName="istio-proxy"
Sep 07 12:07:50.477788 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:07:50.477749    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="3eb6cef9-a0e7-43ba-a582-7a4329013a5b" containerName="istio-proxy"
Sep 07 12:07:50.477788 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:07:50.477761    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fe65fc72-f5b9-4b7a-b1a8-372511f3dddc" containerName="main"
Sep 07 12:07:50.477788 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:07:50.477766    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="fe65fc72-f5b9-4b7a-b1a8-372511f3dddc" containerName="main"
Sep 07 12:07:50.477902 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:07:50.477840    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="3eb6cef9-a0e7-43ba-a582-7a4329013a5b" containerName="istio-proxy"
Sep 07 12:07:50.477902 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:07:50.477848    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="fe65fc72-f5b9-4b7a-b1a8-372511f3dddc" containerName="main"
Sep 07 12:07:50.477902 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:07:50.477855    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="3eb6cef9-a0e7-43ba-a582-7a4329013a5b" containerName="istio-proxy"
Sep 07 12:07:50.482242 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:07:50.482217    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-noten-cae4d5/noten-model-cae4d5-kserve-74f4ffc967-2jfpf"
Sep 07 12:07:50.484649 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:07:50.484623    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-noten-cae4d5\"/\"noten-model-cae4d5-kserve-self-signed-certs\""
Sep 07 12:07:50.484783 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:07:50.484625    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-noten-cae4d5\"/\"default-dockercfg-v2qg5\""
Sep 07 12:07:50.484783 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:07:50.484668    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-noten-cae4d5\"/\"openshift-service-ca.crt\""
Sep 07 12:07:50.484783 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:07:50.484734    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-noten-cae4d5\"/\"kube-root-ca.crt\""
Sep 07 12:07:50.488568 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:07:50.488547    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-noten-cae4d5/noten-model-cae4d5-kserve-74f4ffc967-2jfpf"]
Sep 07 12:07:50.511031 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:07:50.511005    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:07:50.632664 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:07:50.632631    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/114d5a72-28a0-49a6-9285-4f8f743bf17e-tls-certs\") pod \"noten-model-cae4d5-kserve-74f4ffc967-2jfpf\" (UID: \"114d5a72-28a0-49a6-9285-4f8f743bf17e\") " pod="ai-tenant-e2e-noten-cae4d5/noten-model-cae4d5-kserve-74f4ffc967-2jfpf"
Sep 07 12:07:50.632835 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:07:50.632678    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/114d5a72-28a0-49a6-9285-4f8f743bf17e-dshm\") pod \"noten-model-cae4d5-kserve-74f4ffc967-2jfpf\" (UID: \"114d5a72-28a0-49a6-9285-4f8f743bf17e\") " pod="ai-tenant-e2e-noten-cae4d5/noten-model-cae4d5-kserve-74f4ffc967-2jfpf"
Sep 07 12:07:50.632835 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:07:50.632704    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-56qgf\" (UniqueName: \"kubernetes.io/projected/114d5a72-28a0-49a6-9285-4f8f743bf17e-kube-api-access-56qgf\") pod \"noten-model-cae4d5-kserve-74f4ffc967-2jfpf\" (UID: \"114d5a72-28a0-49a6-9285-4f8f743bf17e\") " pod="ai-tenant-e2e-noten-cae4d5/noten-model-cae4d5-kserve-74f4ffc967-2jfpf"
Sep 07 12:07:50.632835 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:07:50.632754    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/114d5a72-28a0-49a6-9285-4f8f743bf17e-model-cache\") pod \"noten-model-cae4d5-kserve-74f4ffc967-2jfpf\" (UID: \"114d5a72-28a0-49a6-9285-4f8f743bf17e\") " pod="ai-tenant-e2e-noten-cae4d5/noten-model-cae4d5-kserve-74f4ffc967-2jfpf"
Sep 07 12:07:50.632835 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:07:50.632785    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/114d5a72-28a0-49a6-9285-4f8f743bf17e-home\") pod \"noten-model-cae4d5-kserve-74f4ffc967-2jfpf\" (UID: \"114d5a72-28a0-49a6-9285-4f8f743bf17e\") " pod="ai-tenant-e2e-noten-cae4d5/noten-model-cae4d5-kserve-74f4ffc967-2jfpf"
Sep 07 12:07:50.632835 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:07:50.632808    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/114d5a72-28a0-49a6-9285-4f8f743bf17e-tmp-dir\") pod \"noten-model-cae4d5-kserve-74f4ffc967-2jfpf\" (UID: \"114d5a72-28a0-49a6-9285-4f8f743bf17e\") " pod="ai-tenant-e2e-noten-cae4d5/noten-model-cae4d5-kserve-74f4ffc967-2jfpf"
Sep 07 12:07:50.684579 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:07:50.684538    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:07:50.733925 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:07:50.733854    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/114d5a72-28a0-49a6-9285-4f8f743bf17e-tls-certs\") pod \"noten-model-cae4d5-kserve-74f4ffc967-2jfpf\" (UID: \"114d5a72-28a0-49a6-9285-4f8f743bf17e\") " pod="ai-tenant-e2e-noten-cae4d5/noten-model-cae4d5-kserve-74f4ffc967-2jfpf"
Sep 07 12:07:50.733925 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:07:50.733894    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/114d5a72-28a0-49a6-9285-4f8f743bf17e-dshm\") pod \"noten-model-cae4d5-kserve-74f4ffc967-2jfpf\" (UID: \"114d5a72-28a0-49a6-9285-4f8f743bf17e\") " pod="ai-tenant-e2e-noten-cae4d5/noten-model-cae4d5-kserve-74f4ffc967-2jfpf"
Sep 07 12:07:50.733925 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:07:50.733912    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-56qgf\" (UniqueName: \"kubernetes.io/projected/114d5a72-28a0-49a6-9285-4f8f743bf17e-kube-api-access-56qgf\") pod \"noten-model-cae4d5-kserve-74f4ffc967-2jfpf\" (UID: \"114d5a72-28a0-49a6-9285-4f8f743bf17e\") " pod="ai-tenant-e2e-noten-cae4d5/noten-model-cae4d5-kserve-74f4ffc967-2jfpf"
Sep 07 12:07:50.734139 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:07:50.733947    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/114d5a72-28a0-49a6-9285-4f8f743bf17e-model-cache\") pod \"noten-model-cae4d5-kserve-74f4ffc967-2jfpf\" (UID: \"114d5a72-28a0-49a6-9285-4f8f743bf17e\") " pod="ai-tenant-e2e-noten-cae4d5/noten-model-cae4d5-kserve-74f4ffc967-2jfpf"
Sep 07 12:07:50.734139 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:07:50.733969    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/114d5a72-28a0-49a6-9285-4f8f743bf17e-home\") pod \"noten-model-cae4d5-kserve-74f4ffc967-2jfpf\" (UID: \"114d5a72-28a0-49a6-9285-4f8f743bf17e\") " pod="ai-tenant-e2e-noten-cae4d5/noten-model-cae4d5-kserve-74f4ffc967-2jfpf"
Sep 07 12:07:50.734139 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:07:50.733987    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/114d5a72-28a0-49a6-9285-4f8f743bf17e-tmp-dir\") pod \"noten-model-cae4d5-kserve-74f4ffc967-2jfpf\" (UID: \"114d5a72-28a0-49a6-9285-4f8f743bf17e\") " pod="ai-tenant-e2e-noten-cae4d5/noten-model-cae4d5-kserve-74f4ffc967-2jfpf"
Sep 07 12:07:50.734370 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:07:50.734337    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/114d5a72-28a0-49a6-9285-4f8f743bf17e-tmp-dir\") pod \"noten-model-cae4d5-kserve-74f4ffc967-2jfpf\" (UID: \"114d5a72-28a0-49a6-9285-4f8f743bf17e\") " pod="ai-tenant-e2e-noten-cae4d5/noten-model-cae4d5-kserve-74f4ffc967-2jfpf"
Sep 07 12:07:50.734459 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:07:50.734393    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/114d5a72-28a0-49a6-9285-4f8f743bf17e-model-cache\") pod \"noten-model-cae4d5-kserve-74f4ffc967-2jfpf\" (UID: \"114d5a72-28a0-49a6-9285-4f8f743bf17e\") " pod="ai-tenant-e2e-noten-cae4d5/noten-model-cae4d5-kserve-74f4ffc967-2jfpf"
Sep 07 12:07:50.734459 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:07:50.734405    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/114d5a72-28a0-49a6-9285-4f8f743bf17e-home\") pod \"noten-model-cae4d5-kserve-74f4ffc967-2jfpf\" (UID: \"114d5a72-28a0-49a6-9285-4f8f743bf17e\") " pod="ai-tenant-e2e-noten-cae4d5/noten-model-cae4d5-kserve-74f4ffc967-2jfpf"
Sep 07 12:07:50.736210 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:07:50.736186    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/114d5a72-28a0-49a6-9285-4f8f743bf17e-dshm\") pod \"noten-model-cae4d5-kserve-74f4ffc967-2jfpf\" (UID: \"114d5a72-28a0-49a6-9285-4f8f743bf17e\") " pod="ai-tenant-e2e-noten-cae4d5/noten-model-cae4d5-kserve-74f4ffc967-2jfpf"
Sep 07 12:07:50.736513 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:07:50.736492    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/114d5a72-28a0-49a6-9285-4f8f743bf17e-tls-certs\") pod \"noten-model-cae4d5-kserve-74f4ffc967-2jfpf\" (UID: \"114d5a72-28a0-49a6-9285-4f8f743bf17e\") " pod="ai-tenant-e2e-noten-cae4d5/noten-model-cae4d5-kserve-74f4ffc967-2jfpf"
Sep 07 12:07:50.741232 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:07:50.741212    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-56qgf\" (UniqueName: \"kubernetes.io/projected/114d5a72-28a0-49a6-9285-4f8f743bf17e-kube-api-access-56qgf\") pod \"noten-model-cae4d5-kserve-74f4ffc967-2jfpf\" (UID: \"114d5a72-28a0-49a6-9285-4f8f743bf17e\") " pod="ai-tenant-e2e-noten-cae4d5/noten-model-cae4d5-kserve-74f4ffc967-2jfpf"
Sep 07 12:07:50.791177 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:07:50.791145    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:07:50.792042 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:07:50.792024    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-noten-cae4d5/noten-model-cae4d5-kserve-74f4ffc967-2jfpf"
Sep 07 12:07:50.885922 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:07:50.885888    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:07:51.063983 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:07:51.063960    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-noten-cae4d5/noten-model-cae4d5-kserve-74f4ffc967-2jfpf"]
Sep 07 12:07:51.066648 ip-10-0-135-173 kubenswrapper[2563]: W0907 12:07:51.066623    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod114d5a72_28a0_49a6_9285_4f8f743bf17e.slice/crio-5304bb034f341be0e21376446c9457bcb353cc8d979ec0c6b74e62726208c7a1 WatchSource:0}: Error finding container 5304bb034f341be0e21376446c9457bcb353cc8d979ec0c6b74e62726208c7a1: Status 404 returned error can't find the container with id 5304bb034f341be0e21376446c9457bcb353cc8d979ec0c6b74e62726208c7a1
Sep 07 12:07:51.132899 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:07:51.132861    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:07:51.418654 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:07:51.418568    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-noten-cae4d5/noten-model-cae4d5-kserve-74f4ffc967-2jfpf" event={"ID":"114d5a72-28a0-49a6-9285-4f8f743bf17e","Type":"ContainerStarted","Data":"d688731184a1e89de9696328f16ccc7dcd61c2e78ccfd5b88653ef23466b0806"}
Sep 07 12:07:51.418654 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:07:51.418605    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-noten-cae4d5/noten-model-cae4d5-kserve-74f4ffc967-2jfpf" event={"ID":"114d5a72-28a0-49a6-9285-4f8f743bf17e","Type":"ContainerStarted","Data":"5304bb034f341be0e21376446c9457bcb353cc8d979ec0c6b74e62726208c7a1"}
Sep 07 12:07:51.434615 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:07:51.434568    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-noten-cae4d5/noten-model-cae4d5-kserve-74f4ffc967-2jfpf" podStartSLOduration=1.434553142 podStartE2EDuration="1.434553142s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 12:07:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 12:07:51.433435534 +0000 UTC m=+1242.952167342" watchObservedRunningTime="2026-09-07 12:07:51.434553142 +0000 UTC m=+1242.953284955"
Sep 07 12:07:51.554334 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:07:51.554299    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:07:51.654494 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:07:51.654457    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:07:51.757659 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:07:51.757627    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:07:53.148166 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:07:53.148131    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:07:55.603499 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:07:55.603469    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-p4l4w"]
Sep 07 12:07:59.223660 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:07:59.223626    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-noten-cae4d5/noten-model-cae4d5-kserve-74f4ffc967-2jfpf"]
Sep 07 12:07:59.384471 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:07:59.384433    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:07:59.450704 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:07:59.450665    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:07:59.613889 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:07:59.613800    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:07:59.720487 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:07:59.720459    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:08:00.004892 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:08:00.004860    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:08:00.792548 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:08:00.792514    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-noten-cae4d5/noten-model-cae4d5-kserve-74f4ffc967-2jfpf"
Sep 07 12:08:04.769818 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:08:04.769783    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:08:05.699009 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:08:05.698974    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:08:05.767841 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:08:05.767809    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:08:05.911919 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:08:05.911872    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:08:06.294587 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:08:06.294553    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:08:06.642719 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:08:06.642621    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:08:09.096544 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:08:09.096505    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:08:09.245791 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:08:09.245710    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:08:10.297390 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:08:10.297351    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:08:10.398057 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:08:10.398028    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:08:10.512068 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:08:10.512032    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:08:10.739285 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:08:10.739258    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:08:10.844053 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:08:10.844017    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:08:11.102683 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:08:11.102591    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-p4l4w"]
Sep 07 12:08:14.243863 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:08:14.243812    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-noten-cae4d5/noten-model-cae4d5-kserve-74f4ffc967-2jfpf" podUID="114d5a72-28a0-49a6-9285-4f8f743bf17e" containerName="main" containerID="cri-o://d688731184a1e89de9696328f16ccc7dcd61c2e78ccfd5b88653ef23466b0806" gracePeriod=45
Sep 07 12:08:14.493249 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:08:14.493224    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-noten-cae4d5/noten-model-cae4d5-kserve-74f4ffc967-2jfpf"
Sep 07 12:08:14.535332 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:08:14.535267    2563 generic.go:356] "Generic (PLEG): container finished" podID="114d5a72-28a0-49a6-9285-4f8f743bf17e" containerID="d688731184a1e89de9696328f16ccc7dcd61c2e78ccfd5b88653ef23466b0806" exitCode=0
Sep 07 12:08:14.535332 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:08:14.535307    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-noten-cae4d5/noten-model-cae4d5-kserve-74f4ffc967-2jfpf"
Sep 07 12:08:14.535504 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:08:14.535352    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-noten-cae4d5/noten-model-cae4d5-kserve-74f4ffc967-2jfpf" event={"ID":"114d5a72-28a0-49a6-9285-4f8f743bf17e","Type":"ContainerDied","Data":"d688731184a1e89de9696328f16ccc7dcd61c2e78ccfd5b88653ef23466b0806"}
Sep 07 12:08:14.535504 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:08:14.535397    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-noten-cae4d5/noten-model-cae4d5-kserve-74f4ffc967-2jfpf" event={"ID":"114d5a72-28a0-49a6-9285-4f8f743bf17e","Type":"ContainerDied","Data":"5304bb034f341be0e21376446c9457bcb353cc8d979ec0c6b74e62726208c7a1"}
Sep 07 12:08:14.535504 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:08:14.535439    2563 scope.go:117] "RemoveContainer" containerID="d688731184a1e89de9696328f16ccc7dcd61c2e78ccfd5b88653ef23466b0806"
Sep 07 12:08:14.543919 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:08:14.543890    2563 scope.go:117] "RemoveContainer" containerID="d688731184a1e89de9696328f16ccc7dcd61c2e78ccfd5b88653ef23466b0806"
Sep 07 12:08:14.544205 ip-10-0-135-173 kubenswrapper[2563]: E0907 12:08:14.544177    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d688731184a1e89de9696328f16ccc7dcd61c2e78ccfd5b88653ef23466b0806\": container with ID starting with d688731184a1e89de9696328f16ccc7dcd61c2e78ccfd5b88653ef23466b0806 not found: ID does not exist" containerID="d688731184a1e89de9696328f16ccc7dcd61c2e78ccfd5b88653ef23466b0806"
Sep 07 12:08:14.544274 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:08:14.544207    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d688731184a1e89de9696328f16ccc7dcd61c2e78ccfd5b88653ef23466b0806"} err="failed to get container status \"d688731184a1e89de9696328f16ccc7dcd61c2e78ccfd5b88653ef23466b0806\": rpc error: code = NotFound desc = could not find container \"d688731184a1e89de9696328f16ccc7dcd61c2e78ccfd5b88653ef23466b0806\": container with ID starting with d688731184a1e89de9696328f16ccc7dcd61c2e78ccfd5b88653ef23466b0806 not found: ID does not exist"
Sep 07 12:08:14.560535 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:08:14.560512    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/114d5a72-28a0-49a6-9285-4f8f743bf17e-home\") pod \"114d5a72-28a0-49a6-9285-4f8f743bf17e\" (UID: \"114d5a72-28a0-49a6-9285-4f8f743bf17e\") "
Sep 07 12:08:14.560622 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:08:14.560566    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/114d5a72-28a0-49a6-9285-4f8f743bf17e-dshm\") pod \"114d5a72-28a0-49a6-9285-4f8f743bf17e\" (UID: \"114d5a72-28a0-49a6-9285-4f8f743bf17e\") "
Sep 07 12:08:14.560622 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:08:14.560599    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/114d5a72-28a0-49a6-9285-4f8f743bf17e-model-cache\") pod \"114d5a72-28a0-49a6-9285-4f8f743bf17e\" (UID: \"114d5a72-28a0-49a6-9285-4f8f743bf17e\") "
Sep 07 12:08:14.560712 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:08:14.560624    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-56qgf\" (UniqueName: \"kubernetes.io/projected/114d5a72-28a0-49a6-9285-4f8f743bf17e-kube-api-access-56qgf\") pod \"114d5a72-28a0-49a6-9285-4f8f743bf17e\" (UID: \"114d5a72-28a0-49a6-9285-4f8f743bf17e\") "
Sep 07 12:08:14.560712 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:08:14.560692    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/114d5a72-28a0-49a6-9285-4f8f743bf17e-tls-certs\") pod \"114d5a72-28a0-49a6-9285-4f8f743bf17e\" (UID: \"114d5a72-28a0-49a6-9285-4f8f743bf17e\") "
Sep 07 12:08:14.560808 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:08:14.560736    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/114d5a72-28a0-49a6-9285-4f8f743bf17e-home" (OuterVolumeSpecName: "home") pod "114d5a72-28a0-49a6-9285-4f8f743bf17e" (UID: "114d5a72-28a0-49a6-9285-4f8f743bf17e"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 12:08:14.560808 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:08:14.560797    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/114d5a72-28a0-49a6-9285-4f8f743bf17e-tmp-dir\") pod \"114d5a72-28a0-49a6-9285-4f8f743bf17e\" (UID: \"114d5a72-28a0-49a6-9285-4f8f743bf17e\") "
Sep 07 12:08:14.560917 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:08:14.560851    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/114d5a72-28a0-49a6-9285-4f8f743bf17e-model-cache" (OuterVolumeSpecName: "model-cache") pod "114d5a72-28a0-49a6-9285-4f8f743bf17e" (UID: "114d5a72-28a0-49a6-9285-4f8f743bf17e"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 12:08:14.561085 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:08:14.561059    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/114d5a72-28a0-49a6-9285-4f8f743bf17e-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "114d5a72-28a0-49a6-9285-4f8f743bf17e" (UID: "114d5a72-28a0-49a6-9285-4f8f743bf17e"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 12:08:14.561178 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:08:14.561157    2563 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/114d5a72-28a0-49a6-9285-4f8f743bf17e-model-cache\") on node \"ip-10-0-135-173.ec2.internal\" DevicePath \"\""
Sep 07 12:08:14.561227 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:08:14.561183    2563 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/114d5a72-28a0-49a6-9285-4f8f743bf17e-tmp-dir\") on node \"ip-10-0-135-173.ec2.internal\" DevicePath \"\""
Sep 07 12:08:14.561227 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:08:14.561196    2563 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/114d5a72-28a0-49a6-9285-4f8f743bf17e-home\") on node \"ip-10-0-135-173.ec2.internal\" DevicePath \"\""
Sep 07 12:08:14.562803 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:08:14.562775    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/114d5a72-28a0-49a6-9285-4f8f743bf17e-dshm" (OuterVolumeSpecName: "dshm") pod "114d5a72-28a0-49a6-9285-4f8f743bf17e" (UID: "114d5a72-28a0-49a6-9285-4f8f743bf17e"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 12:08:14.562900 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:08:14.562853    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/114d5a72-28a0-49a6-9285-4f8f743bf17e-kube-api-access-56qgf" (OuterVolumeSpecName: "kube-api-access-56qgf") pod "114d5a72-28a0-49a6-9285-4f8f743bf17e" (UID: "114d5a72-28a0-49a6-9285-4f8f743bf17e"). InnerVolumeSpecName "kube-api-access-56qgf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 12:08:14.562963 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:08:14.562935    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/114d5a72-28a0-49a6-9285-4f8f743bf17e-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "114d5a72-28a0-49a6-9285-4f8f743bf17e" (UID: "114d5a72-28a0-49a6-9285-4f8f743bf17e"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 07 12:08:14.661665 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:08:14.661636    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-56qgf\" (UniqueName: \"kubernetes.io/projected/114d5a72-28a0-49a6-9285-4f8f743bf17e-kube-api-access-56qgf\") on node \"ip-10-0-135-173.ec2.internal\" DevicePath \"\""
Sep 07 12:08:14.661665 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:08:14.661663    2563 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/114d5a72-28a0-49a6-9285-4f8f743bf17e-tls-certs\") on node \"ip-10-0-135-173.ec2.internal\" DevicePath \"\""
Sep 07 12:08:14.661665 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:08:14.661672    2563 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/114d5a72-28a0-49a6-9285-4f8f743bf17e-dshm\") on node \"ip-10-0-135-173.ec2.internal\" DevicePath \"\""
Sep 07 12:08:14.856003 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:08:14.855973    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-noten-cae4d5/noten-model-cae4d5-kserve-74f4ffc967-2jfpf"]
Sep 07 12:08:14.859252 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:08:14.859230    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-noten-cae4d5/noten-model-cae4d5-kserve-74f4ffc967-2jfpf"]
Sep 07 12:08:15.040310 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:08:15.040269    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="114d5a72-28a0-49a6-9285-4f8f743bf17e" path="/var/lib/kubelet/pods/114d5a72-28a0-49a6-9285-4f8f743bf17e/volumes"
Sep 07 12:08:24.603966 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:08:24.603934    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-p4l4w"]
Sep 07 12:08:24.678632 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:08:24.678603    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-c7d751/test-model-c7d751-kserve-67b985587-d97xz"]
Sep 07 12:08:24.700194 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:08:24.700168    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:08:25.640839 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:08:25.640806    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-af50c7/test-model-af50c7-kserve-6895bd997-8c4w5"]
Sep 07 12:08:25.659158 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:08:25.659127    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:08:25.849811 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:08:25.849773    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:08:31.129092 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:08:31.129054    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:08:33.798598 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:08:33.798567    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-p4l4w"]
Sep 07 12:08:34.010552 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:08:34.010519    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:08:34.617049 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:08:34.617010    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:08:34.716400 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:08:34.716364    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:08:35.036993 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:08:35.036955    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:08:35.897629 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:08:35.897599    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:08:36.724622 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:08:36.724572    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:08:36.775683 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:08:36.775643    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:08:37.035198 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:08:37.035101    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:08:39.699631 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:08:39.699584    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-a-c7d751/test-model-c7d751-kserve-67b985587-d97xz" podUID="cd6bd43f-698a-491f-90b1-0a0d0519939d" containerName="main" containerID="cri-o://f7ff381125aae03c0070a2031bae62bfdae6aaf8d13f6f057824057df678727d" gracePeriod=45
Sep 07 12:08:39.945335 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:08:39.945312    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-c7d751/test-model-c7d751-kserve-67b985587-d97xz"
Sep 07 12:08:40.094294 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:08:40.094216    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cd6bd43f-698a-491f-90b1-0a0d0519939d-home\") pod \"cd6bd43f-698a-491f-90b1-0a0d0519939d\" (UID: \"cd6bd43f-698a-491f-90b1-0a0d0519939d\") "
Sep 07 12:08:40.094294 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:08:40.094254    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cd6bd43f-698a-491f-90b1-0a0d0519939d-dshm\") pod \"cd6bd43f-698a-491f-90b1-0a0d0519939d\" (UID: \"cd6bd43f-698a-491f-90b1-0a0d0519939d\") "
Sep 07 12:08:40.094520 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:08:40.094308    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-72t4d\" (UniqueName: \"kubernetes.io/projected/cd6bd43f-698a-491f-90b1-0a0d0519939d-kube-api-access-72t4d\") pod \"cd6bd43f-698a-491f-90b1-0a0d0519939d\" (UID: \"cd6bd43f-698a-491f-90b1-0a0d0519939d\") "
Sep 07 12:08:40.094520 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:08:40.094330    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cd6bd43f-698a-491f-90b1-0a0d0519939d-tls-certs\") pod \"cd6bd43f-698a-491f-90b1-0a0d0519939d\" (UID: \"cd6bd43f-698a-491f-90b1-0a0d0519939d\") "
Sep 07 12:08:40.094520 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:08:40.094388    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cd6bd43f-698a-491f-90b1-0a0d0519939d-model-cache\") pod \"cd6bd43f-698a-491f-90b1-0a0d0519939d\" (UID: \"cd6bd43f-698a-491f-90b1-0a0d0519939d\") "
Sep 07 12:08:40.094520 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:08:40.094441    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cd6bd43f-698a-491f-90b1-0a0d0519939d-tmp-dir\") pod \"cd6bd43f-698a-491f-90b1-0a0d0519939d\" (UID: \"cd6bd43f-698a-491f-90b1-0a0d0519939d\") "
Sep 07 12:08:40.094744 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:08:40.094525    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cd6bd43f-698a-491f-90b1-0a0d0519939d-home" (OuterVolumeSpecName: "home") pod "cd6bd43f-698a-491f-90b1-0a0d0519939d" (UID: "cd6bd43f-698a-491f-90b1-0a0d0519939d"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 12:08:40.094744 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:08:40.094659    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cd6bd43f-698a-491f-90b1-0a0d0519939d-model-cache" (OuterVolumeSpecName: "model-cache") pod "cd6bd43f-698a-491f-90b1-0a0d0519939d" (UID: "cd6bd43f-698a-491f-90b1-0a0d0519939d"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 12:08:40.094849 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:08:40.094754    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cd6bd43f-698a-491f-90b1-0a0d0519939d-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "cd6bd43f-698a-491f-90b1-0a0d0519939d" (UID: "cd6bd43f-698a-491f-90b1-0a0d0519939d"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 12:08:40.094890 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:08:40.094844    2563 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cd6bd43f-698a-491f-90b1-0a0d0519939d-model-cache\") on node \"ip-10-0-135-173.ec2.internal\" DevicePath \"\""
Sep 07 12:08:40.094890 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:08:40.094862    2563 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cd6bd43f-698a-491f-90b1-0a0d0519939d-home\") on node \"ip-10-0-135-173.ec2.internal\" DevicePath \"\""
Sep 07 12:08:40.096621 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:08:40.096602    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cd6bd43f-698a-491f-90b1-0a0d0519939d-kube-api-access-72t4d" (OuterVolumeSpecName: "kube-api-access-72t4d") pod "cd6bd43f-698a-491f-90b1-0a0d0519939d" (UID: "cd6bd43f-698a-491f-90b1-0a0d0519939d"). InnerVolumeSpecName "kube-api-access-72t4d". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 12:08:40.096790 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:08:40.096768    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cd6bd43f-698a-491f-90b1-0a0d0519939d-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "cd6bd43f-698a-491f-90b1-0a0d0519939d" (UID: "cd6bd43f-698a-491f-90b1-0a0d0519939d"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 07 12:08:40.096850 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:08:40.096828    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cd6bd43f-698a-491f-90b1-0a0d0519939d-dshm" (OuterVolumeSpecName: "dshm") pod "cd6bd43f-698a-491f-90b1-0a0d0519939d" (UID: "cd6bd43f-698a-491f-90b1-0a0d0519939d"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 12:08:40.196011 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:08:40.195983    2563 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cd6bd43f-698a-491f-90b1-0a0d0519939d-tmp-dir\") on node \"ip-10-0-135-173.ec2.internal\" DevicePath \"\""
Sep 07 12:08:40.196011 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:08:40.196004    2563 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cd6bd43f-698a-491f-90b1-0a0d0519939d-dshm\") on node \"ip-10-0-135-173.ec2.internal\" DevicePath \"\""
Sep 07 12:08:40.196011 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:08:40.196014    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-72t4d\" (UniqueName: \"kubernetes.io/projected/cd6bd43f-698a-491f-90b1-0a0d0519939d-kube-api-access-72t4d\") on node \"ip-10-0-135-173.ec2.internal\" DevicePath \"\""
Sep 07 12:08:40.196272 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:08:40.196025    2563 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cd6bd43f-698a-491f-90b1-0a0d0519939d-tls-certs\") on node \"ip-10-0-135-173.ec2.internal\" DevicePath \"\""
Sep 07 12:08:40.660887 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:08:40.660843    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-b-af50c7/test-model-af50c7-kserve-6895bd997-8c4w5" podUID="5141e977-a84d-46f6-adbe-da95fef14eb0" containerName="main" containerID="cri-o://5dd5c23fdd24bb2d28d9c76ff40ad8007469c495ff6c57c3be5e2be88d46eb5a" gracePeriod=45
Sep 07 12:08:40.666869 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:08:40.666843    2563 generic.go:356] "Generic (PLEG): container finished" podID="cd6bd43f-698a-491f-90b1-0a0d0519939d" containerID="f7ff381125aae03c0070a2031bae62bfdae6aaf8d13f6f057824057df678727d" exitCode=0
Sep 07 12:08:40.667000 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:08:40.666882    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-c7d751/test-model-c7d751-kserve-67b985587-d97xz"
Sep 07 12:08:40.667000 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:08:40.666910    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-c7d751/test-model-c7d751-kserve-67b985587-d97xz" event={"ID":"cd6bd43f-698a-491f-90b1-0a0d0519939d","Type":"ContainerDied","Data":"f7ff381125aae03c0070a2031bae62bfdae6aaf8d13f6f057824057df678727d"}
Sep 07 12:08:40.667000 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:08:40.666950    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-c7d751/test-model-c7d751-kserve-67b985587-d97xz" event={"ID":"cd6bd43f-698a-491f-90b1-0a0d0519939d","Type":"ContainerDied","Data":"e07a5b6ad18162364a953d241ad6f1d0f6706d008e82dd639385d57066fe8a13"}
Sep 07 12:08:40.667000 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:08:40.666972    2563 scope.go:117] "RemoveContainer" containerID="f7ff381125aae03c0070a2031bae62bfdae6aaf8d13f6f057824057df678727d"
Sep 07 12:08:40.718806 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:08:40.718759    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:08:40.736809 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:08:40.736783    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-c7d751/test-model-c7d751-kserve-67b985587-d97xz"]
Sep 07 12:08:40.740567 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:08:40.740543    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-a-c7d751/test-model-c7d751-kserve-67b985587-d97xz"]
Sep 07 12:08:40.771705 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:08:40.771562    2563 scope.go:117] "RemoveContainer" containerID="f7ff381125aae03c0070a2031bae62bfdae6aaf8d13f6f057824057df678727d"
Sep 07 12:08:40.771922 ip-10-0-135-173 kubenswrapper[2563]: E0907 12:08:40.771902    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f7ff381125aae03c0070a2031bae62bfdae6aaf8d13f6f057824057df678727d\": container with ID starting with f7ff381125aae03c0070a2031bae62bfdae6aaf8d13f6f057824057df678727d not found: ID does not exist" containerID="f7ff381125aae03c0070a2031bae62bfdae6aaf8d13f6f057824057df678727d"
Sep 07 12:08:40.772046 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:08:40.771933    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f7ff381125aae03c0070a2031bae62bfdae6aaf8d13f6f057824057df678727d"} err="failed to get container status \"f7ff381125aae03c0070a2031bae62bfdae6aaf8d13f6f057824057df678727d\": rpc error: code = NotFound desc = could not find container \"f7ff381125aae03c0070a2031bae62bfdae6aaf8d13f6f057824057df678727d\": container with ID starting with f7ff381125aae03c0070a2031bae62bfdae6aaf8d13f6f057824057df678727d not found: ID does not exist"
Sep 07 12:08:40.819020 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:08:40.818975    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:08:40.916797 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:08:40.916736    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-af50c7/test-model-af50c7-kserve-6895bd997-8c4w5"
Sep 07 12:08:41.004519 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:08:41.004483    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5141e977-a84d-46f6-adbe-da95fef14eb0-tmp-dir\") pod \"5141e977-a84d-46f6-adbe-da95fef14eb0\" (UID: \"5141e977-a84d-46f6-adbe-da95fef14eb0\") "
Sep 07 12:08:41.004691 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:08:41.004560    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5141e977-a84d-46f6-adbe-da95fef14eb0-home\") pod \"5141e977-a84d-46f6-adbe-da95fef14eb0\" (UID: \"5141e977-a84d-46f6-adbe-da95fef14eb0\") "
Sep 07 12:08:41.004691 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:08:41.004593    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5141e977-a84d-46f6-adbe-da95fef14eb0-tls-certs\") pod \"5141e977-a84d-46f6-adbe-da95fef14eb0\" (UID: \"5141e977-a84d-46f6-adbe-da95fef14eb0\") "
Sep 07 12:08:41.004691 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:08:41.004630    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5141e977-a84d-46f6-adbe-da95fef14eb0-model-cache\") pod \"5141e977-a84d-46f6-adbe-da95fef14eb0\" (UID: \"5141e977-a84d-46f6-adbe-da95fef14eb0\") "
Sep 07 12:08:41.004691 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:08:41.004668    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5141e977-a84d-46f6-adbe-da95fef14eb0-dshm\") pod \"5141e977-a84d-46f6-adbe-da95fef14eb0\" (UID: \"5141e977-a84d-46f6-adbe-da95fef14eb0\") "
Sep 07 12:08:41.004925 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:08:41.004691    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5141e977-a84d-46f6-adbe-da95fef14eb0-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "5141e977-a84d-46f6-adbe-da95fef14eb0" (UID: "5141e977-a84d-46f6-adbe-da95fef14eb0"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 12:08:41.004925 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:08:41.004704    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-24vmv\" (UniqueName: \"kubernetes.io/projected/5141e977-a84d-46f6-adbe-da95fef14eb0-kube-api-access-24vmv\") pod \"5141e977-a84d-46f6-adbe-da95fef14eb0\" (UID: \"5141e977-a84d-46f6-adbe-da95fef14eb0\") "
Sep 07 12:08:41.004925 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:08:41.004750    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5141e977-a84d-46f6-adbe-da95fef14eb0-home" (OuterVolumeSpecName: "home") pod "5141e977-a84d-46f6-adbe-da95fef14eb0" (UID: "5141e977-a84d-46f6-adbe-da95fef14eb0"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 12:08:41.004925 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:08:41.004913    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5141e977-a84d-46f6-adbe-da95fef14eb0-model-cache" (OuterVolumeSpecName: "model-cache") pod "5141e977-a84d-46f6-adbe-da95fef14eb0" (UID: "5141e977-a84d-46f6-adbe-da95fef14eb0"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 12:08:41.005149 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:08:41.005088    2563 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5141e977-a84d-46f6-adbe-da95fef14eb0-home\") on node \"ip-10-0-135-173.ec2.internal\" DevicePath \"\""
Sep 07 12:08:41.005149 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:08:41.005101    2563 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5141e977-a84d-46f6-adbe-da95fef14eb0-model-cache\") on node \"ip-10-0-135-173.ec2.internal\" DevicePath \"\""
Sep 07 12:08:41.005149 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:08:41.005111    2563 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5141e977-a84d-46f6-adbe-da95fef14eb0-tmp-dir\") on node \"ip-10-0-135-173.ec2.internal\" DevicePath \"\""
Sep 07 12:08:41.006938 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:08:41.006912    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5141e977-a84d-46f6-adbe-da95fef14eb0-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "5141e977-a84d-46f6-adbe-da95fef14eb0" (UID: "5141e977-a84d-46f6-adbe-da95fef14eb0"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 07 12:08:41.007055 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:08:41.006945    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5141e977-a84d-46f6-adbe-da95fef14eb0-dshm" (OuterVolumeSpecName: "dshm") pod "5141e977-a84d-46f6-adbe-da95fef14eb0" (UID: "5141e977-a84d-46f6-adbe-da95fef14eb0"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 12:08:41.007055 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:08:41.006967    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5141e977-a84d-46f6-adbe-da95fef14eb0-kube-api-access-24vmv" (OuterVolumeSpecName: "kube-api-access-24vmv") pod "5141e977-a84d-46f6-adbe-da95fef14eb0" (UID: "5141e977-a84d-46f6-adbe-da95fef14eb0"). InnerVolumeSpecName "kube-api-access-24vmv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 12:08:41.040648 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:08:41.040625    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cd6bd43f-698a-491f-90b1-0a0d0519939d" path="/var/lib/kubelet/pods/cd6bd43f-698a-491f-90b1-0a0d0519939d/volumes"
Sep 07 12:08:41.106545 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:08:41.106514    2563 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5141e977-a84d-46f6-adbe-da95fef14eb0-tls-certs\") on node \"ip-10-0-135-173.ec2.internal\" DevicePath \"\""
Sep 07 12:08:41.106545 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:08:41.106539    2563 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5141e977-a84d-46f6-adbe-da95fef14eb0-dshm\") on node \"ip-10-0-135-173.ec2.internal\" DevicePath \"\""
Sep 07 12:08:41.106695 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:08:41.106553    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-24vmv\" (UniqueName: \"kubernetes.io/projected/5141e977-a84d-46f6-adbe-da95fef14eb0-kube-api-access-24vmv\") on node \"ip-10-0-135-173.ec2.internal\" DevicePath \"\""
Sep 07 12:08:41.303433 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:08:41.303392    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:08:41.674634 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:08:41.674550    2563 generic.go:356] "Generic (PLEG): container finished" podID="5141e977-a84d-46f6-adbe-da95fef14eb0" containerID="5dd5c23fdd24bb2d28d9c76ff40ad8007469c495ff6c57c3be5e2be88d46eb5a" exitCode=0
Sep 07 12:08:41.674634 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:08:41.674595    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-af50c7/test-model-af50c7-kserve-6895bd997-8c4w5"
Sep 07 12:08:41.674846 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:08:41.674627    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-af50c7/test-model-af50c7-kserve-6895bd997-8c4w5" event={"ID":"5141e977-a84d-46f6-adbe-da95fef14eb0","Type":"ContainerDied","Data":"5dd5c23fdd24bb2d28d9c76ff40ad8007469c495ff6c57c3be5e2be88d46eb5a"}
Sep 07 12:08:41.674846 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:08:41.674674    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-af50c7/test-model-af50c7-kserve-6895bd997-8c4w5" event={"ID":"5141e977-a84d-46f6-adbe-da95fef14eb0","Type":"ContainerDied","Data":"d34cceafed640f567e0ce604cf662c7d095f1dbf1262b9cea3e01188faf35a83"}
Sep 07 12:08:41.674846 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:08:41.674692    2563 scope.go:117] "RemoveContainer" containerID="5dd5c23fdd24bb2d28d9c76ff40ad8007469c495ff6c57c3be5e2be88d46eb5a"
Sep 07 12:08:41.683093 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:08:41.683076    2563 scope.go:117] "RemoveContainer" containerID="5dd5c23fdd24bb2d28d9c76ff40ad8007469c495ff6c57c3be5e2be88d46eb5a"
Sep 07 12:08:41.683333 ip-10-0-135-173 kubenswrapper[2563]: E0907 12:08:41.683312    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5dd5c23fdd24bb2d28d9c76ff40ad8007469c495ff6c57c3be5e2be88d46eb5a\": container with ID starting with 5dd5c23fdd24bb2d28d9c76ff40ad8007469c495ff6c57c3be5e2be88d46eb5a not found: ID does not exist" containerID="5dd5c23fdd24bb2d28d9c76ff40ad8007469c495ff6c57c3be5e2be88d46eb5a"
Sep 07 12:08:41.683389 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:08:41.683340    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5dd5c23fdd24bb2d28d9c76ff40ad8007469c495ff6c57c3be5e2be88d46eb5a"} err="failed to get container status \"5dd5c23fdd24bb2d28d9c76ff40ad8007469c495ff6c57c3be5e2be88d46eb5a\": rpc error: code = NotFound desc = could not find container \"5dd5c23fdd24bb2d28d9c76ff40ad8007469c495ff6c57c3be5e2be88d46eb5a\": container with ID starting with 5dd5c23fdd24bb2d28d9c76ff40ad8007469c495ff6c57c3be5e2be88d46eb5a not found: ID does not exist"
Sep 07 12:08:41.687601 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:08:41.687581    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-af50c7/test-model-af50c7-kserve-6895bd997-8c4w5"]
Sep 07 12:08:41.693746 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:08:41.693724    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-b-af50c7/test-model-af50c7-kserve-6895bd997-8c4w5"]
Sep 07 12:08:42.220746 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:08:42.220698    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:08:43.040515 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:08:43.040482    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5141e977-a84d-46f6-adbe-da95fef14eb0" path="/var/lib/kubelet/pods/5141e977-a84d-46f6-adbe-da95fef14eb0/volumes"
Sep 07 12:08:43.233379 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:08:43.233343    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:08:43.435772 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:08:43.435690    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:08:43.494905 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:08:43.494879    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:08:43.855269 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:08:43.855234    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:08:45.998410 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:08:45.998375    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-p4l4w"]
Sep 07 12:08:51.193974 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:08:51.193939    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:08:51.781341 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:08:51.781307    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:08:51.848962 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:08:51.848928    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:08:51.948868 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:08:51.948835    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:08:52.049489 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:08:52.049389    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:08:52.349969 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:08:52.349875    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:08:54.013787 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:08:54.013753    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:08:54.582334 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:08:54.582299    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-af50c7-96f78668b-6bxt4"]
Sep 07 12:08:54.582635 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:08:54.582562    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-af50c7-96f78668b-6bxt4" podUID="a10d4b3d-8750-4ce2-baa6-68fade76984a" containerName="maas-api" containerID="cri-o://93e675e2a6d3ff82eaf351483123e06067193c6641401f3a2d859fea4328700d" gracePeriod=30
Sep 07 12:08:54.609968 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:08:54.609943    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:08:54.710935 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:08:54.710905    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:08:54.742201 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:08:54.742170    2563 generic.go:356] "Generic (PLEG): container finished" podID="a10d4b3d-8750-4ce2-baa6-68fade76984a" containerID="93e675e2a6d3ff82eaf351483123e06067193c6641401f3a2d859fea4328700d" exitCode=0
Sep 07 12:08:54.742340 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:08:54.742221    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-af50c7-96f78668b-6bxt4" event={"ID":"a10d4b3d-8750-4ce2-baa6-68fade76984a","Type":"ContainerDied","Data":"93e675e2a6d3ff82eaf351483123e06067193c6641401f3a2d859fea4328700d"}
Sep 07 12:08:54.804956 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:08:54.804926    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:08:54.826086 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:08:54.826058    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-af50c7-96f78668b-6bxt4"
Sep 07 12:08:54.828520 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:08:54.828501    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a10d4b3d-8750-4ce2-baa6-68fade76984a-maas-api-tls\") pod \"a10d4b3d-8750-4ce2-baa6-68fade76984a\" (UID: \"a10d4b3d-8750-4ce2-baa6-68fade76984a\") "
Sep 07 12:08:54.828622 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:08:54.828564    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a10d4b3d-8750-4ce2-baa6-68fade76984a-metrics-certs\") pod \"a10d4b3d-8750-4ce2-baa6-68fade76984a\" (UID: \"a10d4b3d-8750-4ce2-baa6-68fade76984a\") "
Sep 07 12:08:54.828622 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:08:54.828614    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pnbqd\" (UniqueName: \"kubernetes.io/projected/a10d4b3d-8750-4ce2-baa6-68fade76984a-kube-api-access-pnbqd\") pod \"a10d4b3d-8750-4ce2-baa6-68fade76984a\" (UID: \"a10d4b3d-8750-4ce2-baa6-68fade76984a\") "
Sep 07 12:08:54.828741 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:08:54.828684    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a10d4b3d-8750-4ce2-baa6-68fade76984a-openshift-service-ca\") pod \"a10d4b3d-8750-4ce2-baa6-68fade76984a\" (UID: \"a10d4b3d-8750-4ce2-baa6-68fade76984a\") "
Sep 07 12:08:54.829097 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:08:54.829062    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a10d4b3d-8750-4ce2-baa6-68fade76984a-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "a10d4b3d-8750-4ce2-baa6-68fade76984a" (UID: "a10d4b3d-8750-4ce2-baa6-68fade76984a"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 07 12:08:54.830604 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:08:54.830582    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a10d4b3d-8750-4ce2-baa6-68fade76984a-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "a10d4b3d-8750-4ce2-baa6-68fade76984a" (UID: "a10d4b3d-8750-4ce2-baa6-68fade76984a"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 07 12:08:54.830684 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:08:54.830603    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a10d4b3d-8750-4ce2-baa6-68fade76984a-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "a10d4b3d-8750-4ce2-baa6-68fade76984a" (UID: "a10d4b3d-8750-4ce2-baa6-68fade76984a"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 07 12:08:54.830684 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:08:54.830615    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a10d4b3d-8750-4ce2-baa6-68fade76984a-kube-api-access-pnbqd" (OuterVolumeSpecName: "kube-api-access-pnbqd") pod "a10d4b3d-8750-4ce2-baa6-68fade76984a" (UID: "a10d4b3d-8750-4ce2-baa6-68fade76984a"). InnerVolumeSpecName "kube-api-access-pnbqd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 12:08:54.860616 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:08:54.860571    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:08:54.929738 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:08:54.929709    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pnbqd\" (UniqueName: \"kubernetes.io/projected/a10d4b3d-8750-4ce2-baa6-68fade76984a-kube-api-access-pnbqd\") on node \"ip-10-0-135-173.ec2.internal\" DevicePath \"\""
Sep 07 12:08:54.929871 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:08:54.929742    2563 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a10d4b3d-8750-4ce2-baa6-68fade76984a-openshift-service-ca\") on node \"ip-10-0-135-173.ec2.internal\" DevicePath \"\""
Sep 07 12:08:54.929871 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:08:54.929757    2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a10d4b3d-8750-4ce2-baa6-68fade76984a-maas-api-tls\") on node \"ip-10-0-135-173.ec2.internal\" DevicePath \"\""
Sep 07 12:08:54.929871 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:08:54.929772    2563 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a10d4b3d-8750-4ce2-baa6-68fade76984a-metrics-certs\") on node \"ip-10-0-135-173.ec2.internal\" DevicePath \"\""
Sep 07 12:08:55.157194 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:08:55.157112    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:08:55.510818 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:08:55.510786    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-p4l4w"]
Sep 07 12:08:55.748179 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:08:55.748153    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-af50c7-96f78668b-6bxt4"
Sep 07 12:08:55.748348 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:08:55.748202    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-af50c7-96f78668b-6bxt4" event={"ID":"a10d4b3d-8750-4ce2-baa6-68fade76984a","Type":"ContainerDied","Data":"abdefd08f60d3cdccdcaf4a8d4fd34f4ed514f4c4b4ede515c75722cd8af233b"}
Sep 07 12:08:55.753545 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:08:55.748924    2563 scope.go:117] "RemoveContainer" containerID="93e675e2a6d3ff82eaf351483123e06067193c6641401f3a2d859fea4328700d"
Sep 07 12:08:55.769730 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:08:55.769708    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-af50c7-96f78668b-6bxt4"]
Sep 07 12:08:55.772853 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:08:55.772832    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-af50c7-96f78668b-6bxt4"]
Sep 07 12:08:57.040206 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:08:57.040173    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a10d4b3d-8750-4ce2-baa6-68fade76984a" path="/var/lib/kubelet/pods/a10d4b3d-8750-4ce2-baa6-68fade76984a/volumes"
Sep 07 12:08:58.232784 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:08:58.232740    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:08:58.335292 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:08:58.335255    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:08:58.477013 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:08:58.476981    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:08:58.802748 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:08:58.802704    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:08:58.904894 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:08:58.904860    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:08:59.011880 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:08:59.011843    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:09:00.854930 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:09:00.854899    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:09:00.931545 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:09:00.931501    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:09:01.081080 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:09:01.081052    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-af50c7-openshift-default-67788bbd-v68lr"]
Sep 07 12:09:01.081285 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:09:01.081258    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-af50c7-openshift-default-67788bbd-v68lr" podUID="bc21582c-d7f9-48e4-9309-ffeefa3f7f03" containerName="istio-proxy" containerID="cri-o://9b88ef6e84375dd2043a9bb56c56e164029c3d12fdb2faa60380a2075a9bf783" gracePeriod=30
Sep 07 12:09:01.102730 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:09:01.102693    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:09:01.619837 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:09:01.619799    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:09:01.651221 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:09:01.651191    2563 patch_prober.go:28] interesting pod/e2e-inf-b-af50c7-openshift-default-67788bbd-v68lr container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 07 12:09:01.651359 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:09:01.651243    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-b-af50c7-openshift-default-67788bbd-v68lr" podUID="bc21582c-d7f9-48e4-9309-ffeefa3f7f03" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 07 12:09:01.897547 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:09:01.897448    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:09:02.692468 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:09:02.692399    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:09:03.004881 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:09:03.004834    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-p4l4w"]
Sep 07 12:09:04.818297 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:09:04.818261    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:09:04.868338 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:09:04.868303    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:09:04.936699 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:09:04.936656    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:09:05.035399 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:09:05.035362    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:09:05.140864 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:09:05.140767    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:09:05.434670 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:09:05.434586    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:09:05.526399 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:09:05.526373    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:09:06.328009 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:09:06.327984    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-af50c7-openshift-default-67788bbd-v68lr"
Sep 07 12:09:06.423428 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:09:06.423336    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bc21582c-d7f9-48e4-9309-ffeefa3f7f03-istio-envoy\") pod \"bc21582c-d7f9-48e4-9309-ffeefa3f7f03\" (UID: \"bc21582c-d7f9-48e4-9309-ffeefa3f7f03\") "
Sep 07 12:09:06.423580 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:09:06.423428    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bc21582c-d7f9-48e4-9309-ffeefa3f7f03-istio-token\") pod \"bc21582c-d7f9-48e4-9309-ffeefa3f7f03\" (UID: \"bc21582c-d7f9-48e4-9309-ffeefa3f7f03\") "
Sep 07 12:09:06.423580 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:09:06.423460    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bc21582c-d7f9-48e4-9309-ffeefa3f7f03-istio-podinfo\") pod \"bc21582c-d7f9-48e4-9309-ffeefa3f7f03\" (UID: \"bc21582c-d7f9-48e4-9309-ffeefa3f7f03\") "
Sep 07 12:09:06.423580 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:09:06.423493    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2tgdf\" (UniqueName: \"kubernetes.io/projected/bc21582c-d7f9-48e4-9309-ffeefa3f7f03-kube-api-access-2tgdf\") pod \"bc21582c-d7f9-48e4-9309-ffeefa3f7f03\" (UID: \"bc21582c-d7f9-48e4-9309-ffeefa3f7f03\") "
Sep 07 12:09:06.423580 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:09:06.423532    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bc21582c-d7f9-48e4-9309-ffeefa3f7f03-workload-socket\") pod \"bc21582c-d7f9-48e4-9309-ffeefa3f7f03\" (UID: \"bc21582c-d7f9-48e4-9309-ffeefa3f7f03\") "
Sep 07 12:09:06.423580 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:09:06.423559    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bc21582c-d7f9-48e4-9309-ffeefa3f7f03-workload-certs\") pod \"bc21582c-d7f9-48e4-9309-ffeefa3f7f03\" (UID: \"bc21582c-d7f9-48e4-9309-ffeefa3f7f03\") "
Sep 07 12:09:06.423834 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:09:06.423583    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bc21582c-d7f9-48e4-9309-ffeefa3f7f03-credential-socket\") pod \"bc21582c-d7f9-48e4-9309-ffeefa3f7f03\" (UID: \"bc21582c-d7f9-48e4-9309-ffeefa3f7f03\") "
Sep 07 12:09:06.423834 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:09:06.423623    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bc21582c-d7f9-48e4-9309-ffeefa3f7f03-istiod-ca-cert\") pod \"bc21582c-d7f9-48e4-9309-ffeefa3f7f03\" (UID: \"bc21582c-d7f9-48e4-9309-ffeefa3f7f03\") "
Sep 07 12:09:06.423834 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:09:06.423703    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bc21582c-d7f9-48e4-9309-ffeefa3f7f03-istio-data\") pod \"bc21582c-d7f9-48e4-9309-ffeefa3f7f03\" (UID: \"bc21582c-d7f9-48e4-9309-ffeefa3f7f03\") "
Sep 07 12:09:06.423984 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:09:06.423879    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bc21582c-d7f9-48e4-9309-ffeefa3f7f03-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "bc21582c-d7f9-48e4-9309-ffeefa3f7f03" (UID: "bc21582c-d7f9-48e4-9309-ffeefa3f7f03"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 12:09:06.424080 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:09:06.424058    2563 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bc21582c-d7f9-48e4-9309-ffeefa3f7f03-workload-socket\") on node \"ip-10-0-135-173.ec2.internal\" DevicePath \"\""
Sep 07 12:09:06.424147 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:09:06.424115    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bc21582c-d7f9-48e4-9309-ffeefa3f7f03-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "bc21582c-d7f9-48e4-9309-ffeefa3f7f03" (UID: "bc21582c-d7f9-48e4-9309-ffeefa3f7f03"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 12:09:06.424200 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:09:06.424133    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bc21582c-d7f9-48e4-9309-ffeefa3f7f03-istio-data" (OuterVolumeSpecName: "istio-data") pod "bc21582c-d7f9-48e4-9309-ffeefa3f7f03" (UID: "bc21582c-d7f9-48e4-9309-ffeefa3f7f03"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 12:09:06.424253 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:09:06.424194    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bc21582c-d7f9-48e4-9309-ffeefa3f7f03-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "bc21582c-d7f9-48e4-9309-ffeefa3f7f03" (UID: "bc21582c-d7f9-48e4-9309-ffeefa3f7f03"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 12:09:06.424399 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:09:06.424378    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bc21582c-d7f9-48e4-9309-ffeefa3f7f03-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "bc21582c-d7f9-48e4-9309-ffeefa3f7f03" (UID: "bc21582c-d7f9-48e4-9309-ffeefa3f7f03"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 07 12:09:06.425599 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:09:06.425575    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/bc21582c-d7f9-48e4-9309-ffeefa3f7f03-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "bc21582c-d7f9-48e4-9309-ffeefa3f7f03" (UID: "bc21582c-d7f9-48e4-9309-ffeefa3f7f03"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 07 12:09:06.425690 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:09:06.425658    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bc21582c-d7f9-48e4-9309-ffeefa3f7f03-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "bc21582c-d7f9-48e4-9309-ffeefa3f7f03" (UID: "bc21582c-d7f9-48e4-9309-ffeefa3f7f03"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 12:09:06.426003 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:09:06.425984    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bc21582c-d7f9-48e4-9309-ffeefa3f7f03-istio-token" (OuterVolumeSpecName: "istio-token") pod "bc21582c-d7f9-48e4-9309-ffeefa3f7f03" (UID: "bc21582c-d7f9-48e4-9309-ffeefa3f7f03"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 12:09:06.426098 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:09:06.426009    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bc21582c-d7f9-48e4-9309-ffeefa3f7f03-kube-api-access-2tgdf" (OuterVolumeSpecName: "kube-api-access-2tgdf") pod "bc21582c-d7f9-48e4-9309-ffeefa3f7f03" (UID: "bc21582c-d7f9-48e4-9309-ffeefa3f7f03"). InnerVolumeSpecName "kube-api-access-2tgdf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 12:09:06.524618 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:09:06.524589    2563 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bc21582c-d7f9-48e4-9309-ffeefa3f7f03-istio-data\") on node \"ip-10-0-135-173.ec2.internal\" DevicePath \"\""
Sep 07 12:09:06.524618 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:09:06.524614    2563 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bc21582c-d7f9-48e4-9309-ffeefa3f7f03-istio-envoy\") on node \"ip-10-0-135-173.ec2.internal\" DevicePath \"\""
Sep 07 12:09:06.524618 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:09:06.524622    2563 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bc21582c-d7f9-48e4-9309-ffeefa3f7f03-istio-token\") on node \"ip-10-0-135-173.ec2.internal\" DevicePath \"\""
Sep 07 12:09:06.524818 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:09:06.524630    2563 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bc21582c-d7f9-48e4-9309-ffeefa3f7f03-istio-podinfo\") on node \"ip-10-0-135-173.ec2.internal\" DevicePath \"\""
Sep 07 12:09:06.524818 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:09:06.524639    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2tgdf\" (UniqueName: \"kubernetes.io/projected/bc21582c-d7f9-48e4-9309-ffeefa3f7f03-kube-api-access-2tgdf\") on node \"ip-10-0-135-173.ec2.internal\" DevicePath \"\""
Sep 07 12:09:06.524818 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:09:06.524647    2563 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bc21582c-d7f9-48e4-9309-ffeefa3f7f03-workload-certs\") on node \"ip-10-0-135-173.ec2.internal\" DevicePath \"\""
Sep 07 12:09:06.524818 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:09:06.524656    2563 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bc21582c-d7f9-48e4-9309-ffeefa3f7f03-credential-socket\") on node \"ip-10-0-135-173.ec2.internal\" DevicePath \"\""
Sep 07 12:09:06.524818 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:09:06.524664    2563 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bc21582c-d7f9-48e4-9309-ffeefa3f7f03-istiod-ca-cert\") on node \"ip-10-0-135-173.ec2.internal\" DevicePath \"\""
Sep 07 12:09:06.556363 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:09:06.556332    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-491296-6b6d75c56c-2w7xh"]
Sep 07 12:09:06.556760 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:09:06.556743    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bc21582c-d7f9-48e4-9309-ffeefa3f7f03" containerName="istio-proxy"
Sep 07 12:09:06.556843 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:09:06.556762    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="bc21582c-d7f9-48e4-9309-ffeefa3f7f03" containerName="istio-proxy"
Sep 07 12:09:06.556843 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:09:06.556780    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cd6bd43f-698a-491f-90b1-0a0d0519939d" containerName="main"
Sep 07 12:09:06.556843 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:09:06.556788    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="cd6bd43f-698a-491f-90b1-0a0d0519939d" containerName="main"
Sep 07 12:09:06.556843 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:09:06.556810    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bc21582c-d7f9-48e4-9309-ffeefa3f7f03" containerName="istio-proxy"
Sep 07 12:09:06.556843 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:09:06.556818    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="bc21582c-d7f9-48e4-9309-ffeefa3f7f03" containerName="istio-proxy"
Sep 07 12:09:06.556843 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:09:06.556837    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5141e977-a84d-46f6-adbe-da95fef14eb0" containerName="main"
Sep 07 12:09:06.557191 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:09:06.556846    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="5141e977-a84d-46f6-adbe-da95fef14eb0" containerName="main"
Sep 07 12:09:06.557191 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:09:06.556856    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="114d5a72-28a0-49a6-9285-4f8f743bf17e" containerName="main"
Sep 07 12:09:06.557191 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:09:06.556864    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="114d5a72-28a0-49a6-9285-4f8f743bf17e" containerName="main"
Sep 07 12:09:06.557191 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:09:06.556883    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a10d4b3d-8750-4ce2-baa6-68fade76984a" containerName="maas-api"
Sep 07 12:09:06.557191 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:09:06.556892    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="a10d4b3d-8750-4ce2-baa6-68fade76984a" containerName="maas-api"
Sep 07 12:09:06.557191 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:09:06.556907    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3eb6cef9-a0e7-43ba-a582-7a4329013a5b" containerName="istio-proxy"
Sep 07 12:09:06.557191 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:09:06.556916    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="3eb6cef9-a0e7-43ba-a582-7a4329013a5b" containerName="istio-proxy"
Sep 07 12:09:06.557191 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:09:06.557015    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="cd6bd43f-698a-491f-90b1-0a0d0519939d" containerName="main"
Sep 07 12:09:06.557191 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:09:06.557029    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="bc21582c-d7f9-48e4-9309-ffeefa3f7f03" containerName="istio-proxy"
Sep 07 12:09:06.557191 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:09:06.557039    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="5141e977-a84d-46f6-adbe-da95fef14eb0" containerName="main"
Sep 07 12:09:06.557191 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:09:06.557050    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="a10d4b3d-8750-4ce2-baa6-68fade76984a" containerName="maas-api"
Sep 07 12:09:06.557191 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:09:06.557059    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="114d5a72-28a0-49a6-9285-4f8f743bf17e" containerName="main"
Sep 07 12:09:06.560664 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:09:06.560645    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-491296-6b6d75c56c-2w7xh"
Sep 07 12:09:06.562266 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:09:06.562244    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Sep 07 12:09:06.562368 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:09:06.562325    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Sep 07 12:09:06.562452 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:09:06.562367    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 07 12:09:06.562513 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:09:06.562450    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-w3-491296-serving-cert\""
Sep 07 12:09:06.562701 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:09:06.562687    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-p4vz9\""
Sep 07 12:09:06.570384 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:09:06.570362    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-491296-6b6d75c56c-2w7xh"]
Sep 07 12:09:06.611957 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:09:06.611930    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:09:06.625538 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:09:06.625514    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/647bfbc0-36fb-4c24-b1f7-a2531da941cb-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w3-491296-6b6d75c56c-2w7xh\" (UID: \"647bfbc0-36fb-4c24-b1f7-a2531da941cb\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-491296-6b6d75c56c-2w7xh"
Sep 07 12:09:06.625637 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:09:06.625567    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/647bfbc0-36fb-4c24-b1f7-a2531da941cb-maas-api-tls\") pod \"maas-api-e2e-shared-a-w3-491296-6b6d75c56c-2w7xh\" (UID: \"647bfbc0-36fb-4c24-b1f7-a2531da941cb\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-491296-6b6d75c56c-2w7xh"
Sep 07 12:09:06.625637 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:09:06.625615    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9zj9h\" (UniqueName: \"kubernetes.io/projected/647bfbc0-36fb-4c24-b1f7-a2531da941cb-kube-api-access-9zj9h\") pod \"maas-api-e2e-shared-a-w3-491296-6b6d75c56c-2w7xh\" (UID: \"647bfbc0-36fb-4c24-b1f7-a2531da941cb\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-491296-6b6d75c56c-2w7xh"
Sep 07 12:09:06.625731 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:09:06.625677    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/647bfbc0-36fb-4c24-b1f7-a2531da941cb-metrics-certs\") pod \"maas-api-e2e-shared-a-w3-491296-6b6d75c56c-2w7xh\" (UID: \"647bfbc0-36fb-4c24-b1f7-a2531da941cb\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-491296-6b6d75c56c-2w7xh"
Sep 07 12:09:06.711902 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:09:06.711872    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:09:06.726973 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:09:06.726951    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/647bfbc0-36fb-4c24-b1f7-a2531da941cb-metrics-certs\") pod \"maas-api-e2e-shared-a-w3-491296-6b6d75c56c-2w7xh\" (UID: \"647bfbc0-36fb-4c24-b1f7-a2531da941cb\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-491296-6b6d75c56c-2w7xh"
Sep 07 12:09:06.727051 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:09:06.726981    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/647bfbc0-36fb-4c24-b1f7-a2531da941cb-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w3-491296-6b6d75c56c-2w7xh\" (UID: \"647bfbc0-36fb-4c24-b1f7-a2531da941cb\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-491296-6b6d75c56c-2w7xh"
Sep 07 12:09:06.727051 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:09:06.727019    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/647bfbc0-36fb-4c24-b1f7-a2531da941cb-maas-api-tls\") pod \"maas-api-e2e-shared-a-w3-491296-6b6d75c56c-2w7xh\" (UID: \"647bfbc0-36fb-4c24-b1f7-a2531da941cb\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-491296-6b6d75c56c-2w7xh"
Sep 07 12:09:06.727051 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:09:06.727042    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9zj9h\" (UniqueName: \"kubernetes.io/projected/647bfbc0-36fb-4c24-b1f7-a2531da941cb-kube-api-access-9zj9h\") pod \"maas-api-e2e-shared-a-w3-491296-6b6d75c56c-2w7xh\" (UID: \"647bfbc0-36fb-4c24-b1f7-a2531da941cb\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-491296-6b6d75c56c-2w7xh"
Sep 07 12:09:06.727725 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:09:06.727697    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/647bfbc0-36fb-4c24-b1f7-a2531da941cb-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w3-491296-6b6d75c56c-2w7xh\" (UID: \"647bfbc0-36fb-4c24-b1f7-a2531da941cb\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-491296-6b6d75c56c-2w7xh"
Sep 07 12:09:06.729531 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:09:06.729512    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/647bfbc0-36fb-4c24-b1f7-a2531da941cb-maas-api-tls\") pod \"maas-api-e2e-shared-a-w3-491296-6b6d75c56c-2w7xh\" (UID: \"647bfbc0-36fb-4c24-b1f7-a2531da941cb\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-491296-6b6d75c56c-2w7xh"
Sep 07 12:09:06.729579 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:09:06.729519    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/647bfbc0-36fb-4c24-b1f7-a2531da941cb-metrics-certs\") pod \"maas-api-e2e-shared-a-w3-491296-6b6d75c56c-2w7xh\" (UID: \"647bfbc0-36fb-4c24-b1f7-a2531da941cb\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-491296-6b6d75c56c-2w7xh"
Sep 07 12:09:06.735467 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:09:06.735447    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9zj9h\" (UniqueName: \"kubernetes.io/projected/647bfbc0-36fb-4c24-b1f7-a2531da941cb-kube-api-access-9zj9h\") pod \"maas-api-e2e-shared-a-w3-491296-6b6d75c56c-2w7xh\" (UID: \"647bfbc0-36fb-4c24-b1f7-a2531da941cb\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-491296-6b6d75c56c-2w7xh"
Sep 07 12:09:06.790631 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:09:06.790599    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:09:06.801687 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:09:06.801664    2563 generic.go:356] "Generic (PLEG): container finished" podID="bc21582c-d7f9-48e4-9309-ffeefa3f7f03" containerID="9b88ef6e84375dd2043a9bb56c56e164029c3d12fdb2faa60380a2075a9bf783" exitCode=0
Sep 07 12:09:06.801796 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:09:06.801709    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-af50c7-openshift-default-67788bbd-v68lr"
Sep 07 12:09:06.801796 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:09:06.801713    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-af50c7-openshift-default-67788bbd-v68lr" event={"ID":"bc21582c-d7f9-48e4-9309-ffeefa3f7f03","Type":"ContainerDied","Data":"9b88ef6e84375dd2043a9bb56c56e164029c3d12fdb2faa60380a2075a9bf783"}
Sep 07 12:09:06.801796 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:09:06.801764    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-af50c7-openshift-default-67788bbd-v68lr" event={"ID":"bc21582c-d7f9-48e4-9309-ffeefa3f7f03","Type":"ContainerDied","Data":"c95e9f45f77fb1baf94b70a2d887d9065c91823db60e98d94d9a59cc726d620e"}
Sep 07 12:09:06.801796 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:09:06.801787    2563 scope.go:117] "RemoveContainer" containerID="9b88ef6e84375dd2043a9bb56c56e164029c3d12fdb2faa60380a2075a9bf783"
Sep 07 12:09:06.811024 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:09:06.811008    2563 scope.go:117] "RemoveContainer" containerID="0dd8e0198d29d62d2fe341baa5f3f2ab5a405b4c862095ecf99ff2f1791ba8c6"
Sep 07 12:09:06.818527 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:09:06.818511    2563 scope.go:117] "RemoveContainer" containerID="9b88ef6e84375dd2043a9bb56c56e164029c3d12fdb2faa60380a2075a9bf783"
Sep 07 12:09:06.818834 ip-10-0-135-173 kubenswrapper[2563]: E0907 12:09:06.818814    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9b88ef6e84375dd2043a9bb56c56e164029c3d12fdb2faa60380a2075a9bf783\": container with ID starting with 9b88ef6e84375dd2043a9bb56c56e164029c3d12fdb2faa60380a2075a9bf783 not found: ID does not exist" containerID="9b88ef6e84375dd2043a9bb56c56e164029c3d12fdb2faa60380a2075a9bf783"
Sep 07 12:09:06.818881 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:09:06.818846    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9b88ef6e84375dd2043a9bb56c56e164029c3d12fdb2faa60380a2075a9bf783"} err="failed to get container status \"9b88ef6e84375dd2043a9bb56c56e164029c3d12fdb2faa60380a2075a9bf783\": rpc error: code = NotFound desc = could not find container \"9b88ef6e84375dd2043a9bb56c56e164029c3d12fdb2faa60380a2075a9bf783\": container with ID starting with 9b88ef6e84375dd2043a9bb56c56e164029c3d12fdb2faa60380a2075a9bf783 not found: ID does not exist"
Sep 07 12:09:06.818881 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:09:06.818863    2563 scope.go:117] "RemoveContainer" containerID="0dd8e0198d29d62d2fe341baa5f3f2ab5a405b4c862095ecf99ff2f1791ba8c6"
Sep 07 12:09:06.819111 ip-10-0-135-173 kubenswrapper[2563]: E0907 12:09:06.819096    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0dd8e0198d29d62d2fe341baa5f3f2ab5a405b4c862095ecf99ff2f1791ba8c6\": container with ID starting with 0dd8e0198d29d62d2fe341baa5f3f2ab5a405b4c862095ecf99ff2f1791ba8c6 not found: ID does not exist" containerID="0dd8e0198d29d62d2fe341baa5f3f2ab5a405b4c862095ecf99ff2f1791ba8c6"
Sep 07 12:09:06.819170 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:09:06.819114    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0dd8e0198d29d62d2fe341baa5f3f2ab5a405b4c862095ecf99ff2f1791ba8c6"} err="failed to get container status \"0dd8e0198d29d62d2fe341baa5f3f2ab5a405b4c862095ecf99ff2f1791ba8c6\": rpc error: code = NotFound desc = could not find container \"0dd8e0198d29d62d2fe341baa5f3f2ab5a405b4c862095ecf99ff2f1791ba8c6\": container with ID starting with 0dd8e0198d29d62d2fe341baa5f3f2ab5a405b4c862095ecf99ff2f1791ba8c6 not found: ID does not exist"
Sep 07 12:09:06.834024 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:09:06.834003    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-af50c7-openshift-default-67788bbd-v68lr"]
Sep 07 12:09:06.843556 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:09:06.843537    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-af50c7-openshift-default-67788bbd-v68lr"]
Sep 07 12:09:06.873358 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:09:06.873341    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-491296-6b6d75c56c-2w7xh"
Sep 07 12:09:07.041276 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:09:07.041187    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bc21582c-d7f9-48e4-9309-ffeefa3f7f03" path="/var/lib/kubelet/pods/bc21582c-d7f9-48e4-9309-ffeefa3f7f03/volumes"
Sep 07 12:09:07.142656 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:09:07.142628    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-491296-6b6d75c56c-2w7xh"]
Sep 07 12:09:07.142872 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:09:07.142852    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:09:07.145266 ip-10-0-135-173 kubenswrapper[2563]: W0907 12:09:07.145232    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod647bfbc0_36fb_4c24_b1f7_a2531da941cb.slice/crio-cac8add44ab99862a440abad746affbd869bd6d2d6d8ff9b91c6ebd6998cf7e0 WatchSource:0}: Error finding container cac8add44ab99862a440abad746affbd869bd6d2d6d8ff9b91c6ebd6998cf7e0: Status 404 returned error can't find the container with id cac8add44ab99862a440abad746affbd869bd6d2d6d8ff9b91c6ebd6998cf7e0
Sep 07 12:09:07.146561 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:09:07.146543    2563 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 07 12:09:07.245777 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:09:07.245743    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:09:07.807487 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:09:07.807453    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-491296-6b6d75c56c-2w7xh" event={"ID":"647bfbc0-36fb-4c24-b1f7-a2531da941cb","Type":"ContainerStarted","Data":"cac8add44ab99862a440abad746affbd869bd6d2d6d8ff9b91c6ebd6998cf7e0"}
Sep 07 12:09:09.820360 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:09:09.820322    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-491296-6b6d75c56c-2w7xh" event={"ID":"647bfbc0-36fb-4c24-b1f7-a2531da941cb","Type":"ContainerStarted","Data":"67d38edf6325e1248b8f8c0a91849edb791fea1b217c5ccf92793b18538967f2"}
Sep 07 12:09:09.820740 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:09:09.820471    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-491296-6b6d75c56c-2w7xh"
Sep 07 12:09:09.836435 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:09:09.836374    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-491296-6b6d75c56c-2w7xh" podStartSLOduration=1.704048379 podStartE2EDuration="3.836359394s" totalImagesPullingTime="2.132311015s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 12:09:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 12:09:09.834532219 +0000 UTC m=+1321.353264032" watchObservedRunningTime="2026-09-07 12:09:09.836359394 +0000 UTC m=+1321.355091236"
Sep 07 12:09:11.037634 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:09:11.037601    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:09:11.108968 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:09:11.108932    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-p4l4w"]
Sep 07 12:09:11.111678 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:09:11.111651    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:09:11.901641 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:09:11.901604    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:09:12.214967 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:09:12.214926    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:09:12.390015 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:09:12.389980    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:09:15.489461 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:09:15.489408    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:09:15.589351 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:09:15.589315    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:09:15.830675 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:09:15.830591    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-491296-6b6d75c56c-2w7xh"
Sep 07 12:09:16.625691 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:09:16.625649    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-3a0bb2-767f497fdc-ck8sf"]
Sep 07 12:09:16.626399 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:09:16.626375    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="bc21582c-d7f9-48e4-9309-ffeefa3f7f03" containerName="istio-proxy"
Sep 07 12:09:16.630402 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:09:16.630380    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-3a0bb2-767f497fdc-ck8sf"
Sep 07 12:09:16.632067 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:09:16.632047    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-w2-3a0bb2-serving-cert\""
Sep 07 12:09:16.640752 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:09:16.640731    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-3a0bb2-767f497fdc-ck8sf"]
Sep 07 12:09:16.658529 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:09:16.658487    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:09:16.714480 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:09:16.714449    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9e938f27-d2a4-405f-ba3b-caa5f2b01d13-metrics-certs\") pod \"maas-api-e2e-shared-a-w2-3a0bb2-767f497fdc-ck8sf\" (UID: \"9e938f27-d2a4-405f-ba3b-caa5f2b01d13\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-3a0bb2-767f497fdc-ck8sf"
Sep 07 12:09:16.714647 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:09:16.714500    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9e938f27-d2a4-405f-ba3b-caa5f2b01d13-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w2-3a0bb2-767f497fdc-ck8sf\" (UID: \"9e938f27-d2a4-405f-ba3b-caa5f2b01d13\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-3a0bb2-767f497fdc-ck8sf"
Sep 07 12:09:16.714647 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:09:16.714564    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9e938f27-d2a4-405f-ba3b-caa5f2b01d13-maas-api-tls\") pod \"maas-api-e2e-shared-a-w2-3a0bb2-767f497fdc-ck8sf\" (UID: \"9e938f27-d2a4-405f-ba3b-caa5f2b01d13\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-3a0bb2-767f497fdc-ck8sf"
Sep 07 12:09:16.714647 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:09:16.714594    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jvx27\" (UniqueName: \"kubernetes.io/projected/9e938f27-d2a4-405f-ba3b-caa5f2b01d13-kube-api-access-jvx27\") pod \"maas-api-e2e-shared-a-w2-3a0bb2-767f497fdc-ck8sf\" (UID: \"9e938f27-d2a4-405f-ba3b-caa5f2b01d13\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-3a0bb2-767f497fdc-ck8sf"
Sep 07 12:09:16.753836 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:09:16.753804    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:09:16.815207 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:09:16.815170    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9e938f27-d2a4-405f-ba3b-caa5f2b01d13-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w2-3a0bb2-767f497fdc-ck8sf\" (UID: \"9e938f27-d2a4-405f-ba3b-caa5f2b01d13\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-3a0bb2-767f497fdc-ck8sf"
Sep 07 12:09:16.815382 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:09:16.815242    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9e938f27-d2a4-405f-ba3b-caa5f2b01d13-maas-api-tls\") pod \"maas-api-e2e-shared-a-w2-3a0bb2-767f497fdc-ck8sf\" (UID: \"9e938f27-d2a4-405f-ba3b-caa5f2b01d13\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-3a0bb2-767f497fdc-ck8sf"
Sep 07 12:09:16.815382 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:09:16.815282    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jvx27\" (UniqueName: \"kubernetes.io/projected/9e938f27-d2a4-405f-ba3b-caa5f2b01d13-kube-api-access-jvx27\") pod \"maas-api-e2e-shared-a-w2-3a0bb2-767f497fdc-ck8sf\" (UID: \"9e938f27-d2a4-405f-ba3b-caa5f2b01d13\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-3a0bb2-767f497fdc-ck8sf"
Sep 07 12:09:16.815382 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:09:16.815352    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9e938f27-d2a4-405f-ba3b-caa5f2b01d13-metrics-certs\") pod \"maas-api-e2e-shared-a-w2-3a0bb2-767f497fdc-ck8sf\" (UID: \"9e938f27-d2a4-405f-ba3b-caa5f2b01d13\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-3a0bb2-767f497fdc-ck8sf"
Sep 07 12:09:16.815592 ip-10-0-135-173 kubenswrapper[2563]: E0907 12:09:16.815407    2563 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-3a0bb2-serving-cert: secret "maas-api-e2e-shared-a-w2-3a0bb2-serving-cert" not found
Sep 07 12:09:16.815592 ip-10-0-135-173 kubenswrapper[2563]: E0907 12:09:16.815509    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9e938f27-d2a4-405f-ba3b-caa5f2b01d13-maas-api-tls podName:9e938f27-d2a4-405f-ba3b-caa5f2b01d13 nodeName:}" failed. No retries permitted until 2026-09-07 12:09:17.315486977 +0000 UTC m=+1328.834218775 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/9e938f27-d2a4-405f-ba3b-caa5f2b01d13-maas-api-tls") pod "maas-api-e2e-shared-a-w2-3a0bb2-767f497fdc-ck8sf" (UID: "9e938f27-d2a4-405f-ba3b-caa5f2b01d13") : secret "maas-api-e2e-shared-a-w2-3a0bb2-serving-cert" not found
Sep 07 12:09:16.815965 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:09:16.815946    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9e938f27-d2a4-405f-ba3b-caa5f2b01d13-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w2-3a0bb2-767f497fdc-ck8sf\" (UID: \"9e938f27-d2a4-405f-ba3b-caa5f2b01d13\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-3a0bb2-767f497fdc-ck8sf"
Sep 07 12:09:16.817866 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:09:16.817844    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9e938f27-d2a4-405f-ba3b-caa5f2b01d13-metrics-certs\") pod \"maas-api-e2e-shared-a-w2-3a0bb2-767f497fdc-ck8sf\" (UID: \"9e938f27-d2a4-405f-ba3b-caa5f2b01d13\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-3a0bb2-767f497fdc-ck8sf"
Sep 07 12:09:16.822630 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:09:16.822605    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jvx27\" (UniqueName: \"kubernetes.io/projected/9e938f27-d2a4-405f-ba3b-caa5f2b01d13-kube-api-access-jvx27\") pod \"maas-api-e2e-shared-a-w2-3a0bb2-767f497fdc-ck8sf\" (UID: \"9e938f27-d2a4-405f-ba3b-caa5f2b01d13\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-3a0bb2-767f497fdc-ck8sf"
Sep 07 12:09:17.146522 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:09:17.146471    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:09:17.244373 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:09:17.244338    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:09:17.320018 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:09:17.319983    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9e938f27-d2a4-405f-ba3b-caa5f2b01d13-maas-api-tls\") pod \"maas-api-e2e-shared-a-w2-3a0bb2-767f497fdc-ck8sf\" (UID: \"9e938f27-d2a4-405f-ba3b-caa5f2b01d13\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-3a0bb2-767f497fdc-ck8sf"
Sep 07 12:09:17.322342 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:09:17.322314    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9e938f27-d2a4-405f-ba3b-caa5f2b01d13-maas-api-tls\") pod \"maas-api-e2e-shared-a-w2-3a0bb2-767f497fdc-ck8sf\" (UID: \"9e938f27-d2a4-405f-ba3b-caa5f2b01d13\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-3a0bb2-767f497fdc-ck8sf"
Sep 07 12:09:17.544583 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:09:17.544554    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-3a0bb2-767f497fdc-ck8sf"
Sep 07 12:09:17.813616 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:09:17.813592    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-3a0bb2-767f497fdc-ck8sf"]
Sep 07 12:09:17.815950 ip-10-0-135-173 kubenswrapper[2563]: W0907 12:09:17.815926    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9e938f27_d2a4_405f_ba3b_caa5f2b01d13.slice/crio-feb53f1608d398841e0bfc3519669df88728988a10101f63a7bd2fd0ebfa42a5 WatchSource:0}: Error finding container feb53f1608d398841e0bfc3519669df88728988a10101f63a7bd2fd0ebfa42a5: Status 404 returned error can't find the container with id feb53f1608d398841e0bfc3519669df88728988a10101f63a7bd2fd0ebfa42a5
Sep 07 12:09:17.860067 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:09:17.860041    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-3a0bb2-767f497fdc-ck8sf" event={"ID":"9e938f27-d2a4-405f-ba3b-caa5f2b01d13","Type":"ContainerStarted","Data":"feb53f1608d398841e0bfc3519669df88728988a10101f63a7bd2fd0ebfa42a5"}
Sep 07 12:09:18.866408 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:09:18.866370    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-3a0bb2-767f497fdc-ck8sf" event={"ID":"9e938f27-d2a4-405f-ba3b-caa5f2b01d13","Type":"ContainerStarted","Data":"9e13e152be884f80b5576478b26632622c4f160532d6a69962496cdd5102975f"}
Sep 07 12:09:18.866824 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:09:18.866463    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-3a0bb2-767f497fdc-ck8sf"
Sep 07 12:09:18.882017 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:09:18.881964    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-3a0bb2-767f497fdc-ck8sf" podStartSLOduration=2.363942358 podStartE2EDuration="2.881946932s" totalImagesPullingTime="518.004574ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 12:09:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 12:09:18.881594358 +0000 UTC m=+1330.400326172" watchObservedRunningTime="2026-09-07 12:09:18.881946932 +0000 UTC m=+1330.400678745"
Sep 07 12:09:21.611664 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:09:21.611628    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-p4l4w"]
Sep 07 12:09:24.876271 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:09:24.876236    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-3a0bb2-767f497fdc-ck8sf"
Sep 07 12:09:27.539886 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:09:27.539848    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:09:30.330949 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:09:30.330911    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:09:30.697517 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:09:30.697479    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:09:30.972708 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:09:30.972623    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:09:31.377005 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:09:31.376891    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:09:33.320470 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:09:33.320438    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:09:33.625059 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:09:33.624976    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:09:33.716178 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:09:33.716139    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:09:34.947198 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:09:34.947162    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:09:35.007216 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:09:35.007175    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:09:35.108393 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:09:35.108356    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:09:35.439016 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:09:35.438956    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:09:35.516381 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:09:35.516271    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:09:37.524684 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:09:37.524634    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:09:39.806534 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:09:39.806504    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-p4l4w"]
Sep 07 12:09:40.428713 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:09:40.428662    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:09:40.728849 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:09:40.728800    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:09:40.990296 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:09:40.990205    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-w2-bff0c9-openshift-default-5ccb878dbd-vv8g2"]
Sep 07 12:09:40.994511 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:09:40.994486    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w2-bff0c9-openshift-default-5ccb878dbd-vv8g2"
Sep 07 12:09:40.996303 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:09:40.996279    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Sep 07 12:09:40.996441 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:09:40.996399    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-w2-bff0c9-openshift-default-dockercfg-gnlrw\""
Sep 07 12:09:41.008035 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:09:41.008008    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-bff0c9-openshift-default-5ccb878dbd-vv8g2"]
Sep 07 12:09:41.019672 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:09:41.019642    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:09:41.043867 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:09:41.043837    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/42e3b8e0-8d8e-4d30-b7b0-ce97fd72afed-istio-token\") pod \"e2e-shared-b-w2-bff0c9-openshift-default-5ccb878dbd-vv8g2\" (UID: \"42e3b8e0-8d8e-4d30-b7b0-ce97fd72afed\") " pod="openshift-ingress/e2e-shared-b-w2-bff0c9-openshift-default-5ccb878dbd-vv8g2"
Sep 07 12:09:41.043989 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:09:41.043879    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/42e3b8e0-8d8e-4d30-b7b0-ce97fd72afed-istio-data\") pod \"e2e-shared-b-w2-bff0c9-openshift-default-5ccb878dbd-vv8g2\" (UID: \"42e3b8e0-8d8e-4d30-b7b0-ce97fd72afed\") " pod="openshift-ingress/e2e-shared-b-w2-bff0c9-openshift-default-5ccb878dbd-vv8g2"
Sep 07 12:09:41.044057 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:09:41.043993    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nq7bm\" (UniqueName: \"kubernetes.io/projected/42e3b8e0-8d8e-4d30-b7b0-ce97fd72afed-kube-api-access-nq7bm\") pod \"e2e-shared-b-w2-bff0c9-openshift-default-5ccb878dbd-vv8g2\" (UID: \"42e3b8e0-8d8e-4d30-b7b0-ce97fd72afed\") " pod="openshift-ingress/e2e-shared-b-w2-bff0c9-openshift-default-5ccb878dbd-vv8g2"
Sep 07 12:09:41.044167 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:09:41.044146    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/42e3b8e0-8d8e-4d30-b7b0-ce97fd72afed-istio-podinfo\") pod \"e2e-shared-b-w2-bff0c9-openshift-default-5ccb878dbd-vv8g2\" (UID: \"42e3b8e0-8d8e-4d30-b7b0-ce97fd72afed\") " pod="openshift-ingress/e2e-shared-b-w2-bff0c9-openshift-default-5ccb878dbd-vv8g2"
Sep 07 12:09:41.044245 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:09:41.044188    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/42e3b8e0-8d8e-4d30-b7b0-ce97fd72afed-istio-envoy\") pod \"e2e-shared-b-w2-bff0c9-openshift-default-5ccb878dbd-vv8g2\" (UID: \"42e3b8e0-8d8e-4d30-b7b0-ce97fd72afed\") " pod="openshift-ingress/e2e-shared-b-w2-bff0c9-openshift-default-5ccb878dbd-vv8g2"
Sep 07 12:09:41.044245 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:09:41.044214    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/42e3b8e0-8d8e-4d30-b7b0-ce97fd72afed-credential-socket\") pod \"e2e-shared-b-w2-bff0c9-openshift-default-5ccb878dbd-vv8g2\" (UID: \"42e3b8e0-8d8e-4d30-b7b0-ce97fd72afed\") " pod="openshift-ingress/e2e-shared-b-w2-bff0c9-openshift-default-5ccb878dbd-vv8g2"
Sep 07 12:09:41.044245 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:09:41.044230    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/42e3b8e0-8d8e-4d30-b7b0-ce97fd72afed-workload-socket\") pod \"e2e-shared-b-w2-bff0c9-openshift-default-5ccb878dbd-vv8g2\" (UID: \"42e3b8e0-8d8e-4d30-b7b0-ce97fd72afed\") " pod="openshift-ingress/e2e-shared-b-w2-bff0c9-openshift-default-5ccb878dbd-vv8g2"
Sep 07 12:09:41.044447 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:09:41.044409    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/42e3b8e0-8d8e-4d30-b7b0-ce97fd72afed-istiod-ca-cert\") pod \"e2e-shared-b-w2-bff0c9-openshift-default-5ccb878dbd-vv8g2\" (UID: \"42e3b8e0-8d8e-4d30-b7b0-ce97fd72afed\") " pod="openshift-ingress/e2e-shared-b-w2-bff0c9-openshift-default-5ccb878dbd-vv8g2"
Sep 07 12:09:41.044514 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:09:41.044490    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/42e3b8e0-8d8e-4d30-b7b0-ce97fd72afed-workload-certs\") pod \"e2e-shared-b-w2-bff0c9-openshift-default-5ccb878dbd-vv8g2\" (UID: \"42e3b8e0-8d8e-4d30-b7b0-ce97fd72afed\") " pod="openshift-ingress/e2e-shared-b-w2-bff0c9-openshift-default-5ccb878dbd-vv8g2"
Sep 07 12:09:41.145640 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:09:41.145615    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/42e3b8e0-8d8e-4d30-b7b0-ce97fd72afed-istiod-ca-cert\") pod \"e2e-shared-b-w2-bff0c9-openshift-default-5ccb878dbd-vv8g2\" (UID: \"42e3b8e0-8d8e-4d30-b7b0-ce97fd72afed\") " pod="openshift-ingress/e2e-shared-b-w2-bff0c9-openshift-default-5ccb878dbd-vv8g2"
Sep 07 12:09:41.145799 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:09:41.145654    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/42e3b8e0-8d8e-4d30-b7b0-ce97fd72afed-workload-certs\") pod \"e2e-shared-b-w2-bff0c9-openshift-default-5ccb878dbd-vv8g2\" (UID: \"42e3b8e0-8d8e-4d30-b7b0-ce97fd72afed\") " pod="openshift-ingress/e2e-shared-b-w2-bff0c9-openshift-default-5ccb878dbd-vv8g2"
Sep 07 12:09:41.145799 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:09:41.145677    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/42e3b8e0-8d8e-4d30-b7b0-ce97fd72afed-istio-token\") pod \"e2e-shared-b-w2-bff0c9-openshift-default-5ccb878dbd-vv8g2\" (UID: \"42e3b8e0-8d8e-4d30-b7b0-ce97fd72afed\") " pod="openshift-ingress/e2e-shared-b-w2-bff0c9-openshift-default-5ccb878dbd-vv8g2"
Sep 07 12:09:41.145799 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:09:41.145698    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/42e3b8e0-8d8e-4d30-b7b0-ce97fd72afed-istio-data\") pod \"e2e-shared-b-w2-bff0c9-openshift-default-5ccb878dbd-vv8g2\" (UID: \"42e3b8e0-8d8e-4d30-b7b0-ce97fd72afed\") " pod="openshift-ingress/e2e-shared-b-w2-bff0c9-openshift-default-5ccb878dbd-vv8g2"
Sep 07 12:09:41.145799 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:09:41.145730    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nq7bm\" (UniqueName: \"kubernetes.io/projected/42e3b8e0-8d8e-4d30-b7b0-ce97fd72afed-kube-api-access-nq7bm\") pod \"e2e-shared-b-w2-bff0c9-openshift-default-5ccb878dbd-vv8g2\" (UID: \"42e3b8e0-8d8e-4d30-b7b0-ce97fd72afed\") " pod="openshift-ingress/e2e-shared-b-w2-bff0c9-openshift-default-5ccb878dbd-vv8g2"
Sep 07 12:09:41.145799 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:09:41.145754    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/42e3b8e0-8d8e-4d30-b7b0-ce97fd72afed-istio-podinfo\") pod \"e2e-shared-b-w2-bff0c9-openshift-default-5ccb878dbd-vv8g2\" (UID: \"42e3b8e0-8d8e-4d30-b7b0-ce97fd72afed\") " pod="openshift-ingress/e2e-shared-b-w2-bff0c9-openshift-default-5ccb878dbd-vv8g2"
Sep 07 12:09:41.145799 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:09:41.145773    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/42e3b8e0-8d8e-4d30-b7b0-ce97fd72afed-istio-envoy\") pod \"e2e-shared-b-w2-bff0c9-openshift-default-5ccb878dbd-vv8g2\" (UID: \"42e3b8e0-8d8e-4d30-b7b0-ce97fd72afed\") " pod="openshift-ingress/e2e-shared-b-w2-bff0c9-openshift-default-5ccb878dbd-vv8g2"
Sep 07 12:09:41.145799 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:09:41.145793    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/42e3b8e0-8d8e-4d30-b7b0-ce97fd72afed-credential-socket\") pod \"e2e-shared-b-w2-bff0c9-openshift-default-5ccb878dbd-vv8g2\" (UID: \"42e3b8e0-8d8e-4d30-b7b0-ce97fd72afed\") " pod="openshift-ingress/e2e-shared-b-w2-bff0c9-openshift-default-5ccb878dbd-vv8g2"
Sep 07 12:09:41.146161 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:09:41.145819    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/42e3b8e0-8d8e-4d30-b7b0-ce97fd72afed-workload-socket\") pod \"e2e-shared-b-w2-bff0c9-openshift-default-5ccb878dbd-vv8g2\" (UID: \"42e3b8e0-8d8e-4d30-b7b0-ce97fd72afed\") " pod="openshift-ingress/e2e-shared-b-w2-bff0c9-openshift-default-5ccb878dbd-vv8g2"
Sep 07 12:09:41.146161 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:09:41.146132    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/42e3b8e0-8d8e-4d30-b7b0-ce97fd72afed-istio-data\") pod \"e2e-shared-b-w2-bff0c9-openshift-default-5ccb878dbd-vv8g2\" (UID: \"42e3b8e0-8d8e-4d30-b7b0-ce97fd72afed\") " pod="openshift-ingress/e2e-shared-b-w2-bff0c9-openshift-default-5ccb878dbd-vv8g2"
Sep 07 12:09:41.146264 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:09:41.146206    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/42e3b8e0-8d8e-4d30-b7b0-ce97fd72afed-workload-certs\") pod \"e2e-shared-b-w2-bff0c9-openshift-default-5ccb878dbd-vv8g2\" (UID: \"42e3b8e0-8d8e-4d30-b7b0-ce97fd72afed\") " pod="openshift-ingress/e2e-shared-b-w2-bff0c9-openshift-default-5ccb878dbd-vv8g2"
Sep 07 12:09:41.146350 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:09:41.146329    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/42e3b8e0-8d8e-4d30-b7b0-ce97fd72afed-workload-socket\") pod \"e2e-shared-b-w2-bff0c9-openshift-default-5ccb878dbd-vv8g2\" (UID: \"42e3b8e0-8d8e-4d30-b7b0-ce97fd72afed\") " pod="openshift-ingress/e2e-shared-b-w2-bff0c9-openshift-default-5ccb878dbd-vv8g2"
Sep 07 12:09:41.146350 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:09:41.146342    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/42e3b8e0-8d8e-4d30-b7b0-ce97fd72afed-credential-socket\") pod \"e2e-shared-b-w2-bff0c9-openshift-default-5ccb878dbd-vv8g2\" (UID: \"42e3b8e0-8d8e-4d30-b7b0-ce97fd72afed\") " pod="openshift-ingress/e2e-shared-b-w2-bff0c9-openshift-default-5ccb878dbd-vv8g2"
Sep 07 12:09:41.146496 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:09:41.146438    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/42e3b8e0-8d8e-4d30-b7b0-ce97fd72afed-istiod-ca-cert\") pod \"e2e-shared-b-w2-bff0c9-openshift-default-5ccb878dbd-vv8g2\" (UID: \"42e3b8e0-8d8e-4d30-b7b0-ce97fd72afed\") " pod="openshift-ingress/e2e-shared-b-w2-bff0c9-openshift-default-5ccb878dbd-vv8g2"
Sep 07 12:09:41.148430 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:09:41.148372    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/42e3b8e0-8d8e-4d30-b7b0-ce97fd72afed-istio-envoy\") pod \"e2e-shared-b-w2-bff0c9-openshift-default-5ccb878dbd-vv8g2\" (UID: \"42e3b8e0-8d8e-4d30-b7b0-ce97fd72afed\") " pod="openshift-ingress/e2e-shared-b-w2-bff0c9-openshift-default-5ccb878dbd-vv8g2"
Sep 07 12:09:41.148722 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:09:41.148700    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/42e3b8e0-8d8e-4d30-b7b0-ce97fd72afed-istio-podinfo\") pod \"e2e-shared-b-w2-bff0c9-openshift-default-5ccb878dbd-vv8g2\" (UID: \"42e3b8e0-8d8e-4d30-b7b0-ce97fd72afed\") " pod="openshift-ingress/e2e-shared-b-w2-bff0c9-openshift-default-5ccb878dbd-vv8g2"
Sep 07 12:09:41.153511 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:09:41.153490    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nq7bm\" (UniqueName: \"kubernetes.io/projected/42e3b8e0-8d8e-4d30-b7b0-ce97fd72afed-kube-api-access-nq7bm\") pod \"e2e-shared-b-w2-bff0c9-openshift-default-5ccb878dbd-vv8g2\" (UID: \"42e3b8e0-8d8e-4d30-b7b0-ce97fd72afed\") " pod="openshift-ingress/e2e-shared-b-w2-bff0c9-openshift-default-5ccb878dbd-vv8g2"
Sep 07 12:09:41.153597 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:09:41.153557    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/42e3b8e0-8d8e-4d30-b7b0-ce97fd72afed-istio-token\") pod \"e2e-shared-b-w2-bff0c9-openshift-default-5ccb878dbd-vv8g2\" (UID: \"42e3b8e0-8d8e-4d30-b7b0-ce97fd72afed\") " pod="openshift-ingress/e2e-shared-b-w2-bff0c9-openshift-default-5ccb878dbd-vv8g2"
Sep 07 12:09:41.168195 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:09:41.168168    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:09:41.314090 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:09:41.314013    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w2-bff0c9-openshift-default-5ccb878dbd-vv8g2"
Sep 07 12:09:41.587142 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:09:41.587085    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-bff0c9-openshift-default-5ccb878dbd-vv8g2"]
Sep 07 12:09:41.589397 ip-10-0-135-173 kubenswrapper[2563]: W0907 12:09:41.589365    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod42e3b8e0_8d8e_4d30_b7b0_ce97fd72afed.slice/crio-d76a94836595e035b6381c82999334574e08c9eae4189ad0b3fd2bbb56490db0 WatchSource:0}: Error finding container d76a94836595e035b6381c82999334574e08c9eae4189ad0b3fd2bbb56490db0: Status 404 returned error can't find the container with id d76a94836595e035b6381c82999334574e08c9eae4189ad0b3fd2bbb56490db0
Sep 07 12:09:41.591730 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:09:41.591698    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 07 12:09:41.591850 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:09:41.591780    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 07 12:09:41.591915 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:09:41.591846    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 07 12:09:41.983723 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:09:41.983683    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-bff0c9-openshift-default-5ccb878dbd-vv8g2" event={"ID":"42e3b8e0-8d8e-4d30-b7b0-ce97fd72afed","Type":"ContainerStarted","Data":"f11d690d4306100794e2365a5a0237e13f4b54f60cc44b49ac1b5152e78d180a"}
Sep 07 12:09:41.983895 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:09:41.983746    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-bff0c9-openshift-default-5ccb878dbd-vv8g2" event={"ID":"42e3b8e0-8d8e-4d30-b7b0-ce97fd72afed","Type":"ContainerStarted","Data":"d76a94836595e035b6381c82999334574e08c9eae4189ad0b3fd2bbb56490db0"}
Sep 07 12:09:42.005264 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:09:42.005201    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-w2-bff0c9-openshift-default-5ccb878dbd-vv8g2" podStartSLOduration=2.005180222 podStartE2EDuration="2.005180222s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 12:09:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 12:09:42.004179341 +0000 UTC m=+1353.522911157" watchObservedRunningTime="2026-09-07 12:09:42.005180222 +0000 UTC m=+1353.523912036"
Sep 07 12:09:42.314781 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:09:42.314693    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w2-bff0c9-openshift-default-5ccb878dbd-vv8g2"
Sep 07 12:09:42.319537 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:09:42.319512    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w2-bff0c9-openshift-default-5ccb878dbd-vv8g2"
Sep 07 12:09:42.988366 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:09:42.988337    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w2-bff0c9-openshift-default-5ccb878dbd-vv8g2"
Sep 07 12:09:42.989119 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:09:42.989094    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w2-bff0c9-openshift-default-5ccb878dbd-vv8g2"
Sep 07 12:09:43.855528 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:09:43.855489    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:09:43.948109 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:09:43.948073    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:09:45.134284 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:09:45.134251    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:09:45.197505 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:09:45.197474    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:09:45.293134 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:09:45.293103    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:09:45.627284 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:09:45.627251    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:09:45.722935 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:09:45.722895    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:09:51.796961 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:09:51.796928    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-p4l4w"]
Sep 07 12:09:55.829919 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:09:55.829883    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:09:58.584644 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:09:58.584599    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:10:01.000308 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:10:01.000265    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-w3-491296/auth-test-model-491296-kserve-775ddd7b8c-cklcl"]
Sep 07 12:10:01.004057 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:10:01.004035    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-491296/auth-test-model-491296-kserve-775ddd7b8c-cklcl"
Sep 07 12:10:01.006261 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:10:01.006153    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-491296\"/\"default-dockercfg-ztltr\""
Sep 07 12:10:01.006261 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:10:01.006186    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w3-491296\"/\"kube-root-ca.crt\""
Sep 07 12:10:01.006261 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:10:01.006191    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w3-491296\"/\"openshift-service-ca.crt\""
Sep 07 12:10:01.006261 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:10:01.006226    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-491296\"/\"auth-test-model-491296-kserve-self-signed-certs\""
Sep 07 12:10:01.010308 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:10:01.010283    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-491296/auth-test-model-491296-kserve-775ddd7b8c-cklcl"]
Sep 07 12:10:01.124405 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:10:01.124365    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8430a642-c45a-4fa2-80d7-765b61ddcd70-tmp-dir\") pod \"auth-test-model-491296-kserve-775ddd7b8c-cklcl\" (UID: \"8430a642-c45a-4fa2-80d7-765b61ddcd70\") " pod="ai-tenant-e2e-shared-a-w3-491296/auth-test-model-491296-kserve-775ddd7b8c-cklcl"
Sep 07 12:10:01.124581 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:10:01.124517    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8430a642-c45a-4fa2-80d7-765b61ddcd70-dshm\") pod \"auth-test-model-491296-kserve-775ddd7b8c-cklcl\" (UID: \"8430a642-c45a-4fa2-80d7-765b61ddcd70\") " pod="ai-tenant-e2e-shared-a-w3-491296/auth-test-model-491296-kserve-775ddd7b8c-cklcl"
Sep 07 12:10:01.124645 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:10:01.124593    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8430a642-c45a-4fa2-80d7-765b61ddcd70-model-cache\") pod \"auth-test-model-491296-kserve-775ddd7b8c-cklcl\" (UID: \"8430a642-c45a-4fa2-80d7-765b61ddcd70\") " pod="ai-tenant-e2e-shared-a-w3-491296/auth-test-model-491296-kserve-775ddd7b8c-cklcl"
Sep 07 12:10:01.124701 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:10:01.124645    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8430a642-c45a-4fa2-80d7-765b61ddcd70-tls-certs\") pod \"auth-test-model-491296-kserve-775ddd7b8c-cklcl\" (UID: \"8430a642-c45a-4fa2-80d7-765b61ddcd70\") " pod="ai-tenant-e2e-shared-a-w3-491296/auth-test-model-491296-kserve-775ddd7b8c-cklcl"
Sep 07 12:10:01.124766 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:10:01.124695    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4mkh8\" (UniqueName: \"kubernetes.io/projected/8430a642-c45a-4fa2-80d7-765b61ddcd70-kube-api-access-4mkh8\") pod \"auth-test-model-491296-kserve-775ddd7b8c-cklcl\" (UID: \"8430a642-c45a-4fa2-80d7-765b61ddcd70\") " pod="ai-tenant-e2e-shared-a-w3-491296/auth-test-model-491296-kserve-775ddd7b8c-cklcl"
Sep 07 12:10:01.124823 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:10:01.124809    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8430a642-c45a-4fa2-80d7-765b61ddcd70-home\") pod \"auth-test-model-491296-kserve-775ddd7b8c-cklcl\" (UID: \"8430a642-c45a-4fa2-80d7-765b61ddcd70\") " pod="ai-tenant-e2e-shared-a-w3-491296/auth-test-model-491296-kserve-775ddd7b8c-cklcl"
Sep 07 12:10:01.174583 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:10:01.174537    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:10:01.225844 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:10:01.225806    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4mkh8\" (UniqueName: \"kubernetes.io/projected/8430a642-c45a-4fa2-80d7-765b61ddcd70-kube-api-access-4mkh8\") pod \"auth-test-model-491296-kserve-775ddd7b8c-cklcl\" (UID: \"8430a642-c45a-4fa2-80d7-765b61ddcd70\") " pod="ai-tenant-e2e-shared-a-w3-491296/auth-test-model-491296-kserve-775ddd7b8c-cklcl"
Sep 07 12:10:01.226043 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:10:01.225865    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8430a642-c45a-4fa2-80d7-765b61ddcd70-home\") pod \"auth-test-model-491296-kserve-775ddd7b8c-cklcl\" (UID: \"8430a642-c45a-4fa2-80d7-765b61ddcd70\") " pod="ai-tenant-e2e-shared-a-w3-491296/auth-test-model-491296-kserve-775ddd7b8c-cklcl"
Sep 07 12:10:01.226043 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:10:01.225891    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8430a642-c45a-4fa2-80d7-765b61ddcd70-tmp-dir\") pod \"auth-test-model-491296-kserve-775ddd7b8c-cklcl\" (UID: \"8430a642-c45a-4fa2-80d7-765b61ddcd70\") " pod="ai-tenant-e2e-shared-a-w3-491296/auth-test-model-491296-kserve-775ddd7b8c-cklcl"
Sep 07 12:10:01.226043 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:10:01.225950    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8430a642-c45a-4fa2-80d7-765b61ddcd70-dshm\") pod \"auth-test-model-491296-kserve-775ddd7b8c-cklcl\" (UID: \"8430a642-c45a-4fa2-80d7-765b61ddcd70\") " pod="ai-tenant-e2e-shared-a-w3-491296/auth-test-model-491296-kserve-775ddd7b8c-cklcl"
Sep 07 12:10:01.226043 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:10:01.225973    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8430a642-c45a-4fa2-80d7-765b61ddcd70-model-cache\") pod \"auth-test-model-491296-kserve-775ddd7b8c-cklcl\" (UID: \"8430a642-c45a-4fa2-80d7-765b61ddcd70\") " pod="ai-tenant-e2e-shared-a-w3-491296/auth-test-model-491296-kserve-775ddd7b8c-cklcl"
Sep 07 12:10:01.226043 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:10:01.226008    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8430a642-c45a-4fa2-80d7-765b61ddcd70-tls-certs\") pod \"auth-test-model-491296-kserve-775ddd7b8c-cklcl\" (UID: \"8430a642-c45a-4fa2-80d7-765b61ddcd70\") " pod="ai-tenant-e2e-shared-a-w3-491296/auth-test-model-491296-kserve-775ddd7b8c-cklcl"
Sep 07 12:10:01.226385 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:10:01.226329    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8430a642-c45a-4fa2-80d7-765b61ddcd70-tmp-dir\") pod \"auth-test-model-491296-kserve-775ddd7b8c-cklcl\" (UID: \"8430a642-c45a-4fa2-80d7-765b61ddcd70\") " pod="ai-tenant-e2e-shared-a-w3-491296/auth-test-model-491296-kserve-775ddd7b8c-cklcl"
Sep 07 12:10:01.226385 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:10:01.226341    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8430a642-c45a-4fa2-80d7-765b61ddcd70-home\") pod \"auth-test-model-491296-kserve-775ddd7b8c-cklcl\" (UID: \"8430a642-c45a-4fa2-80d7-765b61ddcd70\") " pod="ai-tenant-e2e-shared-a-w3-491296/auth-test-model-491296-kserve-775ddd7b8c-cklcl"
Sep 07 12:10:01.226524 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:10:01.226461    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8430a642-c45a-4fa2-80d7-765b61ddcd70-model-cache\") pod \"auth-test-model-491296-kserve-775ddd7b8c-cklcl\" (UID: \"8430a642-c45a-4fa2-80d7-765b61ddcd70\") " pod="ai-tenant-e2e-shared-a-w3-491296/auth-test-model-491296-kserve-775ddd7b8c-cklcl"
Sep 07 12:10:01.228231 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:10:01.228206    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8430a642-c45a-4fa2-80d7-765b61ddcd70-dshm\") pod \"auth-test-model-491296-kserve-775ddd7b8c-cklcl\" (UID: \"8430a642-c45a-4fa2-80d7-765b61ddcd70\") " pod="ai-tenant-e2e-shared-a-w3-491296/auth-test-model-491296-kserve-775ddd7b8c-cklcl"
Sep 07 12:10:01.228585 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:10:01.228566    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8430a642-c45a-4fa2-80d7-765b61ddcd70-tls-certs\") pod \"auth-test-model-491296-kserve-775ddd7b8c-cklcl\" (UID: \"8430a642-c45a-4fa2-80d7-765b61ddcd70\") " pod="ai-tenant-e2e-shared-a-w3-491296/auth-test-model-491296-kserve-775ddd7b8c-cklcl"
Sep 07 12:10:01.232494 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:10:01.232471    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4mkh8\" (UniqueName: \"kubernetes.io/projected/8430a642-c45a-4fa2-80d7-765b61ddcd70-kube-api-access-4mkh8\") pod \"auth-test-model-491296-kserve-775ddd7b8c-cklcl\" (UID: \"8430a642-c45a-4fa2-80d7-765b61ddcd70\") " pod="ai-tenant-e2e-shared-a-w3-491296/auth-test-model-491296-kserve-775ddd7b8c-cklcl"
Sep 07 12:10:01.314773 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:10:01.314669    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-491296/auth-test-model-491296-kserve-775ddd7b8c-cklcl"
Sep 07 12:10:01.586651 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:10:01.586615    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-491296/auth-test-model-491296-kserve-775ddd7b8c-cklcl"]
Sep 07 12:10:01.589721 ip-10-0-135-173 kubenswrapper[2563]: W0907 12:10:01.589696    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8430a642_c45a_4fa2_80d7_765b61ddcd70.slice/crio-9e4b20cf8e5d9271dec81e5c6641482d6e2480fcbf89200fb707b17bc8fb45cc WatchSource:0}: Error finding container 9e4b20cf8e5d9271dec81e5c6641482d6e2480fcbf89200fb707b17bc8fb45cc: Status 404 returned error can't find the container with id 9e4b20cf8e5d9271dec81e5c6641482d6e2480fcbf89200fb707b17bc8fb45cc
Sep 07 12:10:02.103804 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:10:02.103773    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-491296/auth-test-model-491296-kserve-775ddd7b8c-cklcl" event={"ID":"8430a642-c45a-4fa2-80d7-765b61ddcd70","Type":"ContainerStarted","Data":"d8fce54fdc0e63733f8fc969f5e0f8b24602c4218de69ffd4ec30e6f15e157e8"}
Sep 07 12:10:02.104243 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:10:02.103812    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-491296/auth-test-model-491296-kserve-775ddd7b8c-cklcl" event={"ID":"8430a642-c45a-4fa2-80d7-765b61ddcd70","Type":"ContainerStarted","Data":"9e4b20cf8e5d9271dec81e5c6641482d6e2480fcbf89200fb707b17bc8fb45cc"}
Sep 07 12:10:02.119333 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:10:02.119275    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-w3-491296/auth-test-model-491296-kserve-775ddd7b8c-cklcl" podStartSLOduration=2.119258668 podStartE2EDuration="2.119258668s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 12:10:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 12:10:02.117597665 +0000 UTC m=+1373.636329490" watchObservedRunningTime="2026-09-07 12:10:02.119258668 +0000 UTC m=+1373.637990483"
Sep 07 12:10:03.109870 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:10:03.109840    2563 generic.go:356] "Generic (PLEG): container finished" podID="42e3b8e0-8d8e-4d30-b7b0-ce97fd72afed" containerID="f11d690d4306100794e2365a5a0237e13f4b54f60cc44b49ac1b5152e78d180a" exitCode=0
Sep 07 12:10:03.110309 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:10:03.109918    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-bff0c9-openshift-default-5ccb878dbd-vv8g2" event={"ID":"42e3b8e0-8d8e-4d30-b7b0-ce97fd72afed","Type":"ContainerDied","Data":"f11d690d4306100794e2365a5a0237e13f4b54f60cc44b49ac1b5152e78d180a"}
Sep 07 12:10:03.110309 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:10:03.110286    2563 scope.go:117] "RemoveContainer" containerID="f11d690d4306100794e2365a5a0237e13f4b54f60cc44b49ac1b5152e78d180a"
Sep 07 12:10:03.111269 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:10:03.111237    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 07 12:10:03.111355 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:10:03.111306    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 07 12:10:03.111355 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:10:03.111339    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 07 12:10:04.119105 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:10:04.119073    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-bff0c9-openshift-default-5ccb878dbd-vv8g2" event={"ID":"42e3b8e0-8d8e-4d30-b7b0-ce97fd72afed","Type":"ContainerStarted","Data":"7f45739fad4c828f8c0716e0d25c50ab2dfc0384310b8e70fdff9e8e58ff20a2"}
Sep 07 12:10:04.315117 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:10:04.315081    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w2-bff0c9-openshift-default-5ccb878dbd-vv8g2"
Sep 07 12:10:04.315368 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:10:04.315332    2563 patch_prober.go:28] interesting pod/e2e-shared-b-w2-bff0c9-openshift-default-5ccb878dbd-vv8g2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" start-of-body=
Sep 07 12:10:04.315440 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:10:04.315393    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-bff0c9-openshift-default-5ccb878dbd-vv8g2" podUID="42e3b8e0-8d8e-4d30-b7b0-ce97fd72afed" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused"
Sep 07 12:10:05.314553 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:10:05.314517    2563 patch_prober.go:28] interesting pod/e2e-shared-b-w2-bff0c9-openshift-default-5ccb878dbd-vv8g2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" start-of-body=
Sep 07 12:10:05.314936 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:10:05.314572    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-bff0c9-openshift-default-5ccb878dbd-vv8g2" podUID="42e3b8e0-8d8e-4d30-b7b0-ce97fd72afed" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused"
Sep 07 12:10:06.314471 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:10:06.314432    2563 patch_prober.go:28] interesting pod/e2e-shared-b-w2-bff0c9-openshift-default-5ccb878dbd-vv8g2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" start-of-body=
Sep 07 12:10:06.314672 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:10:06.314494    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-bff0c9-openshift-default-5ccb878dbd-vv8g2" podUID="42e3b8e0-8d8e-4d30-b7b0-ce97fd72afed" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused"
Sep 07 12:10:07.049046 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:10:07.049002    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:10:07.314920 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:10:07.314830    2563 patch_prober.go:28] interesting pod/e2e-shared-b-w2-bff0c9-openshift-default-5ccb878dbd-vv8g2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" start-of-body=
Sep 07 12:10:07.314920 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:10:07.314887    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-bff0c9-openshift-default-5ccb878dbd-vv8g2" podUID="42e3b8e0-8d8e-4d30-b7b0-ce97fd72afed" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused"
Sep 07 12:10:08.105833 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:10:08.105794    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-p4l4w"]
Sep 07 12:10:08.314807 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:10:08.314768    2563 patch_prober.go:28] interesting pod/e2e-shared-b-w2-bff0c9-openshift-default-5ccb878dbd-vv8g2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" start-of-body=
Sep 07 12:10:08.315090 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:10:08.315061    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-bff0c9-openshift-default-5ccb878dbd-vv8g2" podUID="42e3b8e0-8d8e-4d30-b7b0-ce97fd72afed" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused"
Sep 07 12:10:08.808819 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:10:08.808771    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:10:09.030189 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:10:09.030152    2563 scope.go:117] "RemoveContainer" containerID="a728e493ed4ed66bd8d047db3a715d0f5dd7e007d51879724b6c2c6d13aa2435"
Sep 07 12:10:09.314835 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:10:09.314802    2563 patch_prober.go:28] interesting pod/e2e-shared-b-w2-bff0c9-openshift-default-5ccb878dbd-vv8g2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" start-of-body=
Sep 07 12:10:09.315000 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:10:09.314856    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-bff0c9-openshift-default-5ccb878dbd-vv8g2" podUID="42e3b8e0-8d8e-4d30-b7b0-ce97fd72afed" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused"
Sep 07 12:10:10.318928 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:10:10.318895    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w2-bff0c9-openshift-default-5ccb878dbd-vv8g2"
Sep 07 12:10:10.319356 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:10:10.319335    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w2-bff0c9-openshift-default-5ccb878dbd-vv8g2"
Sep 07 12:10:10.319950 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:10:10.319931    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w2-bff0c9-openshift-default-5ccb878dbd-vv8g2"
Sep 07 12:10:11.315251 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:10:11.315219    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-w3-491296/auth-test-model-491296-kserve-775ddd7b8c-cklcl"
Sep 07 12:10:11.315409 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:10:11.315261    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-w3-491296/auth-test-model-491296-kserve-775ddd7b8c-cklcl"
Sep 07 12:10:11.327729 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:10:11.327706    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-w3-491296/auth-test-model-491296-kserve-775ddd7b8c-cklcl"
Sep 07 12:10:12.170294 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:10:12.170262    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-w3-491296/auth-test-model-491296-kserve-775ddd7b8c-cklcl"
Sep 07 12:10:12.894628 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:10:12.894594    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7494-140427222083024"]
Sep 07 12:10:12.903706 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:10:12.903676    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7494-140427222083024"
Sep 07 12:10:12.904878 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:10:12.904852    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7494-140427222083024"]
Sep 07 12:10:12.905643 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:10:12.905622    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-c7bf7\""
Sep 07 12:10:13.046256 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:10:13.046221    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ht2sc\" (UniqueName: \"kubernetes.io/projected/16da7892-a4c5-4aeb-a6bf-0b204a33b1b9-kube-api-access-ht2sc\") pod \"test-curl-7494-140427222083024\" (UID: \"16da7892-a4c5-4aeb-a6bf-0b204a33b1b9\") " pod="opendatahub/test-curl-7494-140427222083024"
Sep 07 12:10:13.147518 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:10:13.147433    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ht2sc\" (UniqueName: \"kubernetes.io/projected/16da7892-a4c5-4aeb-a6bf-0b204a33b1b9-kube-api-access-ht2sc\") pod \"test-curl-7494-140427222083024\" (UID: \"16da7892-a4c5-4aeb-a6bf-0b204a33b1b9\") " pod="opendatahub/test-curl-7494-140427222083024"
Sep 07 12:10:13.154973 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:10:13.154950    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ht2sc\" (UniqueName: \"kubernetes.io/projected/16da7892-a4c5-4aeb-a6bf-0b204a33b1b9-kube-api-access-ht2sc\") pod \"test-curl-7494-140427222083024\" (UID: \"16da7892-a4c5-4aeb-a6bf-0b204a33b1b9\") " pod="opendatahub/test-curl-7494-140427222083024"
Sep 07 12:10:13.214468 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:10:13.214435    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7494-140427222083024"
Sep 07 12:10:13.484981 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:10:13.484956    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7494-140427222083024"]
Sep 07 12:10:13.486994 ip-10-0-135-173 kubenswrapper[2563]: W0907 12:10:13.486966    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod16da7892_a4c5_4aeb_a6bf_0b204a33b1b9.slice/crio-8c79eff8e4fcc6b7ec92eee8fcb3c8574ea4ba6d7b4f8551c884290d890e4e56 WatchSource:0}: Error finding container 8c79eff8e4fcc6b7ec92eee8fcb3c8574ea4ba6d7b4f8551c884290d890e4e56: Status 404 returned error can't find the container with id 8c79eff8e4fcc6b7ec92eee8fcb3c8574ea4ba6d7b4f8551c884290d890e4e56
Sep 07 12:10:14.170364 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:10:14.170332    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7494-140427222083024" event={"ID":"16da7892-a4c5-4aeb-a6bf-0b204a33b1b9","Type":"ContainerStarted","Data":"8c79eff8e4fcc6b7ec92eee8fcb3c8574ea4ba6d7b4f8551c884290d890e4e56"}
Sep 07 12:10:15.180029 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:10:15.179995    2563 generic.go:356] "Generic (PLEG): container finished" podID="16da7892-a4c5-4aeb-a6bf-0b204a33b1b9" containerID="09c29f8ee26c6ff0621cfa6390ad75364ee5fa4474d009bc8ce22866944ecf6e" exitCode=0
Sep 07 12:10:15.180402 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:10:15.180080    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7494-140427222083024" event={"ID":"16da7892-a4c5-4aeb-a6bf-0b204a33b1b9","Type":"ContainerDied","Data":"09c29f8ee26c6ff0621cfa6390ad75364ee5fa4474d009bc8ce22866944ecf6e"}
Sep 07 12:10:16.312771 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:10:16.312746    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7494-140427222083024"
Sep 07 12:10:16.329957 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:10:16.329930    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7494-140427222083024_16da7892-a4c5-4aeb-a6bf-0b204a33b1b9/test-curl-7494-140427222083024/0.log"
Sep 07 12:10:16.353600 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:10:16.353576    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7494-140427222083024"]
Sep 07 12:10:16.357384 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:10:16.357362    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7494-140427222083024"]
Sep 07 12:10:16.479927 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:10:16.479902    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ht2sc\" (UniqueName: \"kubernetes.io/projected/16da7892-a4c5-4aeb-a6bf-0b204a33b1b9-kube-api-access-ht2sc\") pod \"16da7892-a4c5-4aeb-a6bf-0b204a33b1b9\" (UID: \"16da7892-a4c5-4aeb-a6bf-0b204a33b1b9\") "
Sep 07 12:10:16.481942 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:10:16.481922    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/16da7892-a4c5-4aeb-a6bf-0b204a33b1b9-kube-api-access-ht2sc" (OuterVolumeSpecName: "kube-api-access-ht2sc") pod "16da7892-a4c5-4aeb-a6bf-0b204a33b1b9" (UID: "16da7892-a4c5-4aeb-a6bf-0b204a33b1b9"). InnerVolumeSpecName "kube-api-access-ht2sc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 12:10:16.581213 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:10:16.581178    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ht2sc\" (UniqueName: \"kubernetes.io/projected/16da7892-a4c5-4aeb-a6bf-0b204a33b1b9-kube-api-access-ht2sc\") on node \"ip-10-0-135-173.ec2.internal\" DevicePath \"\""
Sep 07 12:10:17.041538 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:10:17.041506    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="16da7892-a4c5-4aeb-a6bf-0b204a33b1b9" path="/var/lib/kubelet/pods/16da7892-a4c5-4aeb-a6bf-0b204a33b1b9/volumes"
Sep 07 12:10:17.190896 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:10:17.190866    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7494-140427222083024"
Sep 07 12:10:17.191078 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:10:17.190929    2563 scope.go:117] "RemoveContainer" containerID="09c29f8ee26c6ff0621cfa6390ad75364ee5fa4474d009bc8ce22866944ecf6e"
Sep 07 12:10:21.106669 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:10:21.106629    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:10:22.436682 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:10:22.436637    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:10:22.719062 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:10:22.719030    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-619f5b7c-openshift-default-77c9f9684d-sfckf"]
Sep 07 12:10:22.719518 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:10:22.719498    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="16da7892-a4c5-4aeb-a6bf-0b204a33b1b9" containerName="test-curl-7494-140427222083024"
Sep 07 12:10:22.719518 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:10:22.719519    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="16da7892-a4c5-4aeb-a6bf-0b204a33b1b9" containerName="test-curl-7494-140427222083024"
Sep 07 12:10:22.719677 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:10:22.719640    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="16da7892-a4c5-4aeb-a6bf-0b204a33b1b9" containerName="test-curl-7494-140427222083024"
Sep 07 12:10:22.724246 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:10:22.724228    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-619f5b7c-openshift-default-77c9f9684d-sfckf"
Sep 07 12:10:22.725949 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:10:22.725929    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-619f5b7c-openshift-default-dockercfg-n9f7r\""
Sep 07 12:10:22.732804 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:10:22.732060    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4ad3830f-29e1-43e4-b440-127e3c84faa6-istio-envoy\") pod \"e2e-mt-619f5b7c-openshift-default-77c9f9684d-sfckf\" (UID: \"4ad3830f-29e1-43e4-b440-127e3c84faa6\") " pod="openshift-ingress/e2e-mt-619f5b7c-openshift-default-77c9f9684d-sfckf"
Sep 07 12:10:22.732804 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:10:22.732116    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4ad3830f-29e1-43e4-b440-127e3c84faa6-istio-token\") pod \"e2e-mt-619f5b7c-openshift-default-77c9f9684d-sfckf\" (UID: \"4ad3830f-29e1-43e4-b440-127e3c84faa6\") " pod="openshift-ingress/e2e-mt-619f5b7c-openshift-default-77c9f9684d-sfckf"
Sep 07 12:10:22.732804 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:10:22.732165    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4ad3830f-29e1-43e4-b440-127e3c84faa6-istiod-ca-cert\") pod \"e2e-mt-619f5b7c-openshift-default-77c9f9684d-sfckf\" (UID: \"4ad3830f-29e1-43e4-b440-127e3c84faa6\") " pod="openshift-ingress/e2e-mt-619f5b7c-openshift-default-77c9f9684d-sfckf"
Sep 07 12:10:22.732804 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:10:22.732221    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4ad3830f-29e1-43e4-b440-127e3c84faa6-istio-podinfo\") pod \"e2e-mt-619f5b7c-openshift-default-77c9f9684d-sfckf\" (UID: \"4ad3830f-29e1-43e4-b440-127e3c84faa6\") " pod="openshift-ingress/e2e-mt-619f5b7c-openshift-default-77c9f9684d-sfckf"
Sep 07 12:10:22.732804 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:10:22.732267    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4ad3830f-29e1-43e4-b440-127e3c84faa6-credential-socket\") pod \"e2e-mt-619f5b7c-openshift-default-77c9f9684d-sfckf\" (UID: \"4ad3830f-29e1-43e4-b440-127e3c84faa6\") " pod="openshift-ingress/e2e-mt-619f5b7c-openshift-default-77c9f9684d-sfckf"
Sep 07 12:10:22.732804 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:10:22.732295    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4ad3830f-29e1-43e4-b440-127e3c84faa6-istio-data\") pod \"e2e-mt-619f5b7c-openshift-default-77c9f9684d-sfckf\" (UID: \"4ad3830f-29e1-43e4-b440-127e3c84faa6\") " pod="openshift-ingress/e2e-mt-619f5b7c-openshift-default-77c9f9684d-sfckf"
Sep 07 12:10:22.732804 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:10:22.732319    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zslwx\" (UniqueName: \"kubernetes.io/projected/4ad3830f-29e1-43e4-b440-127e3c84faa6-kube-api-access-zslwx\") pod \"e2e-mt-619f5b7c-openshift-default-77c9f9684d-sfckf\" (UID: \"4ad3830f-29e1-43e4-b440-127e3c84faa6\") " pod="openshift-ingress/e2e-mt-619f5b7c-openshift-default-77c9f9684d-sfckf"
Sep 07 12:10:22.732804 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:10:22.732358    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4ad3830f-29e1-43e4-b440-127e3c84faa6-workload-certs\") pod \"e2e-mt-619f5b7c-openshift-default-77c9f9684d-sfckf\" (UID: \"4ad3830f-29e1-43e4-b440-127e3c84faa6\") " pod="openshift-ingress/e2e-mt-619f5b7c-openshift-default-77c9f9684d-sfckf"
Sep 07 12:10:22.732804 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:10:22.732461    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4ad3830f-29e1-43e4-b440-127e3c84faa6-workload-socket\") pod \"e2e-mt-619f5b7c-openshift-default-77c9f9684d-sfckf\" (UID: \"4ad3830f-29e1-43e4-b440-127e3c84faa6\") " pod="openshift-ingress/e2e-mt-619f5b7c-openshift-default-77c9f9684d-sfckf"
Sep 07 12:10:22.733292 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:10:22.733200    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-619f5b7c-openshift-default-77c9f9684d-sfckf"]
Sep 07 12:10:22.747069 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:10:22.747028    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:10:22.833083 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:10:22.833046    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4ad3830f-29e1-43e4-b440-127e3c84faa6-workload-socket\") pod \"e2e-mt-619f5b7c-openshift-default-77c9f9684d-sfckf\" (UID: \"4ad3830f-29e1-43e4-b440-127e3c84faa6\") " pod="openshift-ingress/e2e-mt-619f5b7c-openshift-default-77c9f9684d-sfckf"
Sep 07 12:10:22.833255 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:10:22.833108    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4ad3830f-29e1-43e4-b440-127e3c84faa6-istio-envoy\") pod \"e2e-mt-619f5b7c-openshift-default-77c9f9684d-sfckf\" (UID: \"4ad3830f-29e1-43e4-b440-127e3c84faa6\") " pod="openshift-ingress/e2e-mt-619f5b7c-openshift-default-77c9f9684d-sfckf"
Sep 07 12:10:22.833255 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:10:22.833149    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4ad3830f-29e1-43e4-b440-127e3c84faa6-istio-token\") pod \"e2e-mt-619f5b7c-openshift-default-77c9f9684d-sfckf\" (UID: \"4ad3830f-29e1-43e4-b440-127e3c84faa6\") " pod="openshift-ingress/e2e-mt-619f5b7c-openshift-default-77c9f9684d-sfckf"
Sep 07 12:10:22.833255 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:10:22.833191    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4ad3830f-29e1-43e4-b440-127e3c84faa6-istiod-ca-cert\") pod \"e2e-mt-619f5b7c-openshift-default-77c9f9684d-sfckf\" (UID: \"4ad3830f-29e1-43e4-b440-127e3c84faa6\") " pod="openshift-ingress/e2e-mt-619f5b7c-openshift-default-77c9f9684d-sfckf"
Sep 07 12:10:22.833255 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:10:22.833235    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4ad3830f-29e1-43e4-b440-127e3c84faa6-istio-podinfo\") pod \"e2e-mt-619f5b7c-openshift-default-77c9f9684d-sfckf\" (UID: \"4ad3830f-29e1-43e4-b440-127e3c84faa6\") " pod="openshift-ingress/e2e-mt-619f5b7c-openshift-default-77c9f9684d-sfckf"
Sep 07 12:10:22.833523 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:10:22.833276    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4ad3830f-29e1-43e4-b440-127e3c84faa6-credential-socket\") pod \"e2e-mt-619f5b7c-openshift-default-77c9f9684d-sfckf\" (UID: \"4ad3830f-29e1-43e4-b440-127e3c84faa6\") " pod="openshift-ingress/e2e-mt-619f5b7c-openshift-default-77c9f9684d-sfckf"
Sep 07 12:10:22.833523 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:10:22.833303    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4ad3830f-29e1-43e4-b440-127e3c84faa6-istio-data\") pod \"e2e-mt-619f5b7c-openshift-default-77c9f9684d-sfckf\" (UID: \"4ad3830f-29e1-43e4-b440-127e3c84faa6\") " pod="openshift-ingress/e2e-mt-619f5b7c-openshift-default-77c9f9684d-sfckf"
Sep 07 12:10:22.833523 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:10:22.833331    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zslwx\" (UniqueName: \"kubernetes.io/projected/4ad3830f-29e1-43e4-b440-127e3c84faa6-kube-api-access-zslwx\") pod \"e2e-mt-619f5b7c-openshift-default-77c9f9684d-sfckf\" (UID: \"4ad3830f-29e1-43e4-b440-127e3c84faa6\") " pod="openshift-ingress/e2e-mt-619f5b7c-openshift-default-77c9f9684d-sfckf"
Sep 07 12:10:22.833523 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:10:22.833366    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4ad3830f-29e1-43e4-b440-127e3c84faa6-workload-certs\") pod \"e2e-mt-619f5b7c-openshift-default-77c9f9684d-sfckf\" (UID: \"4ad3830f-29e1-43e4-b440-127e3c84faa6\") " pod="openshift-ingress/e2e-mt-619f5b7c-openshift-default-77c9f9684d-sfckf"
Sep 07 12:10:22.833523 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:10:22.833514    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4ad3830f-29e1-43e4-b440-127e3c84faa6-workload-socket\") pod \"e2e-mt-619f5b7c-openshift-default-77c9f9684d-sfckf\" (UID: \"4ad3830f-29e1-43e4-b440-127e3c84faa6\") " pod="openshift-ingress/e2e-mt-619f5b7c-openshift-default-77c9f9684d-sfckf"
Sep 07 12:10:22.833849 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:10:22.833817    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4ad3830f-29e1-43e4-b440-127e3c84faa6-istio-data\") pod \"e2e-mt-619f5b7c-openshift-default-77c9f9684d-sfckf\" (UID: \"4ad3830f-29e1-43e4-b440-127e3c84faa6\") " pod="openshift-ingress/e2e-mt-619f5b7c-openshift-default-77c9f9684d-sfckf"
Sep 07 12:10:22.833975 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:10:22.833851    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4ad3830f-29e1-43e4-b440-127e3c84faa6-credential-socket\") pod \"e2e-mt-619f5b7c-openshift-default-77c9f9684d-sfckf\" (UID: \"4ad3830f-29e1-43e4-b440-127e3c84faa6\") " pod="openshift-ingress/e2e-mt-619f5b7c-openshift-default-77c9f9684d-sfckf"
Sep 07 12:10:22.833975 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:10:22.833942    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4ad3830f-29e1-43e4-b440-127e3c84faa6-istiod-ca-cert\") pod \"e2e-mt-619f5b7c-openshift-default-77c9f9684d-sfckf\" (UID: \"4ad3830f-29e1-43e4-b440-127e3c84faa6\") " pod="openshift-ingress/e2e-mt-619f5b7c-openshift-default-77c9f9684d-sfckf"
Sep 07 12:10:22.834064 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:10:22.834038    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4ad3830f-29e1-43e4-b440-127e3c84faa6-workload-certs\") pod \"e2e-mt-619f5b7c-openshift-default-77c9f9684d-sfckf\" (UID: \"4ad3830f-29e1-43e4-b440-127e3c84faa6\") " pod="openshift-ingress/e2e-mt-619f5b7c-openshift-default-77c9f9684d-sfckf"
Sep 07 12:10:22.835508 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:10:22.835486    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4ad3830f-29e1-43e4-b440-127e3c84faa6-istio-envoy\") pod \"e2e-mt-619f5b7c-openshift-default-77c9f9684d-sfckf\" (UID: \"4ad3830f-29e1-43e4-b440-127e3c84faa6\") " pod="openshift-ingress/e2e-mt-619f5b7c-openshift-default-77c9f9684d-sfckf"
Sep 07 12:10:22.836036 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:10:22.836013    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4ad3830f-29e1-43e4-b440-127e3c84faa6-istio-podinfo\") pod \"e2e-mt-619f5b7c-openshift-default-77c9f9684d-sfckf\" (UID: \"4ad3830f-29e1-43e4-b440-127e3c84faa6\") " pod="openshift-ingress/e2e-mt-619f5b7c-openshift-default-77c9f9684d-sfckf"
Sep 07 12:10:22.840428 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:10:22.840395    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4ad3830f-29e1-43e4-b440-127e3c84faa6-istio-token\") pod \"e2e-mt-619f5b7c-openshift-default-77c9f9684d-sfckf\" (UID: \"4ad3830f-29e1-43e4-b440-127e3c84faa6\") " pod="openshift-ingress/e2e-mt-619f5b7c-openshift-default-77c9f9684d-sfckf"
Sep 07 12:10:22.840622 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:10:22.840603    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zslwx\" (UniqueName: \"kubernetes.io/projected/4ad3830f-29e1-43e4-b440-127e3c84faa6-kube-api-access-zslwx\") pod \"e2e-mt-619f5b7c-openshift-default-77c9f9684d-sfckf\" (UID: \"4ad3830f-29e1-43e4-b440-127e3c84faa6\") " pod="openshift-ingress/e2e-mt-619f5b7c-openshift-default-77c9f9684d-sfckf"
Sep 07 12:10:23.037937 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:10:23.037846    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-619f5b7c-openshift-default-77c9f9684d-sfckf"
Sep 07 12:10:23.099749 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:10:23.099563    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:10:23.309964 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:10:23.309940    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-619f5b7c-openshift-default-77c9f9684d-sfckf"]
Sep 07 12:10:23.312085 ip-10-0-135-173 kubenswrapper[2563]: W0907 12:10:23.312049    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4ad3830f_29e1_43e4_b440_127e3c84faa6.slice/crio-0806beb904c04111f8285c6a217bb6bb72f1ac54a12dd619e18f696b1f417055 WatchSource:0}: Error finding container 0806beb904c04111f8285c6a217bb6bb72f1ac54a12dd619e18f696b1f417055: Status 404 returned error can't find the container with id 0806beb904c04111f8285c6a217bb6bb72f1ac54a12dd619e18f696b1f417055
Sep 07 12:10:23.314273 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:10:23.314242    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 07 12:10:23.314355 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:10:23.314305    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 07 12:10:23.314355 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:10:23.314335    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 07 12:10:24.230184 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:10:24.230144    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-619f5b7c-openshift-default-77c9f9684d-sfckf" event={"ID":"4ad3830f-29e1-43e4-b440-127e3c84faa6","Type":"ContainerStarted","Data":"f435c2eaa6957b5287f875248051e0c1f6d2742fc016860a600260b471de649a"}
Sep 07 12:10:24.230564 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:10:24.230245    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-619f5b7c-openshift-default-77c9f9684d-sfckf" event={"ID":"4ad3830f-29e1-43e4-b440-127e3c84faa6","Type":"ContainerStarted","Data":"0806beb904c04111f8285c6a217bb6bb72f1ac54a12dd619e18f696b1f417055"}
Sep 07 12:10:24.248505 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:10:24.248461    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-619f5b7c-openshift-default-77c9f9684d-sfckf" podStartSLOduration=2.248443651 podStartE2EDuration="2.248443651s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 12:10:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 12:10:24.246588306 +0000 UTC m=+1395.765320138" watchObservedRunningTime="2026-09-07 12:10:24.248443651 +0000 UTC m=+1395.767175454"
Sep 07 12:10:25.040713 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:10:25.040676    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-619f5b7c-openshift-default-77c9f9684d-sfckf"
Sep 07 12:10:25.043621 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:10:25.043592    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-619f5b7c-openshift-default-77c9f9684d-sfckf"
Sep 07 12:10:25.235240 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:10:25.235203    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-619f5b7c-openshift-default-77c9f9684d-sfckf"
Sep 07 12:10:25.236218 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:10:25.236198    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-619f5b7c-openshift-default-77c9f9684d-sfckf"
Sep 07 12:10:26.138542 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:10:26.138507    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:10:26.231348 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:10:26.231306    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:10:27.542020 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:10:27.541973    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:10:27.747997 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:10:27.747959    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:10:27.850826 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:10:27.850727    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:10:27.948134 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:10:27.948098    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:10:28.014799 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:10:28.014767    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:10:36.205483 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:10:36.205444    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-p4l4w"]
Sep 07 12:10:46.345863 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:10:46.345825    2563 generic.go:356] "Generic (PLEG): container finished" podID="4ad3830f-29e1-43e4-b440-127e3c84faa6" containerID="f435c2eaa6957b5287f875248051e0c1f6d2742fc016860a600260b471de649a" exitCode=0
Sep 07 12:10:46.346268 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:10:46.345889    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-619f5b7c-openshift-default-77c9f9684d-sfckf" event={"ID":"4ad3830f-29e1-43e4-b440-127e3c84faa6","Type":"ContainerDied","Data":"f435c2eaa6957b5287f875248051e0c1f6d2742fc016860a600260b471de649a"}
Sep 07 12:10:46.346320 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:10:46.346303    2563 scope.go:117] "RemoveContainer" containerID="f435c2eaa6957b5287f875248051e0c1f6d2742fc016860a600260b471de649a"
Sep 07 12:10:46.347503 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:10:46.347474    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 07 12:10:46.347577 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:10:46.347545    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 07 12:10:46.347613 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:10:46.347578    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 07 12:10:47.352245 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:10:47.352214    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-619f5b7c-openshift-default-77c9f9684d-sfckf" event={"ID":"4ad3830f-29e1-43e4-b440-127e3c84faa6","Type":"ContainerStarted","Data":"b8afeb380f02a94d7f8989d53652b491ffbeefa80f0d54a8859e88999de1393e"}
Sep 07 12:10:48.038494 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:10:48.038457    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-619f5b7c-openshift-default-77c9f9684d-sfckf"
Sep 07 12:10:48.038700 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:10:48.038680    2563 patch_prober.go:28] interesting pod/e2e-mt-619f5b7c-openshift-default-77c9f9684d-sfckf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" start-of-body=
Sep 07 12:10:48.038749 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:10:48.038720    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-619f5b7c-openshift-default-77c9f9684d-sfckf" podUID="4ad3830f-29e1-43e4-b440-127e3c84faa6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused"
Sep 07 12:10:49.038152 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:10:49.038115    2563 patch_prober.go:28] interesting pod/e2e-mt-619f5b7c-openshift-default-77c9f9684d-sfckf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" start-of-body=
Sep 07 12:10:49.038713 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:10:49.038169    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-619f5b7c-openshift-default-77c9f9684d-sfckf" podUID="4ad3830f-29e1-43e4-b440-127e3c84faa6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused"
Sep 07 12:10:50.038564 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:10:50.038531    2563 patch_prober.go:28] interesting pod/e2e-mt-619f5b7c-openshift-default-77c9f9684d-sfckf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" start-of-body=
Sep 07 12:10:50.038947 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:10:50.038582    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-619f5b7c-openshift-default-77c9f9684d-sfckf" podUID="4ad3830f-29e1-43e4-b440-127e3c84faa6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused"
Sep 07 12:10:51.038548 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:10:51.038514    2563 patch_prober.go:28] interesting pod/e2e-mt-619f5b7c-openshift-default-77c9f9684d-sfckf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" start-of-body=
Sep 07 12:10:51.038729 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:10:51.038579    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-619f5b7c-openshift-default-77c9f9684d-sfckf" podUID="4ad3830f-29e1-43e4-b440-127e3c84faa6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused"
Sep 07 12:10:51.070881 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:10:51.070847    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:10:52.038521 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:10:52.038485    2563 patch_prober.go:28] interesting pod/e2e-mt-619f5b7c-openshift-default-77c9f9684d-sfckf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" start-of-body=
Sep 07 12:10:52.038783 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:10:52.038543    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-619f5b7c-openshift-default-77c9f9684d-sfckf" podUID="4ad3830f-29e1-43e4-b440-127e3c84faa6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused"
Sep 07 12:10:53.038223 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:10:53.038190    2563 patch_prober.go:28] interesting pod/e2e-mt-619f5b7c-openshift-default-77c9f9684d-sfckf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" start-of-body=
Sep 07 12:10:53.038403 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:10:53.038240    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-619f5b7c-openshift-default-77c9f9684d-sfckf" podUID="4ad3830f-29e1-43e4-b440-127e3c84faa6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused"
Sep 07 12:10:54.038914 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:10:54.038877    2563 patch_prober.go:28] interesting pod/e2e-mt-619f5b7c-openshift-default-77c9f9684d-sfckf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" start-of-body=
Sep 07 12:10:54.039306 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:10:54.038957    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-619f5b7c-openshift-default-77c9f9684d-sfckf" podUID="4ad3830f-29e1-43e4-b440-127e3c84faa6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused"
Sep 07 12:10:55.042607 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:10:55.042576    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-619f5b7c-openshift-default-77c9f9684d-sfckf"
Sep 07 12:10:55.043080 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:10:55.042948    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-619f5b7c-openshift-default-77c9f9684d-sfckf"
Sep 07 12:10:55.043669 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:10:55.043648    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-619f5b7c-openshift-default-77c9f9684d-sfckf"
Sep 07 12:10:55.389589 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:10:55.389518    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-w3-613886/auth-test-model-613886-kserve-595985b959-4222v"]
Sep 07 12:10:55.392674 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:10:55.392649    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-613886/auth-test-model-613886-kserve-595985b959-4222v"
Sep 07 12:10:55.394763 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:10:55.394741    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-613886\"/\"kube-root-ca.crt\""
Sep 07 12:10:55.394919 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:10:55.394845    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-613886\"/\"default-dockercfg-9r7cs\""
Sep 07 12:10:55.394919 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:10:55.394844    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-613886\"/\"openshift-service-ca.crt\""
Sep 07 12:10:55.394919 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:10:55.394881    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-613886\"/\"auth-test-model-613886-kserve-self-signed-certs\""
Sep 07 12:10:55.405677 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:10:55.405650    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-613886/auth-test-model-613886-kserve-595985b959-4222v"]
Sep 07 12:10:55.411401 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:10:55.411366    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:10:55.548170 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:10:55.548143    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6209c8a2-bb44-41c3-9ca9-5b5858047114-home\") pod \"auth-test-model-613886-kserve-595985b959-4222v\" (UID: \"6209c8a2-bb44-41c3-9ca9-5b5858047114\") " pod="ai-tenant-e2e-shared-b-w3-613886/auth-test-model-613886-kserve-595985b959-4222v"
Sep 07 12:10:55.548297 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:10:55.548173    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6209c8a2-bb44-41c3-9ca9-5b5858047114-dshm\") pod \"auth-test-model-613886-kserve-595985b959-4222v\" (UID: \"6209c8a2-bb44-41c3-9ca9-5b5858047114\") " pod="ai-tenant-e2e-shared-b-w3-613886/auth-test-model-613886-kserve-595985b959-4222v"
Sep 07 12:10:55.548297 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:10:55.548191    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6209c8a2-bb44-41c3-9ca9-5b5858047114-tmp-dir\") pod \"auth-test-model-613886-kserve-595985b959-4222v\" (UID: \"6209c8a2-bb44-41c3-9ca9-5b5858047114\") " pod="ai-tenant-e2e-shared-b-w3-613886/auth-test-model-613886-kserve-595985b959-4222v"
Sep 07 12:10:55.548297 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:10:55.548242    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6209c8a2-bb44-41c3-9ca9-5b5858047114-model-cache\") pod \"auth-test-model-613886-kserve-595985b959-4222v\" (UID: \"6209c8a2-bb44-41c3-9ca9-5b5858047114\") " pod="ai-tenant-e2e-shared-b-w3-613886/auth-test-model-613886-kserve-595985b959-4222v"
Sep 07 12:10:55.548297 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:10:55.548286    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-klxbc\" (UniqueName: \"kubernetes.io/projected/6209c8a2-bb44-41c3-9ca9-5b5858047114-kube-api-access-klxbc\") pod \"auth-test-model-613886-kserve-595985b959-4222v\" (UID: \"6209c8a2-bb44-41c3-9ca9-5b5858047114\") " pod="ai-tenant-e2e-shared-b-w3-613886/auth-test-model-613886-kserve-595985b959-4222v"
Sep 07 12:10:55.548521 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:10:55.548314    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6209c8a2-bb44-41c3-9ca9-5b5858047114-tls-certs\") pod \"auth-test-model-613886-kserve-595985b959-4222v\" (UID: \"6209c8a2-bb44-41c3-9ca9-5b5858047114\") " pod="ai-tenant-e2e-shared-b-w3-613886/auth-test-model-613886-kserve-595985b959-4222v"
Sep 07 12:10:55.649624 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:10:55.649532    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6209c8a2-bb44-41c3-9ca9-5b5858047114-home\") pod \"auth-test-model-613886-kserve-595985b959-4222v\" (UID: \"6209c8a2-bb44-41c3-9ca9-5b5858047114\") " pod="ai-tenant-e2e-shared-b-w3-613886/auth-test-model-613886-kserve-595985b959-4222v"
Sep 07 12:10:55.649624 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:10:55.649579    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6209c8a2-bb44-41c3-9ca9-5b5858047114-dshm\") pod \"auth-test-model-613886-kserve-595985b959-4222v\" (UID: \"6209c8a2-bb44-41c3-9ca9-5b5858047114\") " pod="ai-tenant-e2e-shared-b-w3-613886/auth-test-model-613886-kserve-595985b959-4222v"
Sep 07 12:10:55.649624 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:10:55.649597    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6209c8a2-bb44-41c3-9ca9-5b5858047114-tmp-dir\") pod \"auth-test-model-613886-kserve-595985b959-4222v\" (UID: \"6209c8a2-bb44-41c3-9ca9-5b5858047114\") " pod="ai-tenant-e2e-shared-b-w3-613886/auth-test-model-613886-kserve-595985b959-4222v"
Sep 07 12:10:55.649624 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:10:55.649621    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6209c8a2-bb44-41c3-9ca9-5b5858047114-model-cache\") pod \"auth-test-model-613886-kserve-595985b959-4222v\" (UID: \"6209c8a2-bb44-41c3-9ca9-5b5858047114\") " pod="ai-tenant-e2e-shared-b-w3-613886/auth-test-model-613886-kserve-595985b959-4222v"
Sep 07 12:10:55.649884 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:10:55.649642    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-klxbc\" (UniqueName: \"kubernetes.io/projected/6209c8a2-bb44-41c3-9ca9-5b5858047114-kube-api-access-klxbc\") pod \"auth-test-model-613886-kserve-595985b959-4222v\" (UID: \"6209c8a2-bb44-41c3-9ca9-5b5858047114\") " pod="ai-tenant-e2e-shared-b-w3-613886/auth-test-model-613886-kserve-595985b959-4222v"
Sep 07 12:10:55.649884 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:10:55.649664    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6209c8a2-bb44-41c3-9ca9-5b5858047114-tls-certs\") pod \"auth-test-model-613886-kserve-595985b959-4222v\" (UID: \"6209c8a2-bb44-41c3-9ca9-5b5858047114\") " pod="ai-tenant-e2e-shared-b-w3-613886/auth-test-model-613886-kserve-595985b959-4222v"
Sep 07 12:10:55.650022 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:10:55.649991    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6209c8a2-bb44-41c3-9ca9-5b5858047114-home\") pod \"auth-test-model-613886-kserve-595985b959-4222v\" (UID: \"6209c8a2-bb44-41c3-9ca9-5b5858047114\") " pod="ai-tenant-e2e-shared-b-w3-613886/auth-test-model-613886-kserve-595985b959-4222v"
Sep 07 12:10:55.650103 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:10:55.650025    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6209c8a2-bb44-41c3-9ca9-5b5858047114-model-cache\") pod \"auth-test-model-613886-kserve-595985b959-4222v\" (UID: \"6209c8a2-bb44-41c3-9ca9-5b5858047114\") " pod="ai-tenant-e2e-shared-b-w3-613886/auth-test-model-613886-kserve-595985b959-4222v"
Sep 07 12:10:55.650103 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:10:55.650081    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6209c8a2-bb44-41c3-9ca9-5b5858047114-tmp-dir\") pod \"auth-test-model-613886-kserve-595985b959-4222v\" (UID: \"6209c8a2-bb44-41c3-9ca9-5b5858047114\") " pod="ai-tenant-e2e-shared-b-w3-613886/auth-test-model-613886-kserve-595985b959-4222v"
Sep 07 12:10:55.652001 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:10:55.651978    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6209c8a2-bb44-41c3-9ca9-5b5858047114-dshm\") pod \"auth-test-model-613886-kserve-595985b959-4222v\" (UID: \"6209c8a2-bb44-41c3-9ca9-5b5858047114\") " pod="ai-tenant-e2e-shared-b-w3-613886/auth-test-model-613886-kserve-595985b959-4222v"
Sep 07 12:10:55.652101 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:10:55.652089    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6209c8a2-bb44-41c3-9ca9-5b5858047114-tls-certs\") pod \"auth-test-model-613886-kserve-595985b959-4222v\" (UID: \"6209c8a2-bb44-41c3-9ca9-5b5858047114\") " pod="ai-tenant-e2e-shared-b-w3-613886/auth-test-model-613886-kserve-595985b959-4222v"
Sep 07 12:10:55.657080 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:10:55.657057    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-klxbc\" (UniqueName: \"kubernetes.io/projected/6209c8a2-bb44-41c3-9ca9-5b5858047114-kube-api-access-klxbc\") pod \"auth-test-model-613886-kserve-595985b959-4222v\" (UID: \"6209c8a2-bb44-41c3-9ca9-5b5858047114\") " pod="ai-tenant-e2e-shared-b-w3-613886/auth-test-model-613886-kserve-595985b959-4222v"
Sep 07 12:10:55.706570 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:10:55.706546    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-613886/auth-test-model-613886-kserve-595985b959-4222v"
Sep 07 12:10:55.976881 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:10:55.976856    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-613886/auth-test-model-613886-kserve-595985b959-4222v"]
Sep 07 12:10:55.979367 ip-10-0-135-173 kubenswrapper[2563]: W0907 12:10:55.979341    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6209c8a2_bb44_41c3_9ca9_5b5858047114.slice/crio-0bc72c4ccfad42ab7d44157d74e7a21787559b539d370774152d2b5e2f235710 WatchSource:0}: Error finding container 0bc72c4ccfad42ab7d44157d74e7a21787559b539d370774152d2b5e2f235710: Status 404 returned error can't find the container with id 0bc72c4ccfad42ab7d44157d74e7a21787559b539d370774152d2b5e2f235710
Sep 07 12:10:56.404549 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:10:56.404459    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-613886/auth-test-model-613886-kserve-595985b959-4222v" event={"ID":"6209c8a2-bb44-41c3-9ca9-5b5858047114","Type":"ContainerStarted","Data":"d009beee82f4c824e57c1a66ace1ef0e0f37fa52a3adf047eefec79b042bc334"}
Sep 07 12:10:56.404549 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:10:56.404509    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-613886/auth-test-model-613886-kserve-595985b959-4222v" event={"ID":"6209c8a2-bb44-41c3-9ca9-5b5858047114","Type":"ContainerStarted","Data":"0bc72c4ccfad42ab7d44157d74e7a21787559b539d370774152d2b5e2f235710"}
Sep 07 12:10:56.419687 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:10:56.419640    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-w3-613886/auth-test-model-613886-kserve-595985b959-4222v" podStartSLOduration=1.419624476 podStartE2EDuration="1.419624476s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 12:10:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 12:10:56.418104376 +0000 UTC m=+1427.936836190" watchObservedRunningTime="2026-09-07 12:10:56.419624476 +0000 UTC m=+1427.938356313"
Sep 07 12:11:00.207529 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:11:00.207492    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-p4l4w"]
Sep 07 12:11:05.707274 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:11:05.707192    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-w3-613886/auth-test-model-613886-kserve-595985b959-4222v"
Sep 07 12:11:05.707274 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:11:05.707234    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-w3-613886/auth-test-model-613886-kserve-595985b959-4222v"
Sep 07 12:11:05.719873 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:11:05.719847    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-w3-613886/auth-test-model-613886-kserve-595985b959-4222v"
Sep 07 12:11:06.465536 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:11:06.465506    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-w3-613886/auth-test-model-613886-kserve-595985b959-4222v"
Sep 07 12:11:07.609453 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:11:07.609397    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-mt-619f5b7c/e2e-lifecycle-model-619f5b7c-kserve-c44f876b7-tqs2s"]
Sep 07 12:11:07.612661 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:11:07.612645    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-619f5b7c/e2e-lifecycle-model-619f5b7c-kserve-c44f876b7-tqs2s"
Sep 07 12:11:07.615666 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:11:07.615644    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-619f5b7c\"/\"openshift-service-ca.crt\""
Sep 07 12:11:07.615850 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:11:07.615710    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-619f5b7c\"/\"kube-root-ca.crt\""
Sep 07 12:11:07.615850 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:11:07.615782    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-619f5b7c\"/\"e2e-lifecycle-model-619f5b7c-kserve-self-signed-certs\""
Sep 07 12:11:07.615850 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:11:07.615782    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-619f5b7c\"/\"default-dockercfg-sv8rr\""
Sep 07 12:11:07.623592 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:11:07.623566    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-619f5b7c/e2e-lifecycle-model-619f5b7c-kserve-c44f876b7-tqs2s"]
Sep 07 12:11:07.676243 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:11:07.676219    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0115b2ee-d578-4de1-a4c9-a4b6db2f18f8-home\") pod \"e2e-lifecycle-model-619f5b7c-kserve-c44f876b7-tqs2s\" (UID: \"0115b2ee-d578-4de1-a4c9-a4b6db2f18f8\") " pod="ai-tenant-e2e-mt-619f5b7c/e2e-lifecycle-model-619f5b7c-kserve-c44f876b7-tqs2s"
Sep 07 12:11:07.676348 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:11:07.676268    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0115b2ee-d578-4de1-a4c9-a4b6db2f18f8-tmp-dir\") pod \"e2e-lifecycle-model-619f5b7c-kserve-c44f876b7-tqs2s\" (UID: \"0115b2ee-d578-4de1-a4c9-a4b6db2f18f8\") " pod="ai-tenant-e2e-mt-619f5b7c/e2e-lifecycle-model-619f5b7c-kserve-c44f876b7-tqs2s"
Sep 07 12:11:07.676390 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:11:07.676364    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0115b2ee-d578-4de1-a4c9-a4b6db2f18f8-model-cache\") pod \"e2e-lifecycle-model-619f5b7c-kserve-c44f876b7-tqs2s\" (UID: \"0115b2ee-d578-4de1-a4c9-a4b6db2f18f8\") " pod="ai-tenant-e2e-mt-619f5b7c/e2e-lifecycle-model-619f5b7c-kserve-c44f876b7-tqs2s"
Sep 07 12:11:07.676473 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:11:07.676460    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0115b2ee-d578-4de1-a4c9-a4b6db2f18f8-dshm\") pod \"e2e-lifecycle-model-619f5b7c-kserve-c44f876b7-tqs2s\" (UID: \"0115b2ee-d578-4de1-a4c9-a4b6db2f18f8\") " pod="ai-tenant-e2e-mt-619f5b7c/e2e-lifecycle-model-619f5b7c-kserve-c44f876b7-tqs2s"
Sep 07 12:11:07.676516 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:11:07.676489    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0115b2ee-d578-4de1-a4c9-a4b6db2f18f8-tls-certs\") pod \"e2e-lifecycle-model-619f5b7c-kserve-c44f876b7-tqs2s\" (UID: \"0115b2ee-d578-4de1-a4c9-a4b6db2f18f8\") " pod="ai-tenant-e2e-mt-619f5b7c/e2e-lifecycle-model-619f5b7c-kserve-c44f876b7-tqs2s"
Sep 07 12:11:07.676554 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:11:07.676519    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7vqqv\" (UniqueName: \"kubernetes.io/projected/0115b2ee-d578-4de1-a4c9-a4b6db2f18f8-kube-api-access-7vqqv\") pod \"e2e-lifecycle-model-619f5b7c-kserve-c44f876b7-tqs2s\" (UID: \"0115b2ee-d578-4de1-a4c9-a4b6db2f18f8\") " pod="ai-tenant-e2e-mt-619f5b7c/e2e-lifecycle-model-619f5b7c-kserve-c44f876b7-tqs2s"
Sep 07 12:11:07.777429 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:11:07.777391    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0115b2ee-d578-4de1-a4c9-a4b6db2f18f8-home\") pod \"e2e-lifecycle-model-619f5b7c-kserve-c44f876b7-tqs2s\" (UID: \"0115b2ee-d578-4de1-a4c9-a4b6db2f18f8\") " pod="ai-tenant-e2e-mt-619f5b7c/e2e-lifecycle-model-619f5b7c-kserve-c44f876b7-tqs2s"
Sep 07 12:11:07.777548 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:11:07.777475    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0115b2ee-d578-4de1-a4c9-a4b6db2f18f8-tmp-dir\") pod \"e2e-lifecycle-model-619f5b7c-kserve-c44f876b7-tqs2s\" (UID: \"0115b2ee-d578-4de1-a4c9-a4b6db2f18f8\") " pod="ai-tenant-e2e-mt-619f5b7c/e2e-lifecycle-model-619f5b7c-kserve-c44f876b7-tqs2s"
Sep 07 12:11:07.777548 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:11:07.777517    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0115b2ee-d578-4de1-a4c9-a4b6db2f18f8-model-cache\") pod \"e2e-lifecycle-model-619f5b7c-kserve-c44f876b7-tqs2s\" (UID: \"0115b2ee-d578-4de1-a4c9-a4b6db2f18f8\") " pod="ai-tenant-e2e-mt-619f5b7c/e2e-lifecycle-model-619f5b7c-kserve-c44f876b7-tqs2s"
Sep 07 12:11:07.777673 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:11:07.777584    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0115b2ee-d578-4de1-a4c9-a4b6db2f18f8-dshm\") pod \"e2e-lifecycle-model-619f5b7c-kserve-c44f876b7-tqs2s\" (UID: \"0115b2ee-d578-4de1-a4c9-a4b6db2f18f8\") " pod="ai-tenant-e2e-mt-619f5b7c/e2e-lifecycle-model-619f5b7c-kserve-c44f876b7-tqs2s"
Sep 07 12:11:07.777673 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:11:07.777619    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0115b2ee-d578-4de1-a4c9-a4b6db2f18f8-tls-certs\") pod \"e2e-lifecycle-model-619f5b7c-kserve-c44f876b7-tqs2s\" (UID: \"0115b2ee-d578-4de1-a4c9-a4b6db2f18f8\") " pod="ai-tenant-e2e-mt-619f5b7c/e2e-lifecycle-model-619f5b7c-kserve-c44f876b7-tqs2s"
Sep 07 12:11:07.777673 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:11:07.777661    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7vqqv\" (UniqueName: \"kubernetes.io/projected/0115b2ee-d578-4de1-a4c9-a4b6db2f18f8-kube-api-access-7vqqv\") pod \"e2e-lifecycle-model-619f5b7c-kserve-c44f876b7-tqs2s\" (UID: \"0115b2ee-d578-4de1-a4c9-a4b6db2f18f8\") " pod="ai-tenant-e2e-mt-619f5b7c/e2e-lifecycle-model-619f5b7c-kserve-c44f876b7-tqs2s"
Sep 07 12:11:07.777838 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:11:07.777775    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0115b2ee-d578-4de1-a4c9-a4b6db2f18f8-home\") pod \"e2e-lifecycle-model-619f5b7c-kserve-c44f876b7-tqs2s\" (UID: \"0115b2ee-d578-4de1-a4c9-a4b6db2f18f8\") " pod="ai-tenant-e2e-mt-619f5b7c/e2e-lifecycle-model-619f5b7c-kserve-c44f876b7-tqs2s"
Sep 07 12:11:07.777895 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:11:07.777856    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0115b2ee-d578-4de1-a4c9-a4b6db2f18f8-tmp-dir\") pod \"e2e-lifecycle-model-619f5b7c-kserve-c44f876b7-tqs2s\" (UID: \"0115b2ee-d578-4de1-a4c9-a4b6db2f18f8\") " pod="ai-tenant-e2e-mt-619f5b7c/e2e-lifecycle-model-619f5b7c-kserve-c44f876b7-tqs2s"
Sep 07 12:11:07.777978 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:11:07.777957    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0115b2ee-d578-4de1-a4c9-a4b6db2f18f8-model-cache\") pod \"e2e-lifecycle-model-619f5b7c-kserve-c44f876b7-tqs2s\" (UID: \"0115b2ee-d578-4de1-a4c9-a4b6db2f18f8\") " pod="ai-tenant-e2e-mt-619f5b7c/e2e-lifecycle-model-619f5b7c-kserve-c44f876b7-tqs2s"
Sep 07 12:11:07.779957 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:11:07.779929    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0115b2ee-d578-4de1-a4c9-a4b6db2f18f8-dshm\") pod \"e2e-lifecycle-model-619f5b7c-kserve-c44f876b7-tqs2s\" (UID: \"0115b2ee-d578-4de1-a4c9-a4b6db2f18f8\") " pod="ai-tenant-e2e-mt-619f5b7c/e2e-lifecycle-model-619f5b7c-kserve-c44f876b7-tqs2s"
Sep 07 12:11:07.780094 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:11:07.780075    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0115b2ee-d578-4de1-a4c9-a4b6db2f18f8-tls-certs\") pod \"e2e-lifecycle-model-619f5b7c-kserve-c44f876b7-tqs2s\" (UID: \"0115b2ee-d578-4de1-a4c9-a4b6db2f18f8\") " pod="ai-tenant-e2e-mt-619f5b7c/e2e-lifecycle-model-619f5b7c-kserve-c44f876b7-tqs2s"
Sep 07 12:11:07.795506 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:11:07.795484    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7vqqv\" (UniqueName: \"kubernetes.io/projected/0115b2ee-d578-4de1-a4c9-a4b6db2f18f8-kube-api-access-7vqqv\") pod \"e2e-lifecycle-model-619f5b7c-kserve-c44f876b7-tqs2s\" (UID: \"0115b2ee-d578-4de1-a4c9-a4b6db2f18f8\") " pod="ai-tenant-e2e-mt-619f5b7c/e2e-lifecycle-model-619f5b7c-kserve-c44f876b7-tqs2s"
Sep 07 12:11:07.923839 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:11:07.923736    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-619f5b7c/e2e-lifecycle-model-619f5b7c-kserve-c44f876b7-tqs2s"
Sep 07 12:11:08.193978 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:11:08.193897    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-619f5b7c/e2e-lifecycle-model-619f5b7c-kserve-c44f876b7-tqs2s"]
Sep 07 12:11:08.196948 ip-10-0-135-173 kubenswrapper[2563]: W0907 12:11:08.196919    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0115b2ee_d578_4de1_a4c9_a4b6db2f18f8.slice/crio-0c9a9ff6f0ead77cb4a8c0b4237bc6b44ebdf78b161a82e195a1c5f15c15cbb9 WatchSource:0}: Error finding container 0c9a9ff6f0ead77cb4a8c0b4237bc6b44ebdf78b161a82e195a1c5f15c15cbb9: Status 404 returned error can't find the container with id 0c9a9ff6f0ead77cb4a8c0b4237bc6b44ebdf78b161a82e195a1c5f15c15cbb9
Sep 07 12:11:08.465956 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:11:08.465919    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-619f5b7c/e2e-lifecycle-model-619f5b7c-kserve-c44f876b7-tqs2s" event={"ID":"0115b2ee-d578-4de1-a4c9-a4b6db2f18f8","Type":"ContainerStarted","Data":"19ec1e8fbe2b6e8e56cef8ba19afe94a38dc9c64b719054eb8b8b2c17c449f2e"}
Sep 07 12:11:08.466144 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:11:08.465971    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-619f5b7c/e2e-lifecycle-model-619f5b7c-kserve-c44f876b7-tqs2s" event={"ID":"0115b2ee-d578-4de1-a4c9-a4b6db2f18f8","Type":"ContainerStarted","Data":"0c9a9ff6f0ead77cb4a8c0b4237bc6b44ebdf78b161a82e195a1c5f15c15cbb9"}
Sep 07 12:11:08.486117 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:11:08.486071    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-mt-619f5b7c/e2e-lifecycle-model-619f5b7c-kserve-c44f876b7-tqs2s" podStartSLOduration=1.486056314 podStartE2EDuration="1.486056314s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 12:11:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 12:11:08.484769332 +0000 UTC m=+1440.003501144" watchObservedRunningTime="2026-09-07 12:11:08.486056314 +0000 UTC m=+1440.004788126"
Sep 07 12:11:08.672818 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:11:08.672782    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:11:14.105826 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:11:14.105793    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-p4l4w"]
Sep 07 12:11:17.924271 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:11:17.924236    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-619f5b7c/e2e-lifecycle-model-619f5b7c-kserve-c44f876b7-tqs2s"
Sep 07 12:11:17.924694 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:11:17.924493    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-mt-619f5b7c/e2e-lifecycle-model-619f5b7c-kserve-c44f876b7-tqs2s"
Sep 07 12:11:17.937212 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:11:17.937190    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-mt-619f5b7c/e2e-lifecycle-model-619f5b7c-kserve-c44f876b7-tqs2s"
Sep 07 12:11:18.527521 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:11:18.527490    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-619f5b7c/e2e-lifecycle-model-619f5b7c-kserve-c44f876b7-tqs2s"
Sep 07 12:11:23.140427 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:11:23.140370    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:11:45.298883 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:11:45.298848    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-p4l4w"]
Sep 07 12:12:01.232633 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:12:01.232594    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:12:04.200740 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:12:04.200702    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-p4l4w"]
Sep 07 12:12:09.202668 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:12:09.202627    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-4nfbc_b34ca85d-ac51-4724-a508-1026106b3f3a/console-operator/2.log"
Sep 07 12:12:09.205327 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:12:09.205303    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-4nfbc_b34ca85d-ac51-4724-a508-1026106b3f3a/console-operator/2.log"
Sep 07 12:12:16.639006 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:12:16.638965    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:12:16.773786 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:12:16.773750    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:12:16.876802 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:12:16.876761    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:12:16.961653 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:12:16.961615    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:12:17.031634 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:12:17.031605    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:12:18.816561 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:12:18.816530    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-p4l4w"]
Sep 07 12:12:20.039432 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:12:20.039382    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:12:25.020791 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:12:25.020757    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-619f5b7c/e2e-lifecycle-model-619f5b7c-kserve-c44f876b7-tqs2s"]
Sep 07 12:12:25.115040 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:12:25.115008    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:12:35.608862 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:12:35.608765    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-p4l4w"]
Sep 07 12:12:40.043053 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:12:40.042999    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-mt-619f5b7c/e2e-lifecycle-model-619f5b7c-kserve-c44f876b7-tqs2s" podUID="0115b2ee-d578-4de1-a4c9-a4b6db2f18f8" containerName="main" containerID="cri-o://19ec1e8fbe2b6e8e56cef8ba19afe94a38dc9c64b719054eb8b8b2c17c449f2e" gracePeriod=45
Sep 07 12:12:40.290470 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:12:40.290445    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-619f5b7c/e2e-lifecycle-model-619f5b7c-kserve-c44f876b7-tqs2s"
Sep 07 12:12:40.444545 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:12:40.444522    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0115b2ee-d578-4de1-a4c9-a4b6db2f18f8-model-cache\") pod \"0115b2ee-d578-4de1-a4c9-a4b6db2f18f8\" (UID: \"0115b2ee-d578-4de1-a4c9-a4b6db2f18f8\") "
Sep 07 12:12:40.444691 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:12:40.444596    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0115b2ee-d578-4de1-a4c9-a4b6db2f18f8-dshm\") pod \"0115b2ee-d578-4de1-a4c9-a4b6db2f18f8\" (UID: \"0115b2ee-d578-4de1-a4c9-a4b6db2f18f8\") "
Sep 07 12:12:40.444691 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:12:40.444632    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0115b2ee-d578-4de1-a4c9-a4b6db2f18f8-home\") pod \"0115b2ee-d578-4de1-a4c9-a4b6db2f18f8\" (UID: \"0115b2ee-d578-4de1-a4c9-a4b6db2f18f8\") "
Sep 07 12:12:40.444691 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:12:40.444682    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0115b2ee-d578-4de1-a4c9-a4b6db2f18f8-tls-certs\") pod \"0115b2ee-d578-4de1-a4c9-a4b6db2f18f8\" (UID: \"0115b2ee-d578-4de1-a4c9-a4b6db2f18f8\") "
Sep 07 12:12:40.444857 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:12:40.444716    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7vqqv\" (UniqueName: \"kubernetes.io/projected/0115b2ee-d578-4de1-a4c9-a4b6db2f18f8-kube-api-access-7vqqv\") pod \"0115b2ee-d578-4de1-a4c9-a4b6db2f18f8\" (UID: \"0115b2ee-d578-4de1-a4c9-a4b6db2f18f8\") "
Sep 07 12:12:40.444857 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:12:40.444777    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0115b2ee-d578-4de1-a4c9-a4b6db2f18f8-model-cache" (OuterVolumeSpecName: "model-cache") pod "0115b2ee-d578-4de1-a4c9-a4b6db2f18f8" (UID: "0115b2ee-d578-4de1-a4c9-a4b6db2f18f8"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 12:12:40.444857 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:12:40.444787    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0115b2ee-d578-4de1-a4c9-a4b6db2f18f8-tmp-dir\") pod \"0115b2ee-d578-4de1-a4c9-a4b6db2f18f8\" (UID: \"0115b2ee-d578-4de1-a4c9-a4b6db2f18f8\") "
Sep 07 12:12:40.445007 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:12:40.444942    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0115b2ee-d578-4de1-a4c9-a4b6db2f18f8-home" (OuterVolumeSpecName: "home") pod "0115b2ee-d578-4de1-a4c9-a4b6db2f18f8" (UID: "0115b2ee-d578-4de1-a4c9-a4b6db2f18f8"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 12:12:40.445066 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:12:40.445020    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0115b2ee-d578-4de1-a4c9-a4b6db2f18f8-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "0115b2ee-d578-4de1-a4c9-a4b6db2f18f8" (UID: "0115b2ee-d578-4de1-a4c9-a4b6db2f18f8"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 12:12:40.445197 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:12:40.445179    2563 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0115b2ee-d578-4de1-a4c9-a4b6db2f18f8-home\") on node \"ip-10-0-135-173.ec2.internal\" DevicePath \"\""
Sep 07 12:12:40.445257 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:12:40.445200    2563 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0115b2ee-d578-4de1-a4c9-a4b6db2f18f8-tmp-dir\") on node \"ip-10-0-135-173.ec2.internal\" DevicePath \"\""
Sep 07 12:12:40.445257 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:12:40.445211    2563 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0115b2ee-d578-4de1-a4c9-a4b6db2f18f8-model-cache\") on node \"ip-10-0-135-173.ec2.internal\" DevicePath \"\""
Sep 07 12:12:40.446865 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:12:40.446842    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0115b2ee-d578-4de1-a4c9-a4b6db2f18f8-dshm" (OuterVolumeSpecName: "dshm") pod "0115b2ee-d578-4de1-a4c9-a4b6db2f18f8" (UID: "0115b2ee-d578-4de1-a4c9-a4b6db2f18f8"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 12:12:40.446975 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:12:40.446896    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0115b2ee-d578-4de1-a4c9-a4b6db2f18f8-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "0115b2ee-d578-4de1-a4c9-a4b6db2f18f8" (UID: "0115b2ee-d578-4de1-a4c9-a4b6db2f18f8"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 07 12:12:40.446975 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:12:40.446938    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0115b2ee-d578-4de1-a4c9-a4b6db2f18f8-kube-api-access-7vqqv" (OuterVolumeSpecName: "kube-api-access-7vqqv") pod "0115b2ee-d578-4de1-a4c9-a4b6db2f18f8" (UID: "0115b2ee-d578-4de1-a4c9-a4b6db2f18f8"). InnerVolumeSpecName "kube-api-access-7vqqv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 12:12:40.546044 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:12:40.546020    2563 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0115b2ee-d578-4de1-a4c9-a4b6db2f18f8-tls-certs\") on node \"ip-10-0-135-173.ec2.internal\" DevicePath \"\""
Sep 07 12:12:40.546044 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:12:40.546043    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7vqqv\" (UniqueName: \"kubernetes.io/projected/0115b2ee-d578-4de1-a4c9-a4b6db2f18f8-kube-api-access-7vqqv\") on node \"ip-10-0-135-173.ec2.internal\" DevicePath \"\""
Sep 07 12:12:40.546182 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:12:40.546053    2563 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0115b2ee-d578-4de1-a4c9-a4b6db2f18f8-dshm\") on node \"ip-10-0-135-173.ec2.internal\" DevicePath \"\""
Sep 07 12:12:40.930777 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:12:40.930692    2563 generic.go:356] "Generic (PLEG): container finished" podID="0115b2ee-d578-4de1-a4c9-a4b6db2f18f8" containerID="19ec1e8fbe2b6e8e56cef8ba19afe94a38dc9c64b719054eb8b8b2c17c449f2e" exitCode=0
Sep 07 12:12:40.930777 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:12:40.930738    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-619f5b7c/e2e-lifecycle-model-619f5b7c-kserve-c44f876b7-tqs2s"
Sep 07 12:12:40.930959 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:12:40.930743    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-619f5b7c/e2e-lifecycle-model-619f5b7c-kserve-c44f876b7-tqs2s" event={"ID":"0115b2ee-d578-4de1-a4c9-a4b6db2f18f8","Type":"ContainerDied","Data":"19ec1e8fbe2b6e8e56cef8ba19afe94a38dc9c64b719054eb8b8b2c17c449f2e"}
Sep 07 12:12:40.930959 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:12:40.930854    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-619f5b7c/e2e-lifecycle-model-619f5b7c-kserve-c44f876b7-tqs2s" event={"ID":"0115b2ee-d578-4de1-a4c9-a4b6db2f18f8","Type":"ContainerDied","Data":"0c9a9ff6f0ead77cb4a8c0b4237bc6b44ebdf78b161a82e195a1c5f15c15cbb9"}
Sep 07 12:12:40.930959 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:12:40.930874    2563 scope.go:117] "RemoveContainer" containerID="19ec1e8fbe2b6e8e56cef8ba19afe94a38dc9c64b719054eb8b8b2c17c449f2e"
Sep 07 12:12:40.940206 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:12:40.940189    2563 scope.go:117] "RemoveContainer" containerID="19ec1e8fbe2b6e8e56cef8ba19afe94a38dc9c64b719054eb8b8b2c17c449f2e"
Sep 07 12:12:40.940475 ip-10-0-135-173 kubenswrapper[2563]: E0907 12:12:40.940452    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"19ec1e8fbe2b6e8e56cef8ba19afe94a38dc9c64b719054eb8b8b2c17c449f2e\": container with ID starting with 19ec1e8fbe2b6e8e56cef8ba19afe94a38dc9c64b719054eb8b8b2c17c449f2e not found: ID does not exist" containerID="19ec1e8fbe2b6e8e56cef8ba19afe94a38dc9c64b719054eb8b8b2c17c449f2e"
Sep 07 12:12:40.940538 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:12:40.940485    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"19ec1e8fbe2b6e8e56cef8ba19afe94a38dc9c64b719054eb8b8b2c17c449f2e"} err="failed to get container status \"19ec1e8fbe2b6e8e56cef8ba19afe94a38dc9c64b719054eb8b8b2c17c449f2e\": rpc error: code = NotFound desc = could not find container \"19ec1e8fbe2b6e8e56cef8ba19afe94a38dc9c64b719054eb8b8b2c17c449f2e\": container with ID starting with 19ec1e8fbe2b6e8e56cef8ba19afe94a38dc9c64b719054eb8b8b2c17c449f2e not found: ID does not exist"
Sep 07 12:12:40.949940 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:12:40.949917    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-619f5b7c/e2e-lifecycle-model-619f5b7c-kserve-c44f876b7-tqs2s"]
Sep 07 12:12:40.953455 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:12:40.953406    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-mt-619f5b7c/e2e-lifecycle-model-619f5b7c-kserve-c44f876b7-tqs2s"]
Sep 07 12:12:41.041141 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:12:41.041118    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0115b2ee-d578-4de1-a4c9-a4b6db2f18f8" path="/var/lib/kubelet/pods/0115b2ee-d578-4de1-a4c9-a4b6db2f18f8/volumes"
Sep 07 12:12:52.599649 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:12:52.599615    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-p4l4w"]
Sep 07 12:13:01.424037 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:13:01.423980    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:13:02.125349 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:13:02.125300    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:13:07.829016 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:13:07.828977    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:13:10.002259 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:13:10.002220    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-p4l4w"]
Sep 07 12:13:13.740616 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:13:13.740577    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:13:13.840306 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:13:13.840272    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:13:13.972259 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:13:13.972229    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-619f5b7c-openshift-default-77c9f9684d-sfckf"]
Sep 07 12:13:13.972544 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:13:13.972517    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-619f5b7c-openshift-default-77c9f9684d-sfckf" podUID="4ad3830f-29e1-43e4-b440-127e3c84faa6" containerName="istio-proxy" containerID="cri-o://b8afeb380f02a94d7f8989d53652b491ffbeefa80f0d54a8859e88999de1393e" gracePeriod=30
Sep 07 12:13:13.994101 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:13:13.994031    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:13:15.708027 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:13:15.707983    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:13:15.757475 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:13:15.757441    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:13:15.942501 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:13:15.942466    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:13:16.042011 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:13:16.041897    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:13:16.142551 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:13:16.142521    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:13:17.411317 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:13:17.411278    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:13:19.125486 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:13:19.125455    2563 generic.go:356] "Generic (PLEG): container finished" podID="4ad3830f-29e1-43e4-b440-127e3c84faa6" containerID="b8afeb380f02a94d7f8989d53652b491ffbeefa80f0d54a8859e88999de1393e" exitCode=0
Sep 07 12:13:19.125877 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:13:19.125528    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-619f5b7c-openshift-default-77c9f9684d-sfckf" event={"ID":"4ad3830f-29e1-43e4-b440-127e3c84faa6","Type":"ContainerDied","Data":"b8afeb380f02a94d7f8989d53652b491ffbeefa80f0d54a8859e88999de1393e"}
Sep 07 12:13:19.125877 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:13:19.125590    2563 scope.go:117] "RemoveContainer" containerID="f435c2eaa6957b5287f875248051e0c1f6d2742fc016860a600260b471de649a"
Sep 07 12:13:19.226741 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:13:19.226718    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-619f5b7c-openshift-default-77c9f9684d-sfckf"
Sep 07 12:13:19.305023 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:13:19.304952    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4ad3830f-29e1-43e4-b440-127e3c84faa6-istio-podinfo\") pod \"4ad3830f-29e1-43e4-b440-127e3c84faa6\" (UID: \"4ad3830f-29e1-43e4-b440-127e3c84faa6\") "
Sep 07 12:13:19.305023 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:13:19.305007    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4ad3830f-29e1-43e4-b440-127e3c84faa6-istio-envoy\") pod \"4ad3830f-29e1-43e4-b440-127e3c84faa6\" (UID: \"4ad3830f-29e1-43e4-b440-127e3c84faa6\") "
Sep 07 12:13:19.305197 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:13:19.305026    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zslwx\" (UniqueName: \"kubernetes.io/projected/4ad3830f-29e1-43e4-b440-127e3c84faa6-kube-api-access-zslwx\") pod \"4ad3830f-29e1-43e4-b440-127e3c84faa6\" (UID: \"4ad3830f-29e1-43e4-b440-127e3c84faa6\") "
Sep 07 12:13:19.305197 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:13:19.305067    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4ad3830f-29e1-43e4-b440-127e3c84faa6-workload-socket\") pod \"4ad3830f-29e1-43e4-b440-127e3c84faa6\" (UID: \"4ad3830f-29e1-43e4-b440-127e3c84faa6\") "
Sep 07 12:13:19.305197 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:13:19.305098    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4ad3830f-29e1-43e4-b440-127e3c84faa6-istio-token\") pod \"4ad3830f-29e1-43e4-b440-127e3c84faa6\" (UID: \"4ad3830f-29e1-43e4-b440-127e3c84faa6\") "
Sep 07 12:13:19.305197 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:13:19.305122    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4ad3830f-29e1-43e4-b440-127e3c84faa6-istiod-ca-cert\") pod \"4ad3830f-29e1-43e4-b440-127e3c84faa6\" (UID: \"4ad3830f-29e1-43e4-b440-127e3c84faa6\") "
Sep 07 12:13:19.305197 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:13:19.305156    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4ad3830f-29e1-43e4-b440-127e3c84faa6-credential-socket\") pod \"4ad3830f-29e1-43e4-b440-127e3c84faa6\" (UID: \"4ad3830f-29e1-43e4-b440-127e3c84faa6\") "
Sep 07 12:13:19.305494 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:13:19.305226    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4ad3830f-29e1-43e4-b440-127e3c84faa6-istio-data\") pod \"4ad3830f-29e1-43e4-b440-127e3c84faa6\" (UID: \"4ad3830f-29e1-43e4-b440-127e3c84faa6\") "
Sep 07 12:13:19.305494 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:13:19.305266    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4ad3830f-29e1-43e4-b440-127e3c84faa6-workload-certs\") pod \"4ad3830f-29e1-43e4-b440-127e3c84faa6\" (UID: \"4ad3830f-29e1-43e4-b440-127e3c84faa6\") "
Sep 07 12:13:19.305494 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:13:19.305380    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4ad3830f-29e1-43e4-b440-127e3c84faa6-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "4ad3830f-29e1-43e4-b440-127e3c84faa6" (UID: "4ad3830f-29e1-43e4-b440-127e3c84faa6"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 12:13:19.305673 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:13:19.305565    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4ad3830f-29e1-43e4-b440-127e3c84faa6-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "4ad3830f-29e1-43e4-b440-127e3c84faa6" (UID: "4ad3830f-29e1-43e4-b440-127e3c84faa6"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 12:13:19.305673 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:13:19.305644    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4ad3830f-29e1-43e4-b440-127e3c84faa6-istio-data" (OuterVolumeSpecName: "istio-data") pod "4ad3830f-29e1-43e4-b440-127e3c84faa6" (UID: "4ad3830f-29e1-43e4-b440-127e3c84faa6"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 12:13:19.305783 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:13:19.305700    2563 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4ad3830f-29e1-43e4-b440-127e3c84faa6-workload-socket\") on node \"ip-10-0-135-173.ec2.internal\" DevicePath \"\""
Sep 07 12:13:19.305783 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:13:19.305721    2563 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4ad3830f-29e1-43e4-b440-127e3c84faa6-credential-socket\") on node \"ip-10-0-135-173.ec2.internal\" DevicePath \"\""
Sep 07 12:13:19.305891 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:13:19.305832    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4ad3830f-29e1-43e4-b440-127e3c84faa6-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "4ad3830f-29e1-43e4-b440-127e3c84faa6" (UID: "4ad3830f-29e1-43e4-b440-127e3c84faa6"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 12:13:19.305891 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:13:19.305760    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4ad3830f-29e1-43e4-b440-127e3c84faa6-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "4ad3830f-29e1-43e4-b440-127e3c84faa6" (UID: "4ad3830f-29e1-43e4-b440-127e3c84faa6"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 07 12:13:19.307280 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:13:19.307257    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/4ad3830f-29e1-43e4-b440-127e3c84faa6-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "4ad3830f-29e1-43e4-b440-127e3c84faa6" (UID: "4ad3830f-29e1-43e4-b440-127e3c84faa6"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 07 12:13:19.307392 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:13:19.307293    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4ad3830f-29e1-43e4-b440-127e3c84faa6-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "4ad3830f-29e1-43e4-b440-127e3c84faa6" (UID: "4ad3830f-29e1-43e4-b440-127e3c84faa6"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 12:13:19.307392 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:13:19.307320    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4ad3830f-29e1-43e4-b440-127e3c84faa6-istio-token" (OuterVolumeSpecName: "istio-token") pod "4ad3830f-29e1-43e4-b440-127e3c84faa6" (UID: "4ad3830f-29e1-43e4-b440-127e3c84faa6"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 12:13:19.307392 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:13:19.307351    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4ad3830f-29e1-43e4-b440-127e3c84faa6-kube-api-access-zslwx" (OuterVolumeSpecName: "kube-api-access-zslwx") pod "4ad3830f-29e1-43e4-b440-127e3c84faa6" (UID: "4ad3830f-29e1-43e4-b440-127e3c84faa6"). InnerVolumeSpecName "kube-api-access-zslwx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 12:13:19.407170 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:13:19.407143    2563 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4ad3830f-29e1-43e4-b440-127e3c84faa6-istio-token\") on node \"ip-10-0-135-173.ec2.internal\" DevicePath \"\""
Sep 07 12:13:19.407170 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:13:19.407166    2563 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4ad3830f-29e1-43e4-b440-127e3c84faa6-istiod-ca-cert\") on node \"ip-10-0-135-173.ec2.internal\" DevicePath \"\""
Sep 07 12:13:19.407170 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:13:19.407175    2563 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4ad3830f-29e1-43e4-b440-127e3c84faa6-istio-data\") on node \"ip-10-0-135-173.ec2.internal\" DevicePath \"\""
Sep 07 12:13:19.407346 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:13:19.407183    2563 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4ad3830f-29e1-43e4-b440-127e3c84faa6-workload-certs\") on node \"ip-10-0-135-173.ec2.internal\" DevicePath \"\""
Sep 07 12:13:19.407346 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:13:19.407191    2563 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4ad3830f-29e1-43e4-b440-127e3c84faa6-istio-podinfo\") on node \"ip-10-0-135-173.ec2.internal\" DevicePath \"\""
Sep 07 12:13:19.407346 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:13:19.407199    2563 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4ad3830f-29e1-43e4-b440-127e3c84faa6-istio-envoy\") on node \"ip-10-0-135-173.ec2.internal\" DevicePath \"\""
Sep 07 12:13:19.407346 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:13:19.407206    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zslwx\" (UniqueName: \"kubernetes.io/projected/4ad3830f-29e1-43e4-b440-127e3c84faa6-kube-api-access-zslwx\") on node \"ip-10-0-135-173.ec2.internal\" DevicePath \"\""
Sep 07 12:13:20.132232 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:13:20.132201    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-619f5b7c-openshift-default-77c9f9684d-sfckf" event={"ID":"4ad3830f-29e1-43e4-b440-127e3c84faa6","Type":"ContainerDied","Data":"0806beb904c04111f8285c6a217bb6bb72f1ac54a12dd619e18f696b1f417055"}
Sep 07 12:13:20.132232 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:13:20.132224    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-619f5b7c-openshift-default-77c9f9684d-sfckf"
Sep 07 12:13:20.132750 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:13:20.132257    2563 scope.go:117] "RemoveContainer" containerID="b8afeb380f02a94d7f8989d53652b491ffbeefa80f0d54a8859e88999de1393e"
Sep 07 12:13:20.152950 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:13:20.152930    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-619f5b7c-openshift-default-77c9f9684d-sfckf"]
Sep 07 12:13:20.156482 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:13:20.156457    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-619f5b7c-openshift-default-77c9f9684d-sfckf"]
Sep 07 12:13:21.040676 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:13:21.040641    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4ad3830f-29e1-43e4-b440-127e3c84faa6" path="/var/lib/kubelet/pods/4ad3830f-29e1-43e4-b440-127e3c84faa6/volumes"
Sep 07 12:13:23.194846 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:13:23.194812    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:13:23.294984 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:13:23.294941    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:13:23.879559 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:13:23.879519    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:13:27.597774 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:13:27.597732    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-p4l4w"]
Sep 07 12:13:29.729245 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:13:29.729204    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:13:29.805280 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:13:29.805241    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:13:29.881766 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:13:29.881729    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:13:29.982259 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:13:29.982186    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:13:34.411365 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:13:34.411331    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-491296/auth-test-model-491296-kserve-775ddd7b8c-cklcl"]
Sep 07 12:13:34.433774 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:13:34.433742    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:13:35.381646 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:13:35.381617    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-613886/auth-test-model-613886-kserve-595985b959-4222v"]
Sep 07 12:13:35.399490 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:13:35.399462    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:13:40.423534 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:13:40.423497    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:13:40.640490 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:13:40.640447    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:13:43.310642 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:13:43.310613    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-w3-491296/sub-test-model-491296-kserve-86c54877cc-tjsw2"]
Sep 07 12:13:43.311166 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:13:43.311145    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4ad3830f-29e1-43e4-b440-127e3c84faa6" containerName="istio-proxy"
Sep 07 12:13:43.311252 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:13:43.311168    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="4ad3830f-29e1-43e4-b440-127e3c84faa6" containerName="istio-proxy"
Sep 07 12:13:43.311252 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:13:43.311203    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0115b2ee-d578-4de1-a4c9-a4b6db2f18f8" containerName="main"
Sep 07 12:13:43.311252 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:13:43.311212    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="0115b2ee-d578-4de1-a4c9-a4b6db2f18f8" containerName="main"
Sep 07 12:13:43.311252 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:13:43.311227    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4ad3830f-29e1-43e4-b440-127e3c84faa6" containerName="istio-proxy"
Sep 07 12:13:43.311252 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:13:43.311235    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="4ad3830f-29e1-43e4-b440-127e3c84faa6" containerName="istio-proxy"
Sep 07 12:13:43.311535 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:13:43.311348    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="4ad3830f-29e1-43e4-b440-127e3c84faa6" containerName="istio-proxy"
Sep 07 12:13:43.311535 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:13:43.311363    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="0115b2ee-d578-4de1-a4c9-a4b6db2f18f8" containerName="main"
Sep 07 12:13:43.314358 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:13:43.314337    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-491296/sub-test-model-491296-kserve-86c54877cc-tjsw2"
Sep 07 12:13:43.315984 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:13:43.315963    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-491296\"/\"sub-test-model-491296-kserve-self-signed-certs\""
Sep 07 12:13:43.318241 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:13:43.318218    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1d4bd4c0-9ee3-4300-b7dd-5bc764c3adb9-tmp-dir\") pod \"sub-test-model-491296-kserve-86c54877cc-tjsw2\" (UID: \"1d4bd4c0-9ee3-4300-b7dd-5bc764c3adb9\") " pod="ai-tenant-e2e-shared-a-w3-491296/sub-test-model-491296-kserve-86c54877cc-tjsw2"
Sep 07 12:13:43.318351 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:13:43.318268    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1d4bd4c0-9ee3-4300-b7dd-5bc764c3adb9-dshm\") pod \"sub-test-model-491296-kserve-86c54877cc-tjsw2\" (UID: \"1d4bd4c0-9ee3-4300-b7dd-5bc764c3adb9\") " pod="ai-tenant-e2e-shared-a-w3-491296/sub-test-model-491296-kserve-86c54877cc-tjsw2"
Sep 07 12:13:43.318407 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:13:43.318355    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1d4bd4c0-9ee3-4300-b7dd-5bc764c3adb9-tls-certs\") pod \"sub-test-model-491296-kserve-86c54877cc-tjsw2\" (UID: \"1d4bd4c0-9ee3-4300-b7dd-5bc764c3adb9\") " pod="ai-tenant-e2e-shared-a-w3-491296/sub-test-model-491296-kserve-86c54877cc-tjsw2"
Sep 07 12:13:43.318490 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:13:43.318405    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1d4bd4c0-9ee3-4300-b7dd-5bc764c3adb9-home\") pod \"sub-test-model-491296-kserve-86c54877cc-tjsw2\" (UID: \"1d4bd4c0-9ee3-4300-b7dd-5bc764c3adb9\") " pod="ai-tenant-e2e-shared-a-w3-491296/sub-test-model-491296-kserve-86c54877cc-tjsw2"
Sep 07 12:13:43.318551 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:13:43.318495    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1d4bd4c0-9ee3-4300-b7dd-5bc764c3adb9-model-cache\") pod \"sub-test-model-491296-kserve-86c54877cc-tjsw2\" (UID: \"1d4bd4c0-9ee3-4300-b7dd-5bc764c3adb9\") " pod="ai-tenant-e2e-shared-a-w3-491296/sub-test-model-491296-kserve-86c54877cc-tjsw2"
Sep 07 12:13:43.318602 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:13:43.318549    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-24kvg\" (UniqueName: \"kubernetes.io/projected/1d4bd4c0-9ee3-4300-b7dd-5bc764c3adb9-kube-api-access-24kvg\") pod \"sub-test-model-491296-kserve-86c54877cc-tjsw2\" (UID: \"1d4bd4c0-9ee3-4300-b7dd-5bc764c3adb9\") " pod="ai-tenant-e2e-shared-a-w3-491296/sub-test-model-491296-kserve-86c54877cc-tjsw2"
Sep 07 12:13:43.325788 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:13:43.325764    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-491296/sub-test-model-491296-kserve-86c54877cc-tjsw2"]
Sep 07 12:13:43.395641 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:13:43.395596    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:13:43.419036 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:13:43.419011    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1d4bd4c0-9ee3-4300-b7dd-5bc764c3adb9-dshm\") pod \"sub-test-model-491296-kserve-86c54877cc-tjsw2\" (UID: \"1d4bd4c0-9ee3-4300-b7dd-5bc764c3adb9\") " pod="ai-tenant-e2e-shared-a-w3-491296/sub-test-model-491296-kserve-86c54877cc-tjsw2"
Sep 07 12:13:43.419149 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:13:43.419048    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1d4bd4c0-9ee3-4300-b7dd-5bc764c3adb9-tls-certs\") pod \"sub-test-model-491296-kserve-86c54877cc-tjsw2\" (UID: \"1d4bd4c0-9ee3-4300-b7dd-5bc764c3adb9\") " pod="ai-tenant-e2e-shared-a-w3-491296/sub-test-model-491296-kserve-86c54877cc-tjsw2"
Sep 07 12:13:43.419149 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:13:43.419071    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1d4bd4c0-9ee3-4300-b7dd-5bc764c3adb9-home\") pod \"sub-test-model-491296-kserve-86c54877cc-tjsw2\" (UID: \"1d4bd4c0-9ee3-4300-b7dd-5bc764c3adb9\") " pod="ai-tenant-e2e-shared-a-w3-491296/sub-test-model-491296-kserve-86c54877cc-tjsw2"
Sep 07 12:13:43.419149 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:13:43.419100    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1d4bd4c0-9ee3-4300-b7dd-5bc764c3adb9-model-cache\") pod \"sub-test-model-491296-kserve-86c54877cc-tjsw2\" (UID: \"1d4bd4c0-9ee3-4300-b7dd-5bc764c3adb9\") " pod="ai-tenant-e2e-shared-a-w3-491296/sub-test-model-491296-kserve-86c54877cc-tjsw2"
Sep 07 12:13:43.419149 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:13:43.419138    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-24kvg\" (UniqueName: \"kubernetes.io/projected/1d4bd4c0-9ee3-4300-b7dd-5bc764c3adb9-kube-api-access-24kvg\") pod \"sub-test-model-491296-kserve-86c54877cc-tjsw2\" (UID: \"1d4bd4c0-9ee3-4300-b7dd-5bc764c3adb9\") " pod="ai-tenant-e2e-shared-a-w3-491296/sub-test-model-491296-kserve-86c54877cc-tjsw2"
Sep 07 12:13:43.419354 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:13:43.419172    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1d4bd4c0-9ee3-4300-b7dd-5bc764c3adb9-tmp-dir\") pod \"sub-test-model-491296-kserve-86c54877cc-tjsw2\" (UID: \"1d4bd4c0-9ee3-4300-b7dd-5bc764c3adb9\") " pod="ai-tenant-e2e-shared-a-w3-491296/sub-test-model-491296-kserve-86c54877cc-tjsw2"
Sep 07 12:13:43.419554 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:13:43.419532    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1d4bd4c0-9ee3-4300-b7dd-5bc764c3adb9-home\") pod \"sub-test-model-491296-kserve-86c54877cc-tjsw2\" (UID: \"1d4bd4c0-9ee3-4300-b7dd-5bc764c3adb9\") " pod="ai-tenant-e2e-shared-a-w3-491296/sub-test-model-491296-kserve-86c54877cc-tjsw2"
Sep 07 12:13:43.419633 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:13:43.419533    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1d4bd4c0-9ee3-4300-b7dd-5bc764c3adb9-model-cache\") pod \"sub-test-model-491296-kserve-86c54877cc-tjsw2\" (UID: \"1d4bd4c0-9ee3-4300-b7dd-5bc764c3adb9\") " pod="ai-tenant-e2e-shared-a-w3-491296/sub-test-model-491296-kserve-86c54877cc-tjsw2"
Sep 07 12:13:43.419633 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:13:43.419583    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1d4bd4c0-9ee3-4300-b7dd-5bc764c3adb9-tmp-dir\") pod \"sub-test-model-491296-kserve-86c54877cc-tjsw2\" (UID: \"1d4bd4c0-9ee3-4300-b7dd-5bc764c3adb9\") " pod="ai-tenant-e2e-shared-a-w3-491296/sub-test-model-491296-kserve-86c54877cc-tjsw2"
Sep 07 12:13:43.421195 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:13:43.421173    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1d4bd4c0-9ee3-4300-b7dd-5bc764c3adb9-dshm\") pod \"sub-test-model-491296-kserve-86c54877cc-tjsw2\" (UID: \"1d4bd4c0-9ee3-4300-b7dd-5bc764c3adb9\") " pod="ai-tenant-e2e-shared-a-w3-491296/sub-test-model-491296-kserve-86c54877cc-tjsw2"
Sep 07 12:13:43.421507 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:13:43.421489    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1d4bd4c0-9ee3-4300-b7dd-5bc764c3adb9-tls-certs\") pod \"sub-test-model-491296-kserve-86c54877cc-tjsw2\" (UID: \"1d4bd4c0-9ee3-4300-b7dd-5bc764c3adb9\") " pod="ai-tenant-e2e-shared-a-w3-491296/sub-test-model-491296-kserve-86c54877cc-tjsw2"
Sep 07 12:13:43.426110 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:13:43.426091    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-24kvg\" (UniqueName: \"kubernetes.io/projected/1d4bd4c0-9ee3-4300-b7dd-5bc764c3adb9-kube-api-access-24kvg\") pod \"sub-test-model-491296-kserve-86c54877cc-tjsw2\" (UID: \"1d4bd4c0-9ee3-4300-b7dd-5bc764c3adb9\") " pod="ai-tenant-e2e-shared-a-w3-491296/sub-test-model-491296-kserve-86c54877cc-tjsw2"
Sep 07 12:13:43.624347 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:13:43.624251    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-491296/sub-test-model-491296-kserve-86c54877cc-tjsw2"
Sep 07 12:13:43.893992 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:13:43.893970    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-491296/sub-test-model-491296-kserve-86c54877cc-tjsw2"]
Sep 07 12:13:43.896015 ip-10-0-135-173 kubenswrapper[2563]: W0907 12:13:43.895986    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1d4bd4c0_9ee3_4300_b7dd_5bc764c3adb9.slice/crio-39486421610c872249947fe35d0df16b928e00c534e7d7ca1a16bfecc65aa378 WatchSource:0}: Error finding container 39486421610c872249947fe35d0df16b928e00c534e7d7ca1a16bfecc65aa378: Status 404 returned error can't find the container with id 39486421610c872249947fe35d0df16b928e00c534e7d7ca1a16bfecc65aa378
Sep 07 12:13:44.263301 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:13:44.263258    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-491296/sub-test-model-491296-kserve-86c54877cc-tjsw2" event={"ID":"1d4bd4c0-9ee3-4300-b7dd-5bc764c3adb9","Type":"ContainerStarted","Data":"c45296494070db814e5da63cb500a7795e663d65c564b361a5d841e64f2bbba1"}
Sep 07 12:13:44.263524 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:13:44.263312    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-491296/sub-test-model-491296-kserve-86c54877cc-tjsw2" event={"ID":"1d4bd4c0-9ee3-4300-b7dd-5bc764c3adb9","Type":"ContainerStarted","Data":"39486421610c872249947fe35d0df16b928e00c534e7d7ca1a16bfecc65aa378"}
Sep 07 12:13:44.278367 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:13:44.278322    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-w3-491296/sub-test-model-491296-kserve-86c54877cc-tjsw2" podStartSLOduration=1.278308454 podStartE2EDuration="1.278308454s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 12:13:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 12:13:44.276608948 +0000 UTC m=+1595.795340760" watchObservedRunningTime="2026-09-07 12:13:44.278308454 +0000 UTC m=+1595.797040266"
Sep 07 12:13:46.610942 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:13:46.610907    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-p4l4w"]
Sep 07 12:13:47.980693 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:13:47.980654    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:13:49.431537 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:13:49.431494    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-w3-491296/auth-test-model-491296-kserve-775ddd7b8c-cklcl" podUID="8430a642-c45a-4fa2-80d7-765b61ddcd70" containerName="main" containerID="cri-o://d8fce54fdc0e63733f8fc969f5e0f8b24602c4218de69ffd4ec30e6f15e157e8" gracePeriod=45
Sep 07 12:13:49.672650 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:13:49.672627    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-491296/auth-test-model-491296-kserve-775ddd7b8c-cklcl"
Sep 07 12:13:49.777012 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:13:49.776985    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8430a642-c45a-4fa2-80d7-765b61ddcd70-dshm\") pod \"8430a642-c45a-4fa2-80d7-765b61ddcd70\" (UID: \"8430a642-c45a-4fa2-80d7-765b61ddcd70\") "
Sep 07 12:13:49.777161 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:13:49.777071    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8430a642-c45a-4fa2-80d7-765b61ddcd70-model-cache\") pod \"8430a642-c45a-4fa2-80d7-765b61ddcd70\" (UID: \"8430a642-c45a-4fa2-80d7-765b61ddcd70\") "
Sep 07 12:13:49.777161 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:13:49.777095    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8430a642-c45a-4fa2-80d7-765b61ddcd70-tls-certs\") pod \"8430a642-c45a-4fa2-80d7-765b61ddcd70\" (UID: \"8430a642-c45a-4fa2-80d7-765b61ddcd70\") "
Sep 07 12:13:49.777161 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:13:49.777113    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4mkh8\" (UniqueName: \"kubernetes.io/projected/8430a642-c45a-4fa2-80d7-765b61ddcd70-kube-api-access-4mkh8\") pod \"8430a642-c45a-4fa2-80d7-765b61ddcd70\" (UID: \"8430a642-c45a-4fa2-80d7-765b61ddcd70\") "
Sep 07 12:13:49.777161 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:13:49.777131    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8430a642-c45a-4fa2-80d7-765b61ddcd70-home\") pod \"8430a642-c45a-4fa2-80d7-765b61ddcd70\" (UID: \"8430a642-c45a-4fa2-80d7-765b61ddcd70\") "
Sep 07 12:13:49.777390 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:13:49.777204    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8430a642-c45a-4fa2-80d7-765b61ddcd70-tmp-dir\") pod \"8430a642-c45a-4fa2-80d7-765b61ddcd70\" (UID: \"8430a642-c45a-4fa2-80d7-765b61ddcd70\") "
Sep 07 12:13:49.777475 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:13:49.777384    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8430a642-c45a-4fa2-80d7-765b61ddcd70-model-cache" (OuterVolumeSpecName: "model-cache") pod "8430a642-c45a-4fa2-80d7-765b61ddcd70" (UID: "8430a642-c45a-4fa2-80d7-765b61ddcd70"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 12:13:49.777547 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:13:49.777502    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8430a642-c45a-4fa2-80d7-765b61ddcd70-home" (OuterVolumeSpecName: "home") pod "8430a642-c45a-4fa2-80d7-765b61ddcd70" (UID: "8430a642-c45a-4fa2-80d7-765b61ddcd70"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 12:13:49.777606 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:13:49.777562    2563 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8430a642-c45a-4fa2-80d7-765b61ddcd70-model-cache\") on node \"ip-10-0-135-173.ec2.internal\" DevicePath \"\""
Sep 07 12:13:49.777606 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:13:49.777562    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8430a642-c45a-4fa2-80d7-765b61ddcd70-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "8430a642-c45a-4fa2-80d7-765b61ddcd70" (UID: "8430a642-c45a-4fa2-80d7-765b61ddcd70"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 12:13:49.779329 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:13:49.779289    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8430a642-c45a-4fa2-80d7-765b61ddcd70-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "8430a642-c45a-4fa2-80d7-765b61ddcd70" (UID: "8430a642-c45a-4fa2-80d7-765b61ddcd70"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 07 12:13:49.779329 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:13:49.779289    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8430a642-c45a-4fa2-80d7-765b61ddcd70-dshm" (OuterVolumeSpecName: "dshm") pod "8430a642-c45a-4fa2-80d7-765b61ddcd70" (UID: "8430a642-c45a-4fa2-80d7-765b61ddcd70"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 12:13:49.779492 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:13:49.779359    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8430a642-c45a-4fa2-80d7-765b61ddcd70-kube-api-access-4mkh8" (OuterVolumeSpecName: "kube-api-access-4mkh8") pod "8430a642-c45a-4fa2-80d7-765b61ddcd70" (UID: "8430a642-c45a-4fa2-80d7-765b61ddcd70"). InnerVolumeSpecName "kube-api-access-4mkh8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 12:13:49.878070 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:13:49.878043    2563 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8430a642-c45a-4fa2-80d7-765b61ddcd70-tmp-dir\") on node \"ip-10-0-135-173.ec2.internal\" DevicePath \"\""
Sep 07 12:13:49.878070 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:13:49.878068    2563 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8430a642-c45a-4fa2-80d7-765b61ddcd70-dshm\") on node \"ip-10-0-135-173.ec2.internal\" DevicePath \"\""
Sep 07 12:13:49.878070 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:13:49.878076    2563 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8430a642-c45a-4fa2-80d7-765b61ddcd70-tls-certs\") on node \"ip-10-0-135-173.ec2.internal\" DevicePath \"\""
Sep 07 12:13:49.878070 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:13:49.878085    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4mkh8\" (UniqueName: \"kubernetes.io/projected/8430a642-c45a-4fa2-80d7-765b61ddcd70-kube-api-access-4mkh8\") on node \"ip-10-0-135-173.ec2.internal\" DevicePath \"\""
Sep 07 12:13:49.878312 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:13:49.878094    2563 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8430a642-c45a-4fa2-80d7-765b61ddcd70-home\") on node \"ip-10-0-135-173.ec2.internal\" DevicePath \"\""
Sep 07 12:13:50.295359 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:13:50.295324    2563 generic.go:356] "Generic (PLEG): container finished" podID="8430a642-c45a-4fa2-80d7-765b61ddcd70" containerID="d8fce54fdc0e63733f8fc969f5e0f8b24602c4218de69ffd4ec30e6f15e157e8" exitCode=0
Sep 07 12:13:50.295541 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:13:50.295364    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-491296/auth-test-model-491296-kserve-775ddd7b8c-cklcl"
Sep 07 12:13:50.295541 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:13:50.295408    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-491296/auth-test-model-491296-kserve-775ddd7b8c-cklcl" event={"ID":"8430a642-c45a-4fa2-80d7-765b61ddcd70","Type":"ContainerDied","Data":"d8fce54fdc0e63733f8fc969f5e0f8b24602c4218de69ffd4ec30e6f15e157e8"}
Sep 07 12:13:50.295541 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:13:50.295465    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-491296/auth-test-model-491296-kserve-775ddd7b8c-cklcl" event={"ID":"8430a642-c45a-4fa2-80d7-765b61ddcd70","Type":"ContainerDied","Data":"9e4b20cf8e5d9271dec81e5c6641482d6e2480fcbf89200fb707b17bc8fb45cc"}
Sep 07 12:13:50.295541 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:13:50.295484    2563 scope.go:117] "RemoveContainer" containerID="d8fce54fdc0e63733f8fc969f5e0f8b24602c4218de69ffd4ec30e6f15e157e8"
Sep 07 12:13:50.304410 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:13:50.304392    2563 scope.go:117] "RemoveContainer" containerID="d8fce54fdc0e63733f8fc969f5e0f8b24602c4218de69ffd4ec30e6f15e157e8"
Sep 07 12:13:50.304693 ip-10-0-135-173 kubenswrapper[2563]: E0907 12:13:50.304675    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d8fce54fdc0e63733f8fc969f5e0f8b24602c4218de69ffd4ec30e6f15e157e8\": container with ID starting with d8fce54fdc0e63733f8fc969f5e0f8b24602c4218de69ffd4ec30e6f15e157e8 not found: ID does not exist" containerID="d8fce54fdc0e63733f8fc969f5e0f8b24602c4218de69ffd4ec30e6f15e157e8"
Sep 07 12:13:50.304750 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:13:50.304702    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d8fce54fdc0e63733f8fc969f5e0f8b24602c4218de69ffd4ec30e6f15e157e8"} err="failed to get container status \"d8fce54fdc0e63733f8fc969f5e0f8b24602c4218de69ffd4ec30e6f15e157e8\": rpc error: code = NotFound desc = could not find container \"d8fce54fdc0e63733f8fc969f5e0f8b24602c4218de69ffd4ec30e6f15e157e8\": container with ID starting with d8fce54fdc0e63733f8fc969f5e0f8b24602c4218de69ffd4ec30e6f15e157e8 not found: ID does not exist"
Sep 07 12:13:50.315697 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:13:50.315571    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-491296/auth-test-model-491296-kserve-775ddd7b8c-cklcl"]
Sep 07 12:13:50.317549 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:13:50.317529    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-w3-491296/auth-test-model-491296-kserve-775ddd7b8c-cklcl"]
Sep 07 12:13:50.401937 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:13:50.401899    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-w3-613886/auth-test-model-613886-kserve-595985b959-4222v" podUID="6209c8a2-bb44-41c3-9ca9-5b5858047114" containerName="main" containerID="cri-o://d009beee82f4c824e57c1a66ace1ef0e0f37fa52a3adf047eefec79b042bc334" gracePeriod=45
Sep 07 12:13:50.645770 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:13:50.645746    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-613886/auth-test-model-613886-kserve-595985b959-4222v"
Sep 07 12:13:50.685347 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:13:50.685312    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6209c8a2-bb44-41c3-9ca9-5b5858047114-tls-certs\") pod \"6209c8a2-bb44-41c3-9ca9-5b5858047114\" (UID: \"6209c8a2-bb44-41c3-9ca9-5b5858047114\") "
Sep 07 12:13:50.685498 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:13:50.685373    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6209c8a2-bb44-41c3-9ca9-5b5858047114-dshm\") pod \"6209c8a2-bb44-41c3-9ca9-5b5858047114\" (UID: \"6209c8a2-bb44-41c3-9ca9-5b5858047114\") "
Sep 07 12:13:50.685498 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:13:50.685403    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6209c8a2-bb44-41c3-9ca9-5b5858047114-model-cache\") pod \"6209c8a2-bb44-41c3-9ca9-5b5858047114\" (UID: \"6209c8a2-bb44-41c3-9ca9-5b5858047114\") "
Sep 07 12:13:50.685584 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:13:50.685544    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-klxbc\" (UniqueName: \"kubernetes.io/projected/6209c8a2-bb44-41c3-9ca9-5b5858047114-kube-api-access-klxbc\") pod \"6209c8a2-bb44-41c3-9ca9-5b5858047114\" (UID: \"6209c8a2-bb44-41c3-9ca9-5b5858047114\") "
Sep 07 12:13:50.685632 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:13:50.685590    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6209c8a2-bb44-41c3-9ca9-5b5858047114-home\") pod \"6209c8a2-bb44-41c3-9ca9-5b5858047114\" (UID: \"6209c8a2-bb44-41c3-9ca9-5b5858047114\") "
Sep 07 12:13:50.685685 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:13:50.685631    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6209c8a2-bb44-41c3-9ca9-5b5858047114-tmp-dir\") pod \"6209c8a2-bb44-41c3-9ca9-5b5858047114\" (UID: \"6209c8a2-bb44-41c3-9ca9-5b5858047114\") "
Sep 07 12:13:50.685685 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:13:50.685666    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6209c8a2-bb44-41c3-9ca9-5b5858047114-model-cache" (OuterVolumeSpecName: "model-cache") pod "6209c8a2-bb44-41c3-9ca9-5b5858047114" (UID: "6209c8a2-bb44-41c3-9ca9-5b5858047114"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 12:13:50.685971 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:13:50.685893    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6209c8a2-bb44-41c3-9ca9-5b5858047114-home" (OuterVolumeSpecName: "home") pod "6209c8a2-bb44-41c3-9ca9-5b5858047114" (UID: "6209c8a2-bb44-41c3-9ca9-5b5858047114"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 12:13:50.685971 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:13:50.685918    2563 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6209c8a2-bb44-41c3-9ca9-5b5858047114-model-cache\") on node \"ip-10-0-135-173.ec2.internal\" DevicePath \"\""
Sep 07 12:13:50.685971 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:13:50.685962    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6209c8a2-bb44-41c3-9ca9-5b5858047114-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "6209c8a2-bb44-41c3-9ca9-5b5858047114" (UID: "6209c8a2-bb44-41c3-9ca9-5b5858047114"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 12:13:50.687633 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:13:50.687612    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6209c8a2-bb44-41c3-9ca9-5b5858047114-dshm" (OuterVolumeSpecName: "dshm") pod "6209c8a2-bb44-41c3-9ca9-5b5858047114" (UID: "6209c8a2-bb44-41c3-9ca9-5b5858047114"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 12:13:50.687733 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:13:50.687640    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6209c8a2-bb44-41c3-9ca9-5b5858047114-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "6209c8a2-bb44-41c3-9ca9-5b5858047114" (UID: "6209c8a2-bb44-41c3-9ca9-5b5858047114"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 07 12:13:50.687733 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:13:50.687713    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6209c8a2-bb44-41c3-9ca9-5b5858047114-kube-api-access-klxbc" (OuterVolumeSpecName: "kube-api-access-klxbc") pod "6209c8a2-bb44-41c3-9ca9-5b5858047114" (UID: "6209c8a2-bb44-41c3-9ca9-5b5858047114"). InnerVolumeSpecName "kube-api-access-klxbc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 12:13:50.787361 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:13:50.787320    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-klxbc\" (UniqueName: \"kubernetes.io/projected/6209c8a2-bb44-41c3-9ca9-5b5858047114-kube-api-access-klxbc\") on node \"ip-10-0-135-173.ec2.internal\" DevicePath \"\""
Sep 07 12:13:50.787361 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:13:50.787358    2563 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6209c8a2-bb44-41c3-9ca9-5b5858047114-home\") on node \"ip-10-0-135-173.ec2.internal\" DevicePath \"\""
Sep 07 12:13:50.787361 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:13:50.787368    2563 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6209c8a2-bb44-41c3-9ca9-5b5858047114-tmp-dir\") on node \"ip-10-0-135-173.ec2.internal\" DevicePath \"\""
Sep 07 12:13:50.787361 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:13:50.787377    2563 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6209c8a2-bb44-41c3-9ca9-5b5858047114-tls-certs\") on node \"ip-10-0-135-173.ec2.internal\" DevicePath \"\""
Sep 07 12:13:50.787628 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:13:50.787384    2563 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6209c8a2-bb44-41c3-9ca9-5b5858047114-dshm\") on node \"ip-10-0-135-173.ec2.internal\" DevicePath \"\""
Sep 07 12:13:51.040979 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:13:51.040946    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8430a642-c45a-4fa2-80d7-765b61ddcd70" path="/var/lib/kubelet/pods/8430a642-c45a-4fa2-80d7-765b61ddcd70/volumes"
Sep 07 12:13:51.302788 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:13:51.302702    2563 generic.go:356] "Generic (PLEG): container finished" podID="6209c8a2-bb44-41c3-9ca9-5b5858047114" containerID="d009beee82f4c824e57c1a66ace1ef0e0f37fa52a3adf047eefec79b042bc334" exitCode=0
Sep 07 12:13:51.302788 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:13:51.302746    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-613886/auth-test-model-613886-kserve-595985b959-4222v"
Sep 07 12:13:51.303017 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:13:51.302789    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-613886/auth-test-model-613886-kserve-595985b959-4222v" event={"ID":"6209c8a2-bb44-41c3-9ca9-5b5858047114","Type":"ContainerDied","Data":"d009beee82f4c824e57c1a66ace1ef0e0f37fa52a3adf047eefec79b042bc334"}
Sep 07 12:13:51.303017 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:13:51.302836    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-613886/auth-test-model-613886-kserve-595985b959-4222v" event={"ID":"6209c8a2-bb44-41c3-9ca9-5b5858047114","Type":"ContainerDied","Data":"0bc72c4ccfad42ab7d44157d74e7a21787559b539d370774152d2b5e2f235710"}
Sep 07 12:13:51.303017 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:13:51.302855    2563 scope.go:117] "RemoveContainer" containerID="d009beee82f4c824e57c1a66ace1ef0e0f37fa52a3adf047eefec79b042bc334"
Sep 07 12:13:51.311607 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:13:51.311583    2563 scope.go:117] "RemoveContainer" containerID="d009beee82f4c824e57c1a66ace1ef0e0f37fa52a3adf047eefec79b042bc334"
Sep 07 12:13:51.311861 ip-10-0-135-173 kubenswrapper[2563]: E0907 12:13:51.311844    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d009beee82f4c824e57c1a66ace1ef0e0f37fa52a3adf047eefec79b042bc334\": container with ID starting with d009beee82f4c824e57c1a66ace1ef0e0f37fa52a3adf047eefec79b042bc334 not found: ID does not exist" containerID="d009beee82f4c824e57c1a66ace1ef0e0f37fa52a3adf047eefec79b042bc334"
Sep 07 12:13:51.311910 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:13:51.311867    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d009beee82f4c824e57c1a66ace1ef0e0f37fa52a3adf047eefec79b042bc334"} err="failed to get container status \"d009beee82f4c824e57c1a66ace1ef0e0f37fa52a3adf047eefec79b042bc334\": rpc error: code = NotFound desc = could not find container \"d009beee82f4c824e57c1a66ace1ef0e0f37fa52a3adf047eefec79b042bc334\": container with ID starting with d009beee82f4c824e57c1a66ace1ef0e0f37fa52a3adf047eefec79b042bc334 not found: ID does not exist"
Sep 07 12:13:51.316821 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:13:51.316798    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-613886/auth-test-model-613886-kserve-595985b959-4222v"]
Sep 07 12:13:51.320565 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:13:51.320546    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-w3-613886/auth-test-model-613886-kserve-595985b959-4222v"]
Sep 07 12:13:53.041080 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:13:53.041049    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6209c8a2-bb44-41c3-9ca9-5b5858047114" path="/var/lib/kubelet/pods/6209c8a2-bb44-41c3-9ca9-5b5858047114/volumes"
Sep 07 12:13:53.624832 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:13:53.624797    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-w3-491296/sub-test-model-491296-kserve-86c54877cc-tjsw2"
Sep 07 12:13:53.625006 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:13:53.624848    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-w3-491296/sub-test-model-491296-kserve-86c54877cc-tjsw2"
Sep 07 12:13:53.641462 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:13:53.641441    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-w3-491296/sub-test-model-491296-kserve-86c54877cc-tjsw2"
Sep 07 12:13:54.330851 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:13:54.330818    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-w3-491296/sub-test-model-491296-kserve-86c54877cc-tjsw2"
Sep 07 12:13:56.114673 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:13:56.114628    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:13:56.169001 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:13:56.168967    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:13:58.121678 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:13:58.121642    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:14:01.568131 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:14:01.568049    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:14:07.404345 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:14:07.404313    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:14:07.508638 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:14:07.508602    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:14:07.754786 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:14:07.754756    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:14:08.023079 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:14:08.022996    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-f7f063-openshift-default-766d6644cb-fdxmb"]
Sep 07 12:14:08.023554 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:14:08.023531    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6209c8a2-bb44-41c3-9ca9-5b5858047114" containerName="main"
Sep 07 12:14:08.023554 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:14:08.023553    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="6209c8a2-bb44-41c3-9ca9-5b5858047114" containerName="main"
Sep 07 12:14:08.023741 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:14:08.023605    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8430a642-c45a-4fa2-80d7-765b61ddcd70" containerName="main"
Sep 07 12:14:08.023741 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:14:08.023614    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="8430a642-c45a-4fa2-80d7-765b61ddcd70" containerName="main"
Sep 07 12:14:08.023741 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:14:08.023701    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="8430a642-c45a-4fa2-80d7-765b61ddcd70" containerName="main"
Sep 07 12:14:08.023741 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:14:08.023714    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="4ad3830f-29e1-43e4-b440-127e3c84faa6" containerName="istio-proxy"
Sep 07 12:14:08.023741 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:14:08.023728    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="6209c8a2-bb44-41c3-9ca9-5b5858047114" containerName="main"
Sep 07 12:14:08.028543 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:14:08.028523    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-f7f063-openshift-default-766d6644cb-fdxmb"
Sep 07 12:14:08.030456 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:14:08.030434    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-f7f063-openshift-default-dockercfg-gwhj6\""
Sep 07 12:14:08.035702 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:14:08.035681    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-f7f063-openshift-default-766d6644cb-fdxmb"]
Sep 07 12:14:08.143840 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:14:08.143808    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f12e7fd5-0004-46c0-a944-929644ac9263-workload-socket\") pod \"e2e-api-a-f7f063-openshift-default-766d6644cb-fdxmb\" (UID: \"f12e7fd5-0004-46c0-a944-929644ac9263\") " pod="openshift-ingress/e2e-api-a-f7f063-openshift-default-766d6644cb-fdxmb"
Sep 07 12:14:08.144022 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:14:08.143854    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f12e7fd5-0004-46c0-a944-929644ac9263-workload-certs\") pod \"e2e-api-a-f7f063-openshift-default-766d6644cb-fdxmb\" (UID: \"f12e7fd5-0004-46c0-a944-929644ac9263\") " pod="openshift-ingress/e2e-api-a-f7f063-openshift-default-766d6644cb-fdxmb"
Sep 07 12:14:08.144022 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:14:08.143890    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nh6mv\" (UniqueName: \"kubernetes.io/projected/f12e7fd5-0004-46c0-a944-929644ac9263-kube-api-access-nh6mv\") pod \"e2e-api-a-f7f063-openshift-default-766d6644cb-fdxmb\" (UID: \"f12e7fd5-0004-46c0-a944-929644ac9263\") " pod="openshift-ingress/e2e-api-a-f7f063-openshift-default-766d6644cb-fdxmb"
Sep 07 12:14:08.144022 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:14:08.143915    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f12e7fd5-0004-46c0-a944-929644ac9263-istio-envoy\") pod \"e2e-api-a-f7f063-openshift-default-766d6644cb-fdxmb\" (UID: \"f12e7fd5-0004-46c0-a944-929644ac9263\") " pod="openshift-ingress/e2e-api-a-f7f063-openshift-default-766d6644cb-fdxmb"
Sep 07 12:14:08.144022 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:14:08.143959    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f12e7fd5-0004-46c0-a944-929644ac9263-istiod-ca-cert\") pod \"e2e-api-a-f7f063-openshift-default-766d6644cb-fdxmb\" (UID: \"f12e7fd5-0004-46c0-a944-929644ac9263\") " pod="openshift-ingress/e2e-api-a-f7f063-openshift-default-766d6644cb-fdxmb"
Sep 07 12:14:08.144022 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:14:08.144000    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f12e7fd5-0004-46c0-a944-929644ac9263-istio-token\") pod \"e2e-api-a-f7f063-openshift-default-766d6644cb-fdxmb\" (UID: \"f12e7fd5-0004-46c0-a944-929644ac9263\") " pod="openshift-ingress/e2e-api-a-f7f063-openshift-default-766d6644cb-fdxmb"
Sep 07 12:14:08.144250 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:14:08.144054    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f12e7fd5-0004-46c0-a944-929644ac9263-istio-data\") pod \"e2e-api-a-f7f063-openshift-default-766d6644cb-fdxmb\" (UID: \"f12e7fd5-0004-46c0-a944-929644ac9263\") " pod="openshift-ingress/e2e-api-a-f7f063-openshift-default-766d6644cb-fdxmb"
Sep 07 12:14:08.144250 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:14:08.144098    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f12e7fd5-0004-46c0-a944-929644ac9263-credential-socket\") pod \"e2e-api-a-f7f063-openshift-default-766d6644cb-fdxmb\" (UID: \"f12e7fd5-0004-46c0-a944-929644ac9263\") " pod="openshift-ingress/e2e-api-a-f7f063-openshift-default-766d6644cb-fdxmb"
Sep 07 12:14:08.144250 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:14:08.144127    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f12e7fd5-0004-46c0-a944-929644ac9263-istio-podinfo\") pod \"e2e-api-a-f7f063-openshift-default-766d6644cb-fdxmb\" (UID: \"f12e7fd5-0004-46c0-a944-929644ac9263\") " pod="openshift-ingress/e2e-api-a-f7f063-openshift-default-766d6644cb-fdxmb"
Sep 07 12:14:08.198546 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:14:08.198509    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:14:08.244987 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:14:08.244961    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f12e7fd5-0004-46c0-a944-929644ac9263-istio-podinfo\") pod \"e2e-api-a-f7f063-openshift-default-766d6644cb-fdxmb\" (UID: \"f12e7fd5-0004-46c0-a944-929644ac9263\") " pod="openshift-ingress/e2e-api-a-f7f063-openshift-default-766d6644cb-fdxmb"
Sep 07 12:14:08.245130 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:14:08.245013    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f12e7fd5-0004-46c0-a944-929644ac9263-workload-socket\") pod \"e2e-api-a-f7f063-openshift-default-766d6644cb-fdxmb\" (UID: \"f12e7fd5-0004-46c0-a944-929644ac9263\") " pod="openshift-ingress/e2e-api-a-f7f063-openshift-default-766d6644cb-fdxmb"
Sep 07 12:14:08.245130 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:14:08.245034    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f12e7fd5-0004-46c0-a944-929644ac9263-workload-certs\") pod \"e2e-api-a-f7f063-openshift-default-766d6644cb-fdxmb\" (UID: \"f12e7fd5-0004-46c0-a944-929644ac9263\") " pod="openshift-ingress/e2e-api-a-f7f063-openshift-default-766d6644cb-fdxmb"
Sep 07 12:14:08.245130 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:14:08.245058    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nh6mv\" (UniqueName: \"kubernetes.io/projected/f12e7fd5-0004-46c0-a944-929644ac9263-kube-api-access-nh6mv\") pod \"e2e-api-a-f7f063-openshift-default-766d6644cb-fdxmb\" (UID: \"f12e7fd5-0004-46c0-a944-929644ac9263\") " pod="openshift-ingress/e2e-api-a-f7f063-openshift-default-766d6644cb-fdxmb"
Sep 07 12:14:08.245288 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:14:08.245226    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f12e7fd5-0004-46c0-a944-929644ac9263-istio-envoy\") pod \"e2e-api-a-f7f063-openshift-default-766d6644cb-fdxmb\" (UID: \"f12e7fd5-0004-46c0-a944-929644ac9263\") " pod="openshift-ingress/e2e-api-a-f7f063-openshift-default-766d6644cb-fdxmb"
Sep 07 12:14:08.245347 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:14:08.245307    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f12e7fd5-0004-46c0-a944-929644ac9263-istiod-ca-cert\") pod \"e2e-api-a-f7f063-openshift-default-766d6644cb-fdxmb\" (UID: \"f12e7fd5-0004-46c0-a944-929644ac9263\") " pod="openshift-ingress/e2e-api-a-f7f063-openshift-default-766d6644cb-fdxmb"
Sep 07 12:14:08.245400 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:14:08.245348    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f12e7fd5-0004-46c0-a944-929644ac9263-istio-token\") pod \"e2e-api-a-f7f063-openshift-default-766d6644cb-fdxmb\" (UID: \"f12e7fd5-0004-46c0-a944-929644ac9263\") " pod="openshift-ingress/e2e-api-a-f7f063-openshift-default-766d6644cb-fdxmb"
Sep 07 12:14:08.245400 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:14:08.245394    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f12e7fd5-0004-46c0-a944-929644ac9263-istio-data\") pod \"e2e-api-a-f7f063-openshift-default-766d6644cb-fdxmb\" (UID: \"f12e7fd5-0004-46c0-a944-929644ac9263\") " pod="openshift-ingress/e2e-api-a-f7f063-openshift-default-766d6644cb-fdxmb"
Sep 07 12:14:08.245534 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:14:08.245403    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f12e7fd5-0004-46c0-a944-929644ac9263-workload-socket\") pod \"e2e-api-a-f7f063-openshift-default-766d6644cb-fdxmb\" (UID: \"f12e7fd5-0004-46c0-a944-929644ac9263\") " pod="openshift-ingress/e2e-api-a-f7f063-openshift-default-766d6644cb-fdxmb"
Sep 07 12:14:08.245534 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:14:08.245473    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f12e7fd5-0004-46c0-a944-929644ac9263-credential-socket\") pod \"e2e-api-a-f7f063-openshift-default-766d6644cb-fdxmb\" (UID: \"f12e7fd5-0004-46c0-a944-929644ac9263\") " pod="openshift-ingress/e2e-api-a-f7f063-openshift-default-766d6644cb-fdxmb"
Sep 07 12:14:08.245618 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:14:08.245454    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f12e7fd5-0004-46c0-a944-929644ac9263-workload-certs\") pod \"e2e-api-a-f7f063-openshift-default-766d6644cb-fdxmb\" (UID: \"f12e7fd5-0004-46c0-a944-929644ac9263\") " pod="openshift-ingress/e2e-api-a-f7f063-openshift-default-766d6644cb-fdxmb"
Sep 07 12:14:08.245768 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:14:08.245738    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f12e7fd5-0004-46c0-a944-929644ac9263-istio-data\") pod \"e2e-api-a-f7f063-openshift-default-766d6644cb-fdxmb\" (UID: \"f12e7fd5-0004-46c0-a944-929644ac9263\") " pod="openshift-ingress/e2e-api-a-f7f063-openshift-default-766d6644cb-fdxmb"
Sep 07 12:14:08.245843 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:14:08.245819    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f12e7fd5-0004-46c0-a944-929644ac9263-credential-socket\") pod \"e2e-api-a-f7f063-openshift-default-766d6644cb-fdxmb\" (UID: \"f12e7fd5-0004-46c0-a944-929644ac9263\") " pod="openshift-ingress/e2e-api-a-f7f063-openshift-default-766d6644cb-fdxmb"
Sep 07 12:14:08.245986 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:14:08.245967    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f12e7fd5-0004-46c0-a944-929644ac9263-istiod-ca-cert\") pod \"e2e-api-a-f7f063-openshift-default-766d6644cb-fdxmb\" (UID: \"f12e7fd5-0004-46c0-a944-929644ac9263\") " pod="openshift-ingress/e2e-api-a-f7f063-openshift-default-766d6644cb-fdxmb"
Sep 07 12:14:08.247493 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:14:08.247471    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f12e7fd5-0004-46c0-a944-929644ac9263-istio-envoy\") pod \"e2e-api-a-f7f063-openshift-default-766d6644cb-fdxmb\" (UID: \"f12e7fd5-0004-46c0-a944-929644ac9263\") " pod="openshift-ingress/e2e-api-a-f7f063-openshift-default-766d6644cb-fdxmb"
Sep 07 12:14:08.247793 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:14:08.247774    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f12e7fd5-0004-46c0-a944-929644ac9263-istio-podinfo\") pod \"e2e-api-a-f7f063-openshift-default-766d6644cb-fdxmb\" (UID: \"f12e7fd5-0004-46c0-a944-929644ac9263\") " pod="openshift-ingress/e2e-api-a-f7f063-openshift-default-766d6644cb-fdxmb"
Sep 07 12:14:08.251812 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:14:08.251788    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f12e7fd5-0004-46c0-a944-929644ac9263-istio-token\") pod \"e2e-api-a-f7f063-openshift-default-766d6644cb-fdxmb\" (UID: \"f12e7fd5-0004-46c0-a944-929644ac9263\") " pod="openshift-ingress/e2e-api-a-f7f063-openshift-default-766d6644cb-fdxmb"
Sep 07 12:14:08.252014 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:14:08.251993    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nh6mv\" (UniqueName: \"kubernetes.io/projected/f12e7fd5-0004-46c0-a944-929644ac9263-kube-api-access-nh6mv\") pod \"e2e-api-a-f7f063-openshift-default-766d6644cb-fdxmb\" (UID: \"f12e7fd5-0004-46c0-a944-929644ac9263\") " pod="openshift-ingress/e2e-api-a-f7f063-openshift-default-766d6644cb-fdxmb"
Sep 07 12:14:08.257770 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:14:08.257742    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:14:08.341155 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:14:08.341093    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-f7f063-openshift-default-766d6644cb-fdxmb"
Sep 07 12:14:08.493439 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:14:08.493390    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-w3-613886/sub-test-model-613886-kserve-5467c658b7-nnw9k"]
Sep 07 12:14:08.497900 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:14:08.497884    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-613886/sub-test-model-613886-kserve-5467c658b7-nnw9k"
Sep 07 12:14:08.499807 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:14:08.499776    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-613886\"/\"kube-root-ca.crt\""
Sep 07 12:14:08.499939 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:14:08.499806    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-613886\"/\"sub-test-model-613886-kserve-self-signed-certs\""
Sep 07 12:14:08.499939 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:14:08.499827    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-613886\"/\"default-dockercfg-9r7cs\""
Sep 07 12:14:08.499939 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:14:08.499811    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-613886\"/\"openshift-service-ca.crt\""
Sep 07 12:14:08.503992 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:14:08.503967    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-613886/sub-test-model-613886-kserve-5467c658b7-nnw9k"]
Sep 07 12:14:08.633840 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:14:08.633753    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:14:08.648813 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:14:08.648788    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ca4a17f5-413b-4f28-9230-c10dae69ab0f-model-cache\") pod \"sub-test-model-613886-kserve-5467c658b7-nnw9k\" (UID: \"ca4a17f5-413b-4f28-9230-c10dae69ab0f\") " pod="ai-tenant-e2e-shared-b-w3-613886/sub-test-model-613886-kserve-5467c658b7-nnw9k"
Sep 07 12:14:08.648922 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:14:08.648828    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ca4a17f5-413b-4f28-9230-c10dae69ab0f-dshm\") pod \"sub-test-model-613886-kserve-5467c658b7-nnw9k\" (UID: \"ca4a17f5-413b-4f28-9230-c10dae69ab0f\") " pod="ai-tenant-e2e-shared-b-w3-613886/sub-test-model-613886-kserve-5467c658b7-nnw9k"
Sep 07 12:14:08.648922 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:14:08.648849    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ca4a17f5-413b-4f28-9230-c10dae69ab0f-home\") pod \"sub-test-model-613886-kserve-5467c658b7-nnw9k\" (UID: \"ca4a17f5-413b-4f28-9230-c10dae69ab0f\") " pod="ai-tenant-e2e-shared-b-w3-613886/sub-test-model-613886-kserve-5467c658b7-nnw9k"
Sep 07 12:14:08.649003 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:14:08.648976    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nbz5p\" (UniqueName: \"kubernetes.io/projected/ca4a17f5-413b-4f28-9230-c10dae69ab0f-kube-api-access-nbz5p\") pod \"sub-test-model-613886-kserve-5467c658b7-nnw9k\" (UID: \"ca4a17f5-413b-4f28-9230-c10dae69ab0f\") " pod="ai-tenant-e2e-shared-b-w3-613886/sub-test-model-613886-kserve-5467c658b7-nnw9k"
Sep 07 12:14:08.649058 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:14:08.649042    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ca4a17f5-413b-4f28-9230-c10dae69ab0f-tls-certs\") pod \"sub-test-model-613886-kserve-5467c658b7-nnw9k\" (UID: \"ca4a17f5-413b-4f28-9230-c10dae69ab0f\") " pod="ai-tenant-e2e-shared-b-w3-613886/sub-test-model-613886-kserve-5467c658b7-nnw9k"
Sep 07 12:14:08.649096 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:14:08.649067    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ca4a17f5-413b-4f28-9230-c10dae69ab0f-tmp-dir\") pod \"sub-test-model-613886-kserve-5467c658b7-nnw9k\" (UID: \"ca4a17f5-413b-4f28-9230-c10dae69ab0f\") " pod="ai-tenant-e2e-shared-b-w3-613886/sub-test-model-613886-kserve-5467c658b7-nnw9k"
Sep 07 12:14:08.657788 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:14:08.657770    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-f7f063-openshift-default-766d6644cb-fdxmb"]
Sep 07 12:14:08.659771 ip-10-0-135-173 kubenswrapper[2563]: W0907 12:14:08.659741    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf12e7fd5_0004_46c0_a944_929644ac9263.slice/crio-426619fc2e4e9b3c6fc4627dd1c769333ad956350614f0eba6dd56554afaa514 WatchSource:0}: Error finding container 426619fc2e4e9b3c6fc4627dd1c769333ad956350614f0eba6dd56554afaa514: Status 404 returned error can't find the container with id 426619fc2e4e9b3c6fc4627dd1c769333ad956350614f0eba6dd56554afaa514
Sep 07 12:14:08.662043 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:14:08.662003    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 07 12:14:08.662126 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:14:08.662093    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 07 12:14:08.662168 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:14:08.662139    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 07 12:14:08.749727 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:14:08.749698    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ca4a17f5-413b-4f28-9230-c10dae69ab0f-tls-certs\") pod \"sub-test-model-613886-kserve-5467c658b7-nnw9k\" (UID: \"ca4a17f5-413b-4f28-9230-c10dae69ab0f\") " pod="ai-tenant-e2e-shared-b-w3-613886/sub-test-model-613886-kserve-5467c658b7-nnw9k"
Sep 07 12:14:08.749727 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:14:08.749730    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ca4a17f5-413b-4f28-9230-c10dae69ab0f-tmp-dir\") pod \"sub-test-model-613886-kserve-5467c658b7-nnw9k\" (UID: \"ca4a17f5-413b-4f28-9230-c10dae69ab0f\") " pod="ai-tenant-e2e-shared-b-w3-613886/sub-test-model-613886-kserve-5467c658b7-nnw9k"
Sep 07 12:14:08.749955 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:14:08.749832    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ca4a17f5-413b-4f28-9230-c10dae69ab0f-model-cache\") pod \"sub-test-model-613886-kserve-5467c658b7-nnw9k\" (UID: \"ca4a17f5-413b-4f28-9230-c10dae69ab0f\") " pod="ai-tenant-e2e-shared-b-w3-613886/sub-test-model-613886-kserve-5467c658b7-nnw9k"
Sep 07 12:14:08.749955 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:14:08.749865    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ca4a17f5-413b-4f28-9230-c10dae69ab0f-dshm\") pod \"sub-test-model-613886-kserve-5467c658b7-nnw9k\" (UID: \"ca4a17f5-413b-4f28-9230-c10dae69ab0f\") " pod="ai-tenant-e2e-shared-b-w3-613886/sub-test-model-613886-kserve-5467c658b7-nnw9k"
Sep 07 12:14:08.749955 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:14:08.749884    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ca4a17f5-413b-4f28-9230-c10dae69ab0f-home\") pod \"sub-test-model-613886-kserve-5467c658b7-nnw9k\" (UID: \"ca4a17f5-413b-4f28-9230-c10dae69ab0f\") " pod="ai-tenant-e2e-shared-b-w3-613886/sub-test-model-613886-kserve-5467c658b7-nnw9k"
Sep 07 12:14:08.750122 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:14:08.749982    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nbz5p\" (UniqueName: \"kubernetes.io/projected/ca4a17f5-413b-4f28-9230-c10dae69ab0f-kube-api-access-nbz5p\") pod \"sub-test-model-613886-kserve-5467c658b7-nnw9k\" (UID: \"ca4a17f5-413b-4f28-9230-c10dae69ab0f\") " pod="ai-tenant-e2e-shared-b-w3-613886/sub-test-model-613886-kserve-5467c658b7-nnw9k"
Sep 07 12:14:08.750229 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:14:08.750204    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ca4a17f5-413b-4f28-9230-c10dae69ab0f-tmp-dir\") pod \"sub-test-model-613886-kserve-5467c658b7-nnw9k\" (UID: \"ca4a17f5-413b-4f28-9230-c10dae69ab0f\") " pod="ai-tenant-e2e-shared-b-w3-613886/sub-test-model-613886-kserve-5467c658b7-nnw9k"
Sep 07 12:14:08.750293 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:14:08.750250    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ca4a17f5-413b-4f28-9230-c10dae69ab0f-home\") pod \"sub-test-model-613886-kserve-5467c658b7-nnw9k\" (UID: \"ca4a17f5-413b-4f28-9230-c10dae69ab0f\") " pod="ai-tenant-e2e-shared-b-w3-613886/sub-test-model-613886-kserve-5467c658b7-nnw9k"
Sep 07 12:14:08.750411 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:14:08.750391    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ca4a17f5-413b-4f28-9230-c10dae69ab0f-model-cache\") pod \"sub-test-model-613886-kserve-5467c658b7-nnw9k\" (UID: \"ca4a17f5-413b-4f28-9230-c10dae69ab0f\") " pod="ai-tenant-e2e-shared-b-w3-613886/sub-test-model-613886-kserve-5467c658b7-nnw9k"
Sep 07 12:14:08.752353 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:14:08.752324    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ca4a17f5-413b-4f28-9230-c10dae69ab0f-dshm\") pod \"sub-test-model-613886-kserve-5467c658b7-nnw9k\" (UID: \"ca4a17f5-413b-4f28-9230-c10dae69ab0f\") " pod="ai-tenant-e2e-shared-b-w3-613886/sub-test-model-613886-kserve-5467c658b7-nnw9k"
Sep 07 12:14:08.752533 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:14:08.752468    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ca4a17f5-413b-4f28-9230-c10dae69ab0f-tls-certs\") pod \"sub-test-model-613886-kserve-5467c658b7-nnw9k\" (UID: \"ca4a17f5-413b-4f28-9230-c10dae69ab0f\") " pod="ai-tenant-e2e-shared-b-w3-613886/sub-test-model-613886-kserve-5467c658b7-nnw9k"
Sep 07 12:14:08.758768 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:14:08.758746    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nbz5p\" (UniqueName: \"kubernetes.io/projected/ca4a17f5-413b-4f28-9230-c10dae69ab0f-kube-api-access-nbz5p\") pod \"sub-test-model-613886-kserve-5467c658b7-nnw9k\" (UID: \"ca4a17f5-413b-4f28-9230-c10dae69ab0f\") " pod="ai-tenant-e2e-shared-b-w3-613886/sub-test-model-613886-kserve-5467c658b7-nnw9k"
Sep 07 12:14:08.809098 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:14:08.809066    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-613886/sub-test-model-613886-kserve-5467c658b7-nnw9k"
Sep 07 12:14:09.080677 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:14:09.080645    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-613886/sub-test-model-613886-kserve-5467c658b7-nnw9k"]
Sep 07 12:14:09.083544 ip-10-0-135-173 kubenswrapper[2563]: W0907 12:14:09.083513    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podca4a17f5_413b_4f28_9230_c10dae69ab0f.slice/crio-fd7acb9ce20525463977117dbaf2595aa5c3c0f9c1531fef5d404512850d3ebc WatchSource:0}: Error finding container fd7acb9ce20525463977117dbaf2595aa5c3c0f9c1531fef5d404512850d3ebc: Status 404 returned error can't find the container with id fd7acb9ce20525463977117dbaf2595aa5c3c0f9c1531fef5d404512850d3ebc
Sep 07 12:14:09.405753 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:14:09.405661    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-f7f063-openshift-default-766d6644cb-fdxmb" event={"ID":"f12e7fd5-0004-46c0-a944-929644ac9263","Type":"ContainerStarted","Data":"874710699c087b1616f29ffd9b09f2bdef7c9e3ac2b6cdfb1c37c5321f7e1ff4"}
Sep 07 12:14:09.405908 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:14:09.405757    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-f7f063-openshift-default-766d6644cb-fdxmb" event={"ID":"f12e7fd5-0004-46c0-a944-929644ac9263","Type":"ContainerStarted","Data":"426619fc2e4e9b3c6fc4627dd1c769333ad956350614f0eba6dd56554afaa514"}
Sep 07 12:14:09.407242 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:14:09.407216    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-613886/sub-test-model-613886-kserve-5467c658b7-nnw9k" event={"ID":"ca4a17f5-413b-4f28-9230-c10dae69ab0f","Type":"ContainerStarted","Data":"55a42a32d7e94f9714c1c72eb551cc890c01258d53ad3b7b623ef4cebd536fe7"}
Sep 07 12:14:09.407357 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:14:09.407248    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-613886/sub-test-model-613886-kserve-5467c658b7-nnw9k" event={"ID":"ca4a17f5-413b-4f28-9230-c10dae69ab0f","Type":"ContainerStarted","Data":"fd7acb9ce20525463977117dbaf2595aa5c3c0f9c1531fef5d404512850d3ebc"}
Sep 07 12:14:09.423731 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:14:09.423691    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-f7f063-openshift-default-766d6644cb-fdxmb" podStartSLOduration=1.423677073 podStartE2EDuration="1.423677073s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 12:14:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 12:14:09.422222764 +0000 UTC m=+1620.940954577" watchObservedRunningTime="2026-09-07 12:14:09.423677073 +0000 UTC m=+1620.942408886"
Sep 07 12:14:09.437490 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:14:09.437453    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-w3-613886/sub-test-model-613886-kserve-5467c658b7-nnw9k" podStartSLOduration=1.437442473 podStartE2EDuration="1.437442473s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 12:14:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 12:14:09.43603604 +0000 UTC m=+1620.954767852" watchObservedRunningTime="2026-09-07 12:14:09.437442473 +0000 UTC m=+1620.956174283"
Sep 07 12:14:10.341228 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:14:10.341198    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-f7f063-openshift-default-766d6644cb-fdxmb"
Sep 07 12:14:10.345927 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:14:10.345903    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-f7f063-openshift-default-766d6644cb-fdxmb"
Sep 07 12:14:10.412290 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:14:10.412261    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-f7f063-openshift-default-766d6644cb-fdxmb"
Sep 07 12:14:10.413101 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:14:10.413082    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-f7f063-openshift-default-766d6644cb-fdxmb"
Sep 07 12:14:11.307235 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:14:11.307197    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:14:11.406119 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:14:11.406083    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:14:12.476229 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:14:12.476198    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:14:12.574084 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:14:12.574051    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:14:12.674288 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:14:12.674252    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:14:12.944109 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:14:12.944023    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:14:13.036577 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:14:13.036537    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:14:15.101242 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:14:15.101203    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-p4l4w"]
Sep 07 12:14:18.809393 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:14:18.809361    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-w3-613886/sub-test-model-613886-kserve-5467c658b7-nnw9k"
Sep 07 12:14:18.809853 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:14:18.809409    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-w3-613886/sub-test-model-613886-kserve-5467c658b7-nnw9k"
Sep 07 12:14:18.821726 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:14:18.821697    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-w3-613886/sub-test-model-613886-kserve-5467c658b7-nnw9k"
Sep 07 12:14:19.471556 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:14:19.471528    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-w3-613886/sub-test-model-613886-kserve-5467c658b7-nnw9k"
Sep 07 12:14:28.254095 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:14:28.254049    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:14:28.802813 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:14:28.802774    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-p4l4w"]
Sep 07 12:14:31.520880 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:14:31.520850    2563 generic.go:356] "Generic (PLEG): container finished" podID="f12e7fd5-0004-46c0-a944-929644ac9263" containerID="874710699c087b1616f29ffd9b09f2bdef7c9e3ac2b6cdfb1c37c5321f7e1ff4" exitCode=0
Sep 07 12:14:31.521240 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:14:31.520924    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-f7f063-openshift-default-766d6644cb-fdxmb" event={"ID":"f12e7fd5-0004-46c0-a944-929644ac9263","Type":"ContainerDied","Data":"874710699c087b1616f29ffd9b09f2bdef7c9e3ac2b6cdfb1c37c5321f7e1ff4"}
Sep 07 12:14:31.521332 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:14:31.521318    2563 scope.go:117] "RemoveContainer" containerID="874710699c087b1616f29ffd9b09f2bdef7c9e3ac2b6cdfb1c37c5321f7e1ff4"
Sep 07 12:14:31.522646 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:14:31.522621    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 07 12:14:31.522724 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:14:31.522700    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 07 12:14:31.522766 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:14:31.522748    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 07 12:14:32.528116 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:14:32.528083    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-f7f063-openshift-default-766d6644cb-fdxmb" event={"ID":"f12e7fd5-0004-46c0-a944-929644ac9263","Type":"ContainerStarted","Data":"b90362221b21765ac92610531b145ec53a1740836b2f027502ae22e4363f1f28"}
Sep 07 12:14:33.341215 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:14:33.341178    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-f7f063-openshift-default-766d6644cb-fdxmb"
Sep 07 12:14:33.341494 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:14:33.341468    2563 patch_prober.go:28] interesting pod/e2e-api-a-f7f063-openshift-default-766d6644cb-fdxmb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" start-of-body=
Sep 07 12:14:33.341577 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:14:33.341521    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-f7f063-openshift-default-766d6644cb-fdxmb" podUID="f12e7fd5-0004-46c0-a944-929644ac9263" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused"
Sep 07 12:14:33.440224 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:14:33.440188    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:14:34.342002 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:14:34.341967    2563 patch_prober.go:28] interesting pod/e2e-api-a-f7f063-openshift-default-766d6644cb-fdxmb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" start-of-body=
Sep 07 12:14:34.342380 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:14:34.342020    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-f7f063-openshift-default-766d6644cb-fdxmb" podUID="f12e7fd5-0004-46c0-a944-929644ac9263" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused"
Sep 07 12:14:35.341860 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:14:35.341824    2563 patch_prober.go:28] interesting pod/e2e-api-a-f7f063-openshift-default-766d6644cb-fdxmb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" start-of-body=
Sep 07 12:14:35.342032 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:14:35.341880    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-f7f063-openshift-default-766d6644cb-fdxmb" podUID="f12e7fd5-0004-46c0-a944-929644ac9263" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused"
Sep 07 12:14:36.264537 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:14:36.264493    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:14:36.342057 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:14:36.342024    2563 patch_prober.go:28] interesting pod/e2e-api-a-f7f063-openshift-default-766d6644cb-fdxmb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" start-of-body=
Sep 07 12:14:36.342443 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:14:36.342075    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-f7f063-openshift-default-766d6644cb-fdxmb" podUID="f12e7fd5-0004-46c0-a944-929644ac9263" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused"
Sep 07 12:14:36.781803 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:14:36.781762    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:14:36.982438 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:14:36.981971    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-9e422b-openshift-default-6bd67bb6-dqsxv"]
Sep 07 12:14:36.985555 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:14:36.985523    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-9e422b-openshift-default-6bd67bb6-dqsxv"
Sep 07 12:14:36.988144 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:14:36.988120    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-9e422b-openshift-default-dockercfg-fjmnw\""
Sep 07 12:14:36.992274 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:14:36.992248    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-9e422b-openshift-default-6bd67bb6-dqsxv"]
Sep 07 12:14:37.002992 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:14:37.002946    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:14:37.014996 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:14:37.014968    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0ac41394-bb87-4c27-9ab9-6b0d7988cf3a-istio-envoy\") pod \"e2e-api-b-9e422b-openshift-default-6bd67bb6-dqsxv\" (UID: \"0ac41394-bb87-4c27-9ab9-6b0d7988cf3a\") " pod="openshift-ingress/e2e-api-b-9e422b-openshift-default-6bd67bb6-dqsxv"
Sep 07 12:14:37.015131 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:14:37.015025    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0ac41394-bb87-4c27-9ab9-6b0d7988cf3a-istio-data\") pod \"e2e-api-b-9e422b-openshift-default-6bd67bb6-dqsxv\" (UID: \"0ac41394-bb87-4c27-9ab9-6b0d7988cf3a\") " pod="openshift-ingress/e2e-api-b-9e422b-openshift-default-6bd67bb6-dqsxv"
Sep 07 12:14:37.015201 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:14:37.015134    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0ac41394-bb87-4c27-9ab9-6b0d7988cf3a-istio-podinfo\") pod \"e2e-api-b-9e422b-openshift-default-6bd67bb6-dqsxv\" (UID: \"0ac41394-bb87-4c27-9ab9-6b0d7988cf3a\") " pod="openshift-ingress/e2e-api-b-9e422b-openshift-default-6bd67bb6-dqsxv"
Sep 07 12:14:37.015201 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:14:37.015173    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tftqn\" (UniqueName: \"kubernetes.io/projected/0ac41394-bb87-4c27-9ab9-6b0d7988cf3a-kube-api-access-tftqn\") pod \"e2e-api-b-9e422b-openshift-default-6bd67bb6-dqsxv\" (UID: \"0ac41394-bb87-4c27-9ab9-6b0d7988cf3a\") " pod="openshift-ingress/e2e-api-b-9e422b-openshift-default-6bd67bb6-dqsxv"
Sep 07 12:14:37.015290 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:14:37.015244    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0ac41394-bb87-4c27-9ab9-6b0d7988cf3a-istio-token\") pod \"e2e-api-b-9e422b-openshift-default-6bd67bb6-dqsxv\" (UID: \"0ac41394-bb87-4c27-9ab9-6b0d7988cf3a\") " pod="openshift-ingress/e2e-api-b-9e422b-openshift-default-6bd67bb6-dqsxv"
Sep 07 12:14:37.015328 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:14:37.015291    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0ac41394-bb87-4c27-9ab9-6b0d7988cf3a-workload-certs\") pod \"e2e-api-b-9e422b-openshift-default-6bd67bb6-dqsxv\" (UID: \"0ac41394-bb87-4c27-9ab9-6b0d7988cf3a\") " pod="openshift-ingress/e2e-api-b-9e422b-openshift-default-6bd67bb6-dqsxv"
Sep 07 12:14:37.015368 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:14:37.015332    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0ac41394-bb87-4c27-9ab9-6b0d7988cf3a-workload-socket\") pod \"e2e-api-b-9e422b-openshift-default-6bd67bb6-dqsxv\" (UID: \"0ac41394-bb87-4c27-9ab9-6b0d7988cf3a\") " pod="openshift-ingress/e2e-api-b-9e422b-openshift-default-6bd67bb6-dqsxv"
Sep 07 12:14:37.015412 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:14:37.015382    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0ac41394-bb87-4c27-9ab9-6b0d7988cf3a-credential-socket\") pod \"e2e-api-b-9e422b-openshift-default-6bd67bb6-dqsxv\" (UID: \"0ac41394-bb87-4c27-9ab9-6b0d7988cf3a\") " pod="openshift-ingress/e2e-api-b-9e422b-openshift-default-6bd67bb6-dqsxv"
Sep 07 12:14:37.015412 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:14:37.015409    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0ac41394-bb87-4c27-9ab9-6b0d7988cf3a-istiod-ca-cert\") pod \"e2e-api-b-9e422b-openshift-default-6bd67bb6-dqsxv\" (UID: \"0ac41394-bb87-4c27-9ab9-6b0d7988cf3a\") " pod="openshift-ingress/e2e-api-b-9e422b-openshift-default-6bd67bb6-dqsxv"
Sep 07 12:14:37.116626 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:14:37.116539    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0ac41394-bb87-4c27-9ab9-6b0d7988cf3a-istio-podinfo\") pod \"e2e-api-b-9e422b-openshift-default-6bd67bb6-dqsxv\" (UID: \"0ac41394-bb87-4c27-9ab9-6b0d7988cf3a\") " pod="openshift-ingress/e2e-api-b-9e422b-openshift-default-6bd67bb6-dqsxv"
Sep 07 12:14:37.116626 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:14:37.116577    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tftqn\" (UniqueName: \"kubernetes.io/projected/0ac41394-bb87-4c27-9ab9-6b0d7988cf3a-kube-api-access-tftqn\") pod \"e2e-api-b-9e422b-openshift-default-6bd67bb6-dqsxv\" (UID: \"0ac41394-bb87-4c27-9ab9-6b0d7988cf3a\") " pod="openshift-ingress/e2e-api-b-9e422b-openshift-default-6bd67bb6-dqsxv"
Sep 07 12:14:37.116626 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:14:37.116619    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0ac41394-bb87-4c27-9ab9-6b0d7988cf3a-istio-token\") pod \"e2e-api-b-9e422b-openshift-default-6bd67bb6-dqsxv\" (UID: \"0ac41394-bb87-4c27-9ab9-6b0d7988cf3a\") " pod="openshift-ingress/e2e-api-b-9e422b-openshift-default-6bd67bb6-dqsxv"
Sep 07 12:14:37.116886 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:14:37.116649    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0ac41394-bb87-4c27-9ab9-6b0d7988cf3a-workload-certs\") pod \"e2e-api-b-9e422b-openshift-default-6bd67bb6-dqsxv\" (UID: \"0ac41394-bb87-4c27-9ab9-6b0d7988cf3a\") " pod="openshift-ingress/e2e-api-b-9e422b-openshift-default-6bd67bb6-dqsxv"
Sep 07 12:14:37.116886 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:14:37.116684    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0ac41394-bb87-4c27-9ab9-6b0d7988cf3a-workload-socket\") pod \"e2e-api-b-9e422b-openshift-default-6bd67bb6-dqsxv\" (UID: \"0ac41394-bb87-4c27-9ab9-6b0d7988cf3a\") " pod="openshift-ingress/e2e-api-b-9e422b-openshift-default-6bd67bb6-dqsxv"
Sep 07 12:14:37.116886 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:14:37.116740    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0ac41394-bb87-4c27-9ab9-6b0d7988cf3a-credential-socket\") pod \"e2e-api-b-9e422b-openshift-default-6bd67bb6-dqsxv\" (UID: \"0ac41394-bb87-4c27-9ab9-6b0d7988cf3a\") " pod="openshift-ingress/e2e-api-b-9e422b-openshift-default-6bd67bb6-dqsxv"
Sep 07 12:14:37.116886 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:14:37.116765    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0ac41394-bb87-4c27-9ab9-6b0d7988cf3a-istiod-ca-cert\") pod \"e2e-api-b-9e422b-openshift-default-6bd67bb6-dqsxv\" (UID: \"0ac41394-bb87-4c27-9ab9-6b0d7988cf3a\") " pod="openshift-ingress/e2e-api-b-9e422b-openshift-default-6bd67bb6-dqsxv"
Sep 07 12:14:37.116886 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:14:37.116808    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0ac41394-bb87-4c27-9ab9-6b0d7988cf3a-istio-envoy\") pod \"e2e-api-b-9e422b-openshift-default-6bd67bb6-dqsxv\" (UID: \"0ac41394-bb87-4c27-9ab9-6b0d7988cf3a\") " pod="openshift-ingress/e2e-api-b-9e422b-openshift-default-6bd67bb6-dqsxv"
Sep 07 12:14:37.116886 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:14:37.116851    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0ac41394-bb87-4c27-9ab9-6b0d7988cf3a-istio-data\") pod \"e2e-api-b-9e422b-openshift-default-6bd67bb6-dqsxv\" (UID: \"0ac41394-bb87-4c27-9ab9-6b0d7988cf3a\") " pod="openshift-ingress/e2e-api-b-9e422b-openshift-default-6bd67bb6-dqsxv"
Sep 07 12:14:37.117233 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:14:37.117163    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0ac41394-bb87-4c27-9ab9-6b0d7988cf3a-workload-certs\") pod \"e2e-api-b-9e422b-openshift-default-6bd67bb6-dqsxv\" (UID: \"0ac41394-bb87-4c27-9ab9-6b0d7988cf3a\") " pod="openshift-ingress/e2e-api-b-9e422b-openshift-default-6bd67bb6-dqsxv"
Sep 07 12:14:37.117233 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:14:37.117180    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0ac41394-bb87-4c27-9ab9-6b0d7988cf3a-workload-socket\") pod \"e2e-api-b-9e422b-openshift-default-6bd67bb6-dqsxv\" (UID: \"0ac41394-bb87-4c27-9ab9-6b0d7988cf3a\") " pod="openshift-ingress/e2e-api-b-9e422b-openshift-default-6bd67bb6-dqsxv"
Sep 07 12:14:37.117347 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:14:37.117265    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0ac41394-bb87-4c27-9ab9-6b0d7988cf3a-istio-data\") pod \"e2e-api-b-9e422b-openshift-default-6bd67bb6-dqsxv\" (UID: \"0ac41394-bb87-4c27-9ab9-6b0d7988cf3a\") " pod="openshift-ingress/e2e-api-b-9e422b-openshift-default-6bd67bb6-dqsxv"
Sep 07 12:14:37.117525 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:14:37.117501    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0ac41394-bb87-4c27-9ab9-6b0d7988cf3a-credential-socket\") pod \"e2e-api-b-9e422b-openshift-default-6bd67bb6-dqsxv\" (UID: \"0ac41394-bb87-4c27-9ab9-6b0d7988cf3a\") " pod="openshift-ingress/e2e-api-b-9e422b-openshift-default-6bd67bb6-dqsxv"
Sep 07 12:14:37.117525 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:14:37.117514    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0ac41394-bb87-4c27-9ab9-6b0d7988cf3a-istiod-ca-cert\") pod \"e2e-api-b-9e422b-openshift-default-6bd67bb6-dqsxv\" (UID: \"0ac41394-bb87-4c27-9ab9-6b0d7988cf3a\") " pod="openshift-ingress/e2e-api-b-9e422b-openshift-default-6bd67bb6-dqsxv"
Sep 07 12:14:37.119234 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:14:37.119207    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0ac41394-bb87-4c27-9ab9-6b0d7988cf3a-istio-envoy\") pod \"e2e-api-b-9e422b-openshift-default-6bd67bb6-dqsxv\" (UID: \"0ac41394-bb87-4c27-9ab9-6b0d7988cf3a\") " pod="openshift-ingress/e2e-api-b-9e422b-openshift-default-6bd67bb6-dqsxv"
Sep 07 12:14:37.119341 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:14:37.119327    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0ac41394-bb87-4c27-9ab9-6b0d7988cf3a-istio-podinfo\") pod \"e2e-api-b-9e422b-openshift-default-6bd67bb6-dqsxv\" (UID: \"0ac41394-bb87-4c27-9ab9-6b0d7988cf3a\") " pod="openshift-ingress/e2e-api-b-9e422b-openshift-default-6bd67bb6-dqsxv"
Sep 07 12:14:37.124114 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:14:37.124093    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0ac41394-bb87-4c27-9ab9-6b0d7988cf3a-istio-token\") pod \"e2e-api-b-9e422b-openshift-default-6bd67bb6-dqsxv\" (UID: \"0ac41394-bb87-4c27-9ab9-6b0d7988cf3a\") " pod="openshift-ingress/e2e-api-b-9e422b-openshift-default-6bd67bb6-dqsxv"
Sep 07 12:14:37.124368 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:14:37.124350    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tftqn\" (UniqueName: \"kubernetes.io/projected/0ac41394-bb87-4c27-9ab9-6b0d7988cf3a-kube-api-access-tftqn\") pod \"e2e-api-b-9e422b-openshift-default-6bd67bb6-dqsxv\" (UID: \"0ac41394-bb87-4c27-9ab9-6b0d7988cf3a\") " pod="openshift-ingress/e2e-api-b-9e422b-openshift-default-6bd67bb6-dqsxv"
Sep 07 12:14:37.162320 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:14:37.162228    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:14:37.303660 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:14:37.303619    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-9e422b-openshift-default-6bd67bb6-dqsxv"
Sep 07 12:14:37.342472 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:14:37.342443    2563 patch_prober.go:28] interesting pod/e2e-api-a-f7f063-openshift-default-766d6644cb-fdxmb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" start-of-body=
Sep 07 12:14:37.342836 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:14:37.342495    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-f7f063-openshift-default-766d6644cb-fdxmb" podUID="f12e7fd5-0004-46c0-a944-929644ac9263" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused"
Sep 07 12:14:37.584294 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:14:37.584262    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-9e422b-openshift-default-6bd67bb6-dqsxv"]
Sep 07 12:14:37.586363 ip-10-0-135-173 kubenswrapper[2563]: W0907 12:14:37.586337    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0ac41394_bb87_4c27_9ab9_6b0d7988cf3a.slice/crio-d44b9d1bb58e1be15f5c5ca7fc584497f7112e5995962a745fa6f0ed08ddbd7a WatchSource:0}: Error finding container d44b9d1bb58e1be15f5c5ca7fc584497f7112e5995962a745fa6f0ed08ddbd7a: Status 404 returned error can't find the container with id d44b9d1bb58e1be15f5c5ca7fc584497f7112e5995962a745fa6f0ed08ddbd7a
Sep 07 12:14:37.588794 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:14:37.588762    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 07 12:14:37.588885 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:14:37.588839    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 07 12:14:37.588885 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:14:37.588871    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 07 12:14:38.228022 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:14:38.227990    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:14:38.346267 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:14:38.346235    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-f7f063-openshift-default-766d6644cb-fdxmb"
Sep 07 12:14:38.346749 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:14:38.346601    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-f7f063-openshift-default-766d6644cb-fdxmb"
Sep 07 12:14:38.347394 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:14:38.347378    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-f7f063-openshift-default-766d6644cb-fdxmb"
Sep 07 12:14:38.559457 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:14:38.559344    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-9e422b-openshift-default-6bd67bb6-dqsxv" event={"ID":"0ac41394-bb87-4c27-9ab9-6b0d7988cf3a","Type":"ContainerStarted","Data":"b4566c9e92267f1ace947cdba29f97dfc2c8057cc114a8b07ff041f0779283ac"}
Sep 07 12:14:38.559457 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:14:38.559396    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-9e422b-openshift-default-6bd67bb6-dqsxv" event={"ID":"0ac41394-bb87-4c27-9ab9-6b0d7988cf3a","Type":"ContainerStarted","Data":"d44b9d1bb58e1be15f5c5ca7fc584497f7112e5995962a745fa6f0ed08ddbd7a"}
Sep 07 12:14:38.580061 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:14:38.580003    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-9e422b-openshift-default-6bd67bb6-dqsxv" podStartSLOduration=2.579983087 podStartE2EDuration="2.579983087s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 12:14:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 12:14:38.578931505 +0000 UTC m=+1650.097663319" watchObservedRunningTime="2026-09-07 12:14:38.579983087 +0000 UTC m=+1650.098714901"
Sep 07 12:14:39.304261 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:14:39.304227    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-9e422b-openshift-default-6bd67bb6-dqsxv"
Sep 07 12:14:39.309071 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:14:39.309041    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-9e422b-openshift-default-6bd67bb6-dqsxv"
Sep 07 12:14:39.564696 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:14:39.564606    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-9e422b-openshift-default-6bd67bb6-dqsxv"
Sep 07 12:14:39.565524 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:14:39.565504    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-9e422b-openshift-default-6bd67bb6-dqsxv"
Sep 07 12:14:40.451739 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:14:40.451683    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:14:40.548222 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:14:40.548187    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:14:41.644700 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:14:41.644671    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:14:41.744756 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:14:41.744719    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:14:41.895279 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:14:41.895182    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:14:42.214933 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:14:42.214899    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:14:42.317868 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:14:42.317840    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:14:44.003873 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:14:44.003834    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-p4l4w"]
Sep 07 12:14:56.624695 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:14:56.624658    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:15:00.142242 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:15:00.142201    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29813055-qcgd9"]
Sep 07 12:15:00.144985 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:15:00.144968    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29813055-qcgd9"
Sep 07 12:15:00.147096 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:15:00.147078    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-54hhj\""
Sep 07 12:15:00.158868 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:15:00.158835    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29813055-qcgd9"]
Sep 07 12:15:00.173449 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:15:00.173214    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29813055-qcgd9"
Sep 07 12:15:00.181544 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:15:00.181519    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-491296-29813055-hn6bf"]
Sep 07 12:15:00.199503 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:15:00.199471    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-491296-29813055-hn6bf"
Sep 07 12:15:00.201397 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:15:00.201359    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-491296-29813055-hn6bf"]
Sep 07 12:15:00.226538 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:15:00.226513    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-491296-29813055-hn6bf"
Sep 07 12:15:00.477824 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:15:00.477798    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29813055-qcgd9"]
Sep 07 12:15:00.480216 ip-10-0-135-173 kubenswrapper[2563]: W0907 12:15:00.480187    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8de764fa_7c1b_4e38_a8d3_33b96f0825d3.slice/crio-63c671f00340a0176b2ef50a274f8972805f06bb4c79217efd33fedb71af246e WatchSource:0}: Error finding container 63c671f00340a0176b2ef50a274f8972805f06bb4c79217efd33fedb71af246e: Status 404 returned error can't find the container with id 63c671f00340a0176b2ef50a274f8972805f06bb4c79217efd33fedb71af246e
Sep 07 12:15:00.482244 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:15:00.482226    2563 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 07 12:15:00.536059 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:15:00.536034    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-491296-29813055-hn6bf"]
Sep 07 12:15:00.537278 ip-10-0-135-173 kubenswrapper[2563]: W0907 12:15:00.537256    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0c7cdf98_fe7f_470a_82d0_cd71f862e8b0.slice/crio-3a6a53ef61b2be4f774137f36eb2b7158b7e499fe8eaebca7090388665bd401a WatchSource:0}: Error finding container 3a6a53ef61b2be4f774137f36eb2b7158b7e499fe8eaebca7090388665bd401a: Status 404 returned error can't find the container with id 3a6a53ef61b2be4f774137f36eb2b7158b7e499fe8eaebca7090388665bd401a
Sep 07 12:15:00.683011 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:15:00.682971    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-491296-29813055-hn6bf" event={"ID":"0c7cdf98-fe7f-470a-82d0-cd71f862e8b0","Type":"ContainerStarted","Data":"3a6a53ef61b2be4f774137f36eb2b7158b7e499fe8eaebca7090388665bd401a"}
Sep 07 12:15:00.684133 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:15:00.684105    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29813055-qcgd9" event={"ID":"8de764fa-7c1b-4e38-a8d3-33b96f0825d3","Type":"ContainerStarted","Data":"63c671f00340a0176b2ef50a274f8972805f06bb4c79217efd33fedb71af246e"}
Sep 07 12:15:01.801307 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:15:01.801258    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-p4l4w"]
Sep 07 12:15:02.707393 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:15:02.707351    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:15:03.707938 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:15:03.707902    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-491296-29813055-hn6bf" event={"ID":"0c7cdf98-fe7f-470a-82d0-cd71f862e8b0","Type":"ContainerStarted","Data":"b0e2bd4f864221dec8a65188d0c62da75f5a605711e6ef1dd163a92fc8be73a3"}
Sep 07 12:15:03.712613 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:15:03.712581    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29813055-qcgd9" event={"ID":"8de764fa-7c1b-4e38-a8d3-33b96f0825d3","Type":"ContainerStarted","Data":"c46bffd7ce1ecaf00e3a7b55ae2c0816b183f5f07af77f9bac658af9e24a1f9a"}
Sep 07 12:15:03.726314 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:15:03.726264    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-491296-29813055-hn6bf" podStartSLOduration=1.467871566 podStartE2EDuration="3.726248697s" totalImagesPullingTime="2.258377131s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 12:15:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 12:15:03.724950263 +0000 UTC m=+1675.243682078" watchObservedRunningTime="2026-09-07 12:15:03.726248697 +0000 UTC m=+1675.244980510"
Sep 07 12:15:03.744393 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:15:03.743814    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29813055-qcgd9" podStartSLOduration=1.423941606 podStartE2EDuration="3.743800144s" totalImagesPullingTime="2.319858538s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 12:15:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 12:15:03.743284945 +0000 UTC m=+1675.262016759" watchObservedRunningTime="2026-09-07 12:15:03.743800144 +0000 UTC m=+1675.262531968"
Sep 07 12:15:04.718860 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:15:04.718826    2563 generic.go:356] "Generic (PLEG): container finished" podID="8de764fa-7c1b-4e38-a8d3-33b96f0825d3" containerID="c46bffd7ce1ecaf00e3a7b55ae2c0816b183f5f07af77f9bac658af9e24a1f9a" exitCode=0
Sep 07 12:15:04.719296 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:15:04.718889    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29813055-qcgd9" event={"ID":"8de764fa-7c1b-4e38-a8d3-33b96f0825d3","Type":"ContainerDied","Data":"c46bffd7ce1ecaf00e3a7b55ae2c0816b183f5f07af77f9bac658af9e24a1f9a"}
Sep 07 12:15:04.720614 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:15:04.720589    2563 generic.go:356] "Generic (PLEG): container finished" podID="0ac41394-bb87-4c27-9ab9-6b0d7988cf3a" containerID="b4566c9e92267f1ace947cdba29f97dfc2c8057cc114a8b07ff041f0779283ac" exitCode=0
Sep 07 12:15:04.720753 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:15:04.720704    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-9e422b-openshift-default-6bd67bb6-dqsxv" event={"ID":"0ac41394-bb87-4c27-9ab9-6b0d7988cf3a","Type":"ContainerDied","Data":"b4566c9e92267f1ace947cdba29f97dfc2c8057cc114a8b07ff041f0779283ac"}
Sep 07 12:15:04.721175 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:15:04.721148    2563 scope.go:117] "RemoveContainer" containerID="b4566c9e92267f1ace947cdba29f97dfc2c8057cc114a8b07ff041f0779283ac"
Sep 07 12:15:04.722459 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:15:04.722431    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 07 12:15:04.722569 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:15:04.722512    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 07 12:15:04.722569 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:15:04.722552    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 07 12:15:04.722690 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:15:04.722564    2563 generic.go:356] "Generic (PLEG): container finished" podID="0c7cdf98-fe7f-470a-82d0-cd71f862e8b0" containerID="b0e2bd4f864221dec8a65188d0c62da75f5a605711e6ef1dd163a92fc8be73a3" exitCode=0
Sep 07 12:15:04.722690 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:15:04.722629    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-491296-29813055-hn6bf" event={"ID":"0c7cdf98-fe7f-470a-82d0-cd71f862e8b0","Type":"ContainerDied","Data":"b0e2bd4f864221dec8a65188d0c62da75f5a605711e6ef1dd163a92fc8be73a3"}
Sep 07 12:15:05.370710 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:15:05.370668    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:15:05.680617 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:15:05.680509    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-491296/sub-test-model-491296-kserve-86c54877cc-tjsw2"]
Sep 07 12:15:05.708453 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:15:05.700754    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:15:05.730275 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:15:05.730244    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-9e422b-openshift-default-6bd67bb6-dqsxv" event={"ID":"0ac41394-bb87-4c27-9ab9-6b0d7988cf3a","Type":"ContainerStarted","Data":"a22bbbd043ce357e65f09cb4ca8e043b2d90ee088e35f8534330484eae955614"}
Sep 07 12:15:05.937556 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:15:05.937505    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29813055-qcgd9"
Sep 07 12:15:05.940329 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:15:05.940310    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-491296-29813055-hn6bf"
Sep 07 12:15:06.304180 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:15:06.304133    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-9e422b-openshift-default-6bd67bb6-dqsxv"
Sep 07 12:15:06.305522 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:15:06.305496    2563 patch_prober.go:28] interesting pod/e2e-api-b-9e422b-openshift-default-6bd67bb6-dqsxv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" start-of-body=
Sep 07 12:15:06.305648 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:15:06.305545    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-9e422b-openshift-default-6bd67bb6-dqsxv" podUID="0ac41394-bb87-4c27-9ab9-6b0d7988cf3a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused"
Sep 07 12:15:06.736964 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:15:06.736938    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29813055-qcgd9"
Sep 07 12:15:06.737380 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:15:06.736972    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29813055-qcgd9" event={"ID":"8de764fa-7c1b-4e38-a8d3-33b96f0825d3","Type":"ContainerDied","Data":"63c671f00340a0176b2ef50a274f8972805f06bb4c79217efd33fedb71af246e"}
Sep 07 12:15:06.737380 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:15:06.737017    2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="63c671f00340a0176b2ef50a274f8972805f06bb4c79217efd33fedb71af246e"
Sep 07 12:15:06.738403 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:15:06.738377    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-491296-29813055-hn6bf"
Sep 07 12:15:06.738550 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:15:06.738411    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-491296-29813055-hn6bf" event={"ID":"0c7cdf98-fe7f-470a-82d0-cd71f862e8b0","Type":"ContainerDied","Data":"3a6a53ef61b2be4f774137f36eb2b7158b7e499fe8eaebca7090388665bd401a"}
Sep 07 12:15:06.738550 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:15:06.738460    2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3a6a53ef61b2be4f774137f36eb2b7158b7e499fe8eaebca7090388665bd401a"
Sep 07 12:15:06.788110 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:15:06.788072    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-613886/sub-test-model-613886-kserve-5467c658b7-nnw9k"]
Sep 07 12:15:06.997192 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:15:06.997106    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:15:07.304839 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:15:07.304750    2563 patch_prober.go:28] interesting pod/e2e-api-b-9e422b-openshift-default-6bd67bb6-dqsxv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" start-of-body=
Sep 07 12:15:07.304839 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:15:07.304805    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-9e422b-openshift-default-6bd67bb6-dqsxv" podUID="0ac41394-bb87-4c27-9ab9-6b0d7988cf3a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused"
Sep 07 12:15:07.823106 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:15:07.823066    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:15:08.304221 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:15:08.304182    2563 patch_prober.go:28] interesting pod/e2e-api-b-9e422b-openshift-default-6bd67bb6-dqsxv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" start-of-body=
Sep 07 12:15:08.304397 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:15:08.304237    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-9e422b-openshift-default-6bd67bb6-dqsxv" podUID="0ac41394-bb87-4c27-9ab9-6b0d7988cf3a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused"
Sep 07 12:15:08.617568 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:15:08.617475    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:15:09.304493 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:15:09.304459    2563 patch_prober.go:28] interesting pod/e2e-api-b-9e422b-openshift-default-6bd67bb6-dqsxv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" start-of-body=
Sep 07 12:15:09.304867 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:15:09.304516    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-9e422b-openshift-default-6bd67bb6-dqsxv" podUID="0ac41394-bb87-4c27-9ab9-6b0d7988cf3a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused"
Sep 07 12:15:09.571500 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:15:09.568505    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-9e422b-openshift-default-6bd67bb6-dqsxv"
Sep 07 12:15:10.304067 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:15:10.304032    2563 patch_prober.go:28] interesting pod/e2e-api-b-9e422b-openshift-default-6bd67bb6-dqsxv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" start-of-body=
Sep 07 12:15:10.304247 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:15:10.304086    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-9e422b-openshift-default-6bd67bb6-dqsxv" podUID="0ac41394-bb87-4c27-9ab9-6b0d7988cf3a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused"
Sep 07 12:15:11.308824 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:15:11.308790    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-9e422b-openshift-default-6bd67bb6-dqsxv"
Sep 07 12:15:11.309740 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:15:11.309717    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-9e422b-openshift-default-6bd67bb6-dqsxv"
Sep 07 12:15:19.996115 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:15:19.996076    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-p4l4w"]
Sep 07 12:15:20.705753 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:15:20.705710    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-w3-491296/sub-test-model-491296-kserve-86c54877cc-tjsw2" podUID="1d4bd4c0-9ee3-4300-b7dd-5bc764c3adb9" containerName="main" containerID="cri-o://c45296494070db814e5da63cb500a7795e663d65c564b361a5d841e64f2bbba1" gracePeriod=45
Sep 07 12:15:20.823933 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:15:20.823901    2563 generic.go:356] "Generic (PLEG): container finished" podID="1d4bd4c0-9ee3-4300-b7dd-5bc764c3adb9" containerID="c45296494070db814e5da63cb500a7795e663d65c564b361a5d841e64f2bbba1" exitCode=0
Sep 07 12:15:20.824033 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:15:20.823974    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-491296/sub-test-model-491296-kserve-86c54877cc-tjsw2" event={"ID":"1d4bd4c0-9ee3-4300-b7dd-5bc764c3adb9","Type":"ContainerDied","Data":"c45296494070db814e5da63cb500a7795e663d65c564b361a5d841e64f2bbba1"}
Sep 07 12:15:20.956249 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:15:20.956192    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-491296/sub-test-model-491296-kserve-86c54877cc-tjsw2"
Sep 07 12:15:21.080245 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:15:21.080215    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1d4bd4c0-9ee3-4300-b7dd-5bc764c3adb9-tls-certs\") pod \"1d4bd4c0-9ee3-4300-b7dd-5bc764c3adb9\" (UID: \"1d4bd4c0-9ee3-4300-b7dd-5bc764c3adb9\") "
Sep 07 12:15:21.080684 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:15:21.080251    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-24kvg\" (UniqueName: \"kubernetes.io/projected/1d4bd4c0-9ee3-4300-b7dd-5bc764c3adb9-kube-api-access-24kvg\") pod \"1d4bd4c0-9ee3-4300-b7dd-5bc764c3adb9\" (UID: \"1d4bd4c0-9ee3-4300-b7dd-5bc764c3adb9\") "
Sep 07 12:15:21.080684 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:15:21.080298    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1d4bd4c0-9ee3-4300-b7dd-5bc764c3adb9-tmp-dir\") pod \"1d4bd4c0-9ee3-4300-b7dd-5bc764c3adb9\" (UID: \"1d4bd4c0-9ee3-4300-b7dd-5bc764c3adb9\") "
Sep 07 12:15:21.080684 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:15:21.080320    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1d4bd4c0-9ee3-4300-b7dd-5bc764c3adb9-home\") pod \"1d4bd4c0-9ee3-4300-b7dd-5bc764c3adb9\" (UID: \"1d4bd4c0-9ee3-4300-b7dd-5bc764c3adb9\") "
Sep 07 12:15:21.080684 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:15:21.080344    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1d4bd4c0-9ee3-4300-b7dd-5bc764c3adb9-dshm\") pod \"1d4bd4c0-9ee3-4300-b7dd-5bc764c3adb9\" (UID: \"1d4bd4c0-9ee3-4300-b7dd-5bc764c3adb9\") "
Sep 07 12:15:21.080684 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:15:21.080402    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1d4bd4c0-9ee3-4300-b7dd-5bc764c3adb9-model-cache\") pod \"1d4bd4c0-9ee3-4300-b7dd-5bc764c3adb9\" (UID: \"1d4bd4c0-9ee3-4300-b7dd-5bc764c3adb9\") "
Sep 07 12:15:21.080684 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:15:21.080610    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1d4bd4c0-9ee3-4300-b7dd-5bc764c3adb9-home" (OuterVolumeSpecName: "home") pod "1d4bd4c0-9ee3-4300-b7dd-5bc764c3adb9" (UID: "1d4bd4c0-9ee3-4300-b7dd-5bc764c3adb9"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 12:15:21.080684 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:15:21.080649    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1d4bd4c0-9ee3-4300-b7dd-5bc764c3adb9-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "1d4bd4c0-9ee3-4300-b7dd-5bc764c3adb9" (UID: "1d4bd4c0-9ee3-4300-b7dd-5bc764c3adb9"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 12:15:21.081038 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:15:21.080803    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1d4bd4c0-9ee3-4300-b7dd-5bc764c3adb9-model-cache" (OuterVolumeSpecName: "model-cache") pod "1d4bd4c0-9ee3-4300-b7dd-5bc764c3adb9" (UID: "1d4bd4c0-9ee3-4300-b7dd-5bc764c3adb9"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 12:15:21.081038 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:15:21.080866    2563 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1d4bd4c0-9ee3-4300-b7dd-5bc764c3adb9-model-cache\") on node \"ip-10-0-135-173.ec2.internal\" DevicePath \"\""
Sep 07 12:15:21.081038 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:15:21.080876    2563 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1d4bd4c0-9ee3-4300-b7dd-5bc764c3adb9-tmp-dir\") on node \"ip-10-0-135-173.ec2.internal\" DevicePath \"\""
Sep 07 12:15:21.081038 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:15:21.080891    2563 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1d4bd4c0-9ee3-4300-b7dd-5bc764c3adb9-home\") on node \"ip-10-0-135-173.ec2.internal\" DevicePath \"\""
Sep 07 12:15:21.082359 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:15:21.082340    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1d4bd4c0-9ee3-4300-b7dd-5bc764c3adb9-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "1d4bd4c0-9ee3-4300-b7dd-5bc764c3adb9" (UID: "1d4bd4c0-9ee3-4300-b7dd-5bc764c3adb9"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 07 12:15:21.082892 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:15:21.082869    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1d4bd4c0-9ee3-4300-b7dd-5bc764c3adb9-dshm" (OuterVolumeSpecName: "dshm") pod "1d4bd4c0-9ee3-4300-b7dd-5bc764c3adb9" (UID: "1d4bd4c0-9ee3-4300-b7dd-5bc764c3adb9"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 12:15:21.082892 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:15:21.082881    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1d4bd4c0-9ee3-4300-b7dd-5bc764c3adb9-kube-api-access-24kvg" (OuterVolumeSpecName: "kube-api-access-24kvg") pod "1d4bd4c0-9ee3-4300-b7dd-5bc764c3adb9" (UID: "1d4bd4c0-9ee3-4300-b7dd-5bc764c3adb9"). InnerVolumeSpecName "kube-api-access-24kvg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 12:15:21.182154 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:15:21.182129    2563 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1d4bd4c0-9ee3-4300-b7dd-5bc764c3adb9-dshm\") on node \"ip-10-0-135-173.ec2.internal\" DevicePath \"\""
Sep 07 12:15:21.182154 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:15:21.182149    2563 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1d4bd4c0-9ee3-4300-b7dd-5bc764c3adb9-tls-certs\") on node \"ip-10-0-135-173.ec2.internal\" DevicePath \"\""
Sep 07 12:15:21.182290 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:15:21.182160    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-24kvg\" (UniqueName: \"kubernetes.io/projected/1d4bd4c0-9ee3-4300-b7dd-5bc764c3adb9-kube-api-access-24kvg\") on node \"ip-10-0-135-173.ec2.internal\" DevicePath \"\""
Sep 07 12:15:21.808308 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:15:21.808271    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-w3-613886/sub-test-model-613886-kserve-5467c658b7-nnw9k" podUID="ca4a17f5-413b-4f28-9230-c10dae69ab0f" containerName="main" containerID="cri-o://55a42a32d7e94f9714c1c72eb551cc890c01258d53ad3b7b623ef4cebd536fe7" gracePeriod=45
Sep 07 12:15:21.835552 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:15:21.835526    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-491296/sub-test-model-491296-kserve-86c54877cc-tjsw2"
Sep 07 12:15:21.835674 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:15:21.835525    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-491296/sub-test-model-491296-kserve-86c54877cc-tjsw2" event={"ID":"1d4bd4c0-9ee3-4300-b7dd-5bc764c3adb9","Type":"ContainerDied","Data":"39486421610c872249947fe35d0df16b928e00c534e7d7ca1a16bfecc65aa378"}
Sep 07 12:15:21.835674 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:15:21.835644    2563 scope.go:117] "RemoveContainer" containerID="c45296494070db814e5da63cb500a7795e663d65c564b361a5d841e64f2bbba1"
Sep 07 12:15:21.877684 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:15:21.877660    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-491296/sub-test-model-491296-kserve-86c54877cc-tjsw2"]
Sep 07 12:15:21.882453 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:15:21.882408    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-w3-491296/sub-test-model-491296-kserve-86c54877cc-tjsw2"]
Sep 07 12:15:22.052716 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:15:22.052694    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-613886/sub-test-model-613886-kserve-5467c658b7-nnw9k"
Sep 07 12:15:22.191573 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:15:22.191490    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ca4a17f5-413b-4f28-9230-c10dae69ab0f-tmp-dir\") pod \"ca4a17f5-413b-4f28-9230-c10dae69ab0f\" (UID: \"ca4a17f5-413b-4f28-9230-c10dae69ab0f\") "
Sep 07 12:15:22.191573 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:15:22.191524    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ca4a17f5-413b-4f28-9230-c10dae69ab0f-model-cache\") pod \"ca4a17f5-413b-4f28-9230-c10dae69ab0f\" (UID: \"ca4a17f5-413b-4f28-9230-c10dae69ab0f\") "
Sep 07 12:15:22.192036 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:15:22.191586    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ca4a17f5-413b-4f28-9230-c10dae69ab0f-dshm\") pod \"ca4a17f5-413b-4f28-9230-c10dae69ab0f\" (UID: \"ca4a17f5-413b-4f28-9230-c10dae69ab0f\") "
Sep 07 12:15:22.192036 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:15:22.191630    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nbz5p\" (UniqueName: \"kubernetes.io/projected/ca4a17f5-413b-4f28-9230-c10dae69ab0f-kube-api-access-nbz5p\") pod \"ca4a17f5-413b-4f28-9230-c10dae69ab0f\" (UID: \"ca4a17f5-413b-4f28-9230-c10dae69ab0f\") "
Sep 07 12:15:22.192036 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:15:22.191656    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ca4a17f5-413b-4f28-9230-c10dae69ab0f-tls-certs\") pod \"ca4a17f5-413b-4f28-9230-c10dae69ab0f\" (UID: \"ca4a17f5-413b-4f28-9230-c10dae69ab0f\") "
Sep 07 12:15:22.192036 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:15:22.191700    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ca4a17f5-413b-4f28-9230-c10dae69ab0f-home\") pod \"ca4a17f5-413b-4f28-9230-c10dae69ab0f\" (UID: \"ca4a17f5-413b-4f28-9230-c10dae69ab0f\") "
Sep 07 12:15:22.192036 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:15:22.191733    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ca4a17f5-413b-4f28-9230-c10dae69ab0f-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "ca4a17f5-413b-4f28-9230-c10dae69ab0f" (UID: "ca4a17f5-413b-4f28-9230-c10dae69ab0f"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 12:15:22.192036 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:15:22.191869    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ca4a17f5-413b-4f28-9230-c10dae69ab0f-model-cache" (OuterVolumeSpecName: "model-cache") pod "ca4a17f5-413b-4f28-9230-c10dae69ab0f" (UID: "ca4a17f5-413b-4f28-9230-c10dae69ab0f"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 12:15:22.192036 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:15:22.191967    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ca4a17f5-413b-4f28-9230-c10dae69ab0f-home" (OuterVolumeSpecName: "home") pod "ca4a17f5-413b-4f28-9230-c10dae69ab0f" (UID: "ca4a17f5-413b-4f28-9230-c10dae69ab0f"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 12:15:22.192036 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:15:22.192022    2563 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ca4a17f5-413b-4f28-9230-c10dae69ab0f-tmp-dir\") on node \"ip-10-0-135-173.ec2.internal\" DevicePath \"\""
Sep 07 12:15:22.192036 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:15:22.192037    2563 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ca4a17f5-413b-4f28-9230-c10dae69ab0f-model-cache\") on node \"ip-10-0-135-173.ec2.internal\" DevicePath \"\""
Sep 07 12:15:22.193770 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:15:22.193739    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ca4a17f5-413b-4f28-9230-c10dae69ab0f-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "ca4a17f5-413b-4f28-9230-c10dae69ab0f" (UID: "ca4a17f5-413b-4f28-9230-c10dae69ab0f"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 07 12:15:22.193770 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:15:22.193746    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ca4a17f5-413b-4f28-9230-c10dae69ab0f-dshm" (OuterVolumeSpecName: "dshm") pod "ca4a17f5-413b-4f28-9230-c10dae69ab0f" (UID: "ca4a17f5-413b-4f28-9230-c10dae69ab0f"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 12:15:22.193916 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:15:22.193747    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ca4a17f5-413b-4f28-9230-c10dae69ab0f-kube-api-access-nbz5p" (OuterVolumeSpecName: "kube-api-access-nbz5p") pod "ca4a17f5-413b-4f28-9230-c10dae69ab0f" (UID: "ca4a17f5-413b-4f28-9230-c10dae69ab0f"). InnerVolumeSpecName "kube-api-access-nbz5p". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 12:15:22.292581 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:15:22.292552    2563 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ca4a17f5-413b-4f28-9230-c10dae69ab0f-dshm\") on node \"ip-10-0-135-173.ec2.internal\" DevicePath \"\""
Sep 07 12:15:22.292581 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:15:22.292576    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nbz5p\" (UniqueName: \"kubernetes.io/projected/ca4a17f5-413b-4f28-9230-c10dae69ab0f-kube-api-access-nbz5p\") on node \"ip-10-0-135-173.ec2.internal\" DevicePath \"\""
Sep 07 12:15:22.292748 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:15:22.292588    2563 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ca4a17f5-413b-4f28-9230-c10dae69ab0f-tls-certs\") on node \"ip-10-0-135-173.ec2.internal\" DevicePath \"\""
Sep 07 12:15:22.292748 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:15:22.292599    2563 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ca4a17f5-413b-4f28-9230-c10dae69ab0f-home\") on node \"ip-10-0-135-173.ec2.internal\" DevicePath \"\""
Sep 07 12:15:22.843038 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:15:22.843001    2563 generic.go:356] "Generic (PLEG): container finished" podID="ca4a17f5-413b-4f28-9230-c10dae69ab0f" containerID="55a42a32d7e94f9714c1c72eb551cc890c01258d53ad3b7b623ef4cebd536fe7" exitCode=0
Sep 07 12:15:22.843220 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:15:22.843047    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-613886/sub-test-model-613886-kserve-5467c658b7-nnw9k"
Sep 07 12:15:22.843220 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:15:22.843047    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-613886/sub-test-model-613886-kserve-5467c658b7-nnw9k" event={"ID":"ca4a17f5-413b-4f28-9230-c10dae69ab0f","Type":"ContainerDied","Data":"55a42a32d7e94f9714c1c72eb551cc890c01258d53ad3b7b623ef4cebd536fe7"}
Sep 07 12:15:22.843220 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:15:22.843092    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-613886/sub-test-model-613886-kserve-5467c658b7-nnw9k" event={"ID":"ca4a17f5-413b-4f28-9230-c10dae69ab0f","Type":"ContainerDied","Data":"fd7acb9ce20525463977117dbaf2595aa5c3c0f9c1531fef5d404512850d3ebc"}
Sep 07 12:15:22.843220 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:15:22.843114    2563 scope.go:117] "RemoveContainer" containerID="55a42a32d7e94f9714c1c72eb551cc890c01258d53ad3b7b623ef4cebd536fe7"
Sep 07 12:15:22.852599 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:15:22.852581    2563 scope.go:117] "RemoveContainer" containerID="55a42a32d7e94f9714c1c72eb551cc890c01258d53ad3b7b623ef4cebd536fe7"
Sep 07 12:15:22.852828 ip-10-0-135-173 kubenswrapper[2563]: E0907 12:15:22.852810    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"55a42a32d7e94f9714c1c72eb551cc890c01258d53ad3b7b623ef4cebd536fe7\": container with ID starting with 55a42a32d7e94f9714c1c72eb551cc890c01258d53ad3b7b623ef4cebd536fe7 not found: ID does not exist" containerID="55a42a32d7e94f9714c1c72eb551cc890c01258d53ad3b7b623ef4cebd536fe7"
Sep 07 12:15:22.852873 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:15:22.852837    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"55a42a32d7e94f9714c1c72eb551cc890c01258d53ad3b7b623ef4cebd536fe7"} err="failed to get container status \"55a42a32d7e94f9714c1c72eb551cc890c01258d53ad3b7b623ef4cebd536fe7\": rpc error: code = NotFound desc = could not find container \"55a42a32d7e94f9714c1c72eb551cc890c01258d53ad3b7b623ef4cebd536fe7\": container with ID starting with 55a42a32d7e94f9714c1c72eb551cc890c01258d53ad3b7b623ef4cebd536fe7 not found: ID does not exist"
Sep 07 12:15:22.863453 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:15:22.863410    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-613886/sub-test-model-613886-kserve-5467c658b7-nnw9k"]
Sep 07 12:15:22.865245 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:15:22.865226    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-w3-613886/sub-test-model-613886-kserve-5467c658b7-nnw9k"]
Sep 07 12:15:23.040776 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:15:23.040749    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1d4bd4c0-9ee3-4300-b7dd-5bc764c3adb9" path="/var/lib/kubelet/pods/1d4bd4c0-9ee3-4300-b7dd-5bc764c3adb9/volumes"
Sep 07 12:15:23.041186 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:15:23.041172    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ca4a17f5-413b-4f28-9230-c10dae69ab0f" path="/var/lib/kubelet/pods/ca4a17f5-413b-4f28-9230-c10dae69ab0f/volumes"
Sep 07 12:15:23.717702 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:15:23.717668    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:15:23.818102 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:15:23.818069    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:15:23.914638 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:15:23.914611    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:15:23.975805 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:15:23.975737    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:15:24.263954 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:15:24.263870    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:15:26.655950 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:15:26.655917    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:15:27.137798 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:15:27.137766    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:15:27.237896 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:15:27.237860    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:15:28.401965 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:15:28.401937    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-w3-491296/rate-test-model-491296-kserve-8489bd84d7-64c8v"]
Sep 07 12:15:28.402342 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:15:28.402328    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ca4a17f5-413b-4f28-9230-c10dae69ab0f" containerName="main"
Sep 07 12:15:28.402342 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:15:28.402339    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="ca4a17f5-413b-4f28-9230-c10dae69ab0f" containerName="main"
Sep 07 12:15:28.402435 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:15:28.402355    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8de764fa-7c1b-4e38-a8d3-33b96f0825d3" containerName="cleanup"
Sep 07 12:15:28.402435 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:15:28.402361    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="8de764fa-7c1b-4e38-a8d3-33b96f0825d3" containerName="cleanup"
Sep 07 12:15:28.402435 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:15:28.402379    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1d4bd4c0-9ee3-4300-b7dd-5bc764c3adb9" containerName="main"
Sep 07 12:15:28.402435 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:15:28.402385    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="1d4bd4c0-9ee3-4300-b7dd-5bc764c3adb9" containerName="main"
Sep 07 12:15:28.402435 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:15:28.402393    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0c7cdf98-fe7f-470a-82d0-cd71f862e8b0" containerName="cleanup"
Sep 07 12:15:28.402435 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:15:28.402398    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="0c7cdf98-fe7f-470a-82d0-cd71f862e8b0" containerName="cleanup"
Sep 07 12:15:28.402640 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:15:28.402478    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="0c7cdf98-fe7f-470a-82d0-cd71f862e8b0" containerName="cleanup"
Sep 07 12:15:28.402640 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:15:28.402487    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="1d4bd4c0-9ee3-4300-b7dd-5bc764c3adb9" containerName="main"
Sep 07 12:15:28.402640 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:15:28.402497    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="8de764fa-7c1b-4e38-a8d3-33b96f0825d3" containerName="cleanup"
Sep 07 12:15:28.402640 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:15:28.402504    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="ca4a17f5-413b-4f28-9230-c10dae69ab0f" containerName="main"
Sep 07 12:15:28.407360 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:15:28.407337    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-491296/rate-test-model-491296-kserve-8489bd84d7-64c8v"
Sep 07 12:15:28.409535 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:15:28.409513    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-491296\"/\"default-dockercfg-ztltr\""
Sep 07 12:15:28.409903 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:15:28.409650    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w3-491296\"/\"kube-root-ca.crt\""
Sep 07 12:15:28.409903 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:15:28.409538    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-491296\"/\"rate-test-model-491296-kserve-self-signed-certs\""
Sep 07 12:15:28.410235 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:15:28.410219    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w3-491296\"/\"openshift-service-ca.crt\""
Sep 07 12:15:28.413024 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:15:28.413001    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-491296/rate-test-model-491296-kserve-8489bd84d7-64c8v"]
Sep 07 12:15:28.430620 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:15:28.430594    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:15:28.551622 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:15:28.551582    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9ee646c3-95cd-4f77-a713-675c4cd3ffc3-home\") pod \"rate-test-model-491296-kserve-8489bd84d7-64c8v\" (UID: \"9ee646c3-95cd-4f77-a713-675c4cd3ffc3\") " pod="ai-tenant-e2e-shared-a-w3-491296/rate-test-model-491296-kserve-8489bd84d7-64c8v"
Sep 07 12:15:28.551622 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:15:28.551623    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wlz66\" (UniqueName: \"kubernetes.io/projected/9ee646c3-95cd-4f77-a713-675c4cd3ffc3-kube-api-access-wlz66\") pod \"rate-test-model-491296-kserve-8489bd84d7-64c8v\" (UID: \"9ee646c3-95cd-4f77-a713-675c4cd3ffc3\") " pod="ai-tenant-e2e-shared-a-w3-491296/rate-test-model-491296-kserve-8489bd84d7-64c8v"
Sep 07 12:15:28.551793 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:15:28.551714    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9ee646c3-95cd-4f77-a713-675c4cd3ffc3-model-cache\") pod \"rate-test-model-491296-kserve-8489bd84d7-64c8v\" (UID: \"9ee646c3-95cd-4f77-a713-675c4cd3ffc3\") " pod="ai-tenant-e2e-shared-a-w3-491296/rate-test-model-491296-kserve-8489bd84d7-64c8v"
Sep 07 12:15:28.551793 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:15:28.551742    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9ee646c3-95cd-4f77-a713-675c4cd3ffc3-tls-certs\") pod \"rate-test-model-491296-kserve-8489bd84d7-64c8v\" (UID: \"9ee646c3-95cd-4f77-a713-675c4cd3ffc3\") " pod="ai-tenant-e2e-shared-a-w3-491296/rate-test-model-491296-kserve-8489bd84d7-64c8v"
Sep 07 12:15:28.551793 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:15:28.551773    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9ee646c3-95cd-4f77-a713-675c4cd3ffc3-dshm\") pod \"rate-test-model-491296-kserve-8489bd84d7-64c8v\" (UID: \"9ee646c3-95cd-4f77-a713-675c4cd3ffc3\") " pod="ai-tenant-e2e-shared-a-w3-491296/rate-test-model-491296-kserve-8489bd84d7-64c8v"
Sep 07 12:15:28.551914 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:15:28.551818    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9ee646c3-95cd-4f77-a713-675c4cd3ffc3-tmp-dir\") pod \"rate-test-model-491296-kserve-8489bd84d7-64c8v\" (UID: \"9ee646c3-95cd-4f77-a713-675c4cd3ffc3\") " pod="ai-tenant-e2e-shared-a-w3-491296/rate-test-model-491296-kserve-8489bd84d7-64c8v"
Sep 07 12:15:28.653063 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:15:28.652962    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9ee646c3-95cd-4f77-a713-675c4cd3ffc3-dshm\") pod \"rate-test-model-491296-kserve-8489bd84d7-64c8v\" (UID: \"9ee646c3-95cd-4f77-a713-675c4cd3ffc3\") " pod="ai-tenant-e2e-shared-a-w3-491296/rate-test-model-491296-kserve-8489bd84d7-64c8v"
Sep 07 12:15:28.653063 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:15:28.653044    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9ee646c3-95cd-4f77-a713-675c4cd3ffc3-tmp-dir\") pod \"rate-test-model-491296-kserve-8489bd84d7-64c8v\" (UID: \"9ee646c3-95cd-4f77-a713-675c4cd3ffc3\") " pod="ai-tenant-e2e-shared-a-w3-491296/rate-test-model-491296-kserve-8489bd84d7-64c8v"
Sep 07 12:15:28.653296 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:15:28.653115    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9ee646c3-95cd-4f77-a713-675c4cd3ffc3-home\") pod \"rate-test-model-491296-kserve-8489bd84d7-64c8v\" (UID: \"9ee646c3-95cd-4f77-a713-675c4cd3ffc3\") " pod="ai-tenant-e2e-shared-a-w3-491296/rate-test-model-491296-kserve-8489bd84d7-64c8v"
Sep 07 12:15:28.653296 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:15:28.653140    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wlz66\" (UniqueName: \"kubernetes.io/projected/9ee646c3-95cd-4f77-a713-675c4cd3ffc3-kube-api-access-wlz66\") pod \"rate-test-model-491296-kserve-8489bd84d7-64c8v\" (UID: \"9ee646c3-95cd-4f77-a713-675c4cd3ffc3\") " pod="ai-tenant-e2e-shared-a-w3-491296/rate-test-model-491296-kserve-8489bd84d7-64c8v"
Sep 07 12:15:28.653296 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:15:28.653210    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9ee646c3-95cd-4f77-a713-675c4cd3ffc3-model-cache\") pod \"rate-test-model-491296-kserve-8489bd84d7-64c8v\" (UID: \"9ee646c3-95cd-4f77-a713-675c4cd3ffc3\") " pod="ai-tenant-e2e-shared-a-w3-491296/rate-test-model-491296-kserve-8489bd84d7-64c8v"
Sep 07 12:15:28.653296 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:15:28.653232    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9ee646c3-95cd-4f77-a713-675c4cd3ffc3-tls-certs\") pod \"rate-test-model-491296-kserve-8489bd84d7-64c8v\" (UID: \"9ee646c3-95cd-4f77-a713-675c4cd3ffc3\") " pod="ai-tenant-e2e-shared-a-w3-491296/rate-test-model-491296-kserve-8489bd84d7-64c8v"
Sep 07 12:15:28.653554 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:15:28.653448    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9ee646c3-95cd-4f77-a713-675c4cd3ffc3-tmp-dir\") pod \"rate-test-model-491296-kserve-8489bd84d7-64c8v\" (UID: \"9ee646c3-95cd-4f77-a713-675c4cd3ffc3\") " pod="ai-tenant-e2e-shared-a-w3-491296/rate-test-model-491296-kserve-8489bd84d7-64c8v"
Sep 07 12:15:28.653554 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:15:28.653501    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9ee646c3-95cd-4f77-a713-675c4cd3ffc3-home\") pod \"rate-test-model-491296-kserve-8489bd84d7-64c8v\" (UID: \"9ee646c3-95cd-4f77-a713-675c4cd3ffc3\") " pod="ai-tenant-e2e-shared-a-w3-491296/rate-test-model-491296-kserve-8489bd84d7-64c8v"
Sep 07 12:15:28.653554 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:15:28.653551    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9ee646c3-95cd-4f77-a713-675c4cd3ffc3-model-cache\") pod \"rate-test-model-491296-kserve-8489bd84d7-64c8v\" (UID: \"9ee646c3-95cd-4f77-a713-675c4cd3ffc3\") " pod="ai-tenant-e2e-shared-a-w3-491296/rate-test-model-491296-kserve-8489bd84d7-64c8v"
Sep 07 12:15:28.655229 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:15:28.655205    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9ee646c3-95cd-4f77-a713-675c4cd3ffc3-dshm\") pod \"rate-test-model-491296-kserve-8489bd84d7-64c8v\" (UID: \"9ee646c3-95cd-4f77-a713-675c4cd3ffc3\") " pod="ai-tenant-e2e-shared-a-w3-491296/rate-test-model-491296-kserve-8489bd84d7-64c8v"
Sep 07 12:15:28.655725 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:15:28.655703    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9ee646c3-95cd-4f77-a713-675c4cd3ffc3-tls-certs\") pod \"rate-test-model-491296-kserve-8489bd84d7-64c8v\" (UID: \"9ee646c3-95cd-4f77-a713-675c4cd3ffc3\") " pod="ai-tenant-e2e-shared-a-w3-491296/rate-test-model-491296-kserve-8489bd84d7-64c8v"
Sep 07 12:15:28.661530 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:15:28.661508    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wlz66\" (UniqueName: \"kubernetes.io/projected/9ee646c3-95cd-4f77-a713-675c4cd3ffc3-kube-api-access-wlz66\") pod \"rate-test-model-491296-kserve-8489bd84d7-64c8v\" (UID: \"9ee646c3-95cd-4f77-a713-675c4cd3ffc3\") " pod="ai-tenant-e2e-shared-a-w3-491296/rate-test-model-491296-kserve-8489bd84d7-64c8v"
Sep 07 12:15:28.719300 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:15:28.719276    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-491296/rate-test-model-491296-kserve-8489bd84d7-64c8v"
Sep 07 12:15:28.987643 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:15:28.987619    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-491296/rate-test-model-491296-kserve-8489bd84d7-64c8v"]
Sep 07 12:15:28.990196 ip-10-0-135-173 kubenswrapper[2563]: W0907 12:15:28.990172    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9ee646c3_95cd_4f77_a713_675c4cd3ffc3.slice/crio-85563b5c06ea8332a173cb0ee9756a03aec8ca93b99397982eac9964b7deec00 WatchSource:0}: Error finding container 85563b5c06ea8332a173cb0ee9756a03aec8ca93b99397982eac9964b7deec00: Status 404 returned error can't find the container with id 85563b5c06ea8332a173cb0ee9756a03aec8ca93b99397982eac9964b7deec00
Sep 07 12:15:29.884839 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:15:29.884805    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-491296/rate-test-model-491296-kserve-8489bd84d7-64c8v" event={"ID":"9ee646c3-95cd-4f77-a713-675c4cd3ffc3","Type":"ContainerStarted","Data":"0335c2e1ecc41e47836c1018a8a1aaadf9810bb38dc8337ae175d16f280b0964"}
Sep 07 12:15:29.885215 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:15:29.884849    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-491296/rate-test-model-491296-kserve-8489bd84d7-64c8v" event={"ID":"9ee646c3-95cd-4f77-a713-675c4cd3ffc3","Type":"ContainerStarted","Data":"85563b5c06ea8332a173cb0ee9756a03aec8ca93b99397982eac9964b7deec00"}
Sep 07 12:15:29.902524 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:15:29.902468    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-w3-491296/rate-test-model-491296-kserve-8489bd84d7-64c8v" podStartSLOduration=1.902450427 podStartE2EDuration="1.902450427s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 12:15:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 12:15:29.899941592 +0000 UTC m=+1701.418673406" watchObservedRunningTime="2026-09-07 12:15:29.902450427 +0000 UTC m=+1701.421182239"
Sep 07 12:15:30.313935 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:15:30.313896    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:15:30.494009 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:15:30.493972    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:15:30.555505 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:15:30.555472    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-f7f063-openshift-default-766d6644cb-fdxmb"]
Sep 07 12:15:30.555695 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:15:30.555679    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-f7f063-openshift-default-766d6644cb-fdxmb" podUID="f12e7fd5-0004-46c0-a944-929644ac9263" containerName="istio-proxy" containerID="cri-o://b90362221b21765ac92610531b145ec53a1740836b2f027502ae22e4363f1f28" gracePeriod=30
Sep 07 12:15:30.595232 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:15:30.595147    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:15:31.106662 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:15:31.106616    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:15:35.299976 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:15:35.299933    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-p4l4w"]
Sep 07 12:15:35.807524 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:15:35.807493    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-f7f063-openshift-default-766d6644cb-fdxmb"
Sep 07 12:15:35.916642 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:15:35.916552    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f12e7fd5-0004-46c0-a944-929644ac9263-credential-socket\") pod \"f12e7fd5-0004-46c0-a944-929644ac9263\" (UID: \"f12e7fd5-0004-46c0-a944-929644ac9263\") "
Sep 07 12:15:35.916642 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:15:35.916606    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nh6mv\" (UniqueName: \"kubernetes.io/projected/f12e7fd5-0004-46c0-a944-929644ac9263-kube-api-access-nh6mv\") pod \"f12e7fd5-0004-46c0-a944-929644ac9263\" (UID: \"f12e7fd5-0004-46c0-a944-929644ac9263\") "
Sep 07 12:15:35.916881 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:15:35.916649    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f12e7fd5-0004-46c0-a944-929644ac9263-workload-socket\") pod \"f12e7fd5-0004-46c0-a944-929644ac9263\" (UID: \"f12e7fd5-0004-46c0-a944-929644ac9263\") "
Sep 07 12:15:35.916881 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:15:35.916680    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f12e7fd5-0004-46c0-a944-929644ac9263-istio-podinfo\") pod \"f12e7fd5-0004-46c0-a944-929644ac9263\" (UID: \"f12e7fd5-0004-46c0-a944-929644ac9263\") "
Sep 07 12:15:35.916881 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:15:35.916700    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f12e7fd5-0004-46c0-a944-929644ac9263-istiod-ca-cert\") pod \"f12e7fd5-0004-46c0-a944-929644ac9263\" (UID: \"f12e7fd5-0004-46c0-a944-929644ac9263\") "
Sep 07 12:15:35.916881 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:15:35.916727    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f12e7fd5-0004-46c0-a944-929644ac9263-istio-data\") pod \"f12e7fd5-0004-46c0-a944-929644ac9263\" (UID: \"f12e7fd5-0004-46c0-a944-929644ac9263\") "
Sep 07 12:15:35.916881 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:15:35.916795    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f12e7fd5-0004-46c0-a944-929644ac9263-workload-certs\") pod \"f12e7fd5-0004-46c0-a944-929644ac9263\" (UID: \"f12e7fd5-0004-46c0-a944-929644ac9263\") "
Sep 07 12:15:35.916881 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:15:35.916827    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f12e7fd5-0004-46c0-a944-929644ac9263-istio-token\") pod \"f12e7fd5-0004-46c0-a944-929644ac9263\" (UID: \"f12e7fd5-0004-46c0-a944-929644ac9263\") "
Sep 07 12:15:35.916881 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:15:35.916853    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f12e7fd5-0004-46c0-a944-929644ac9263-istio-envoy\") pod \"f12e7fd5-0004-46c0-a944-929644ac9263\" (UID: \"f12e7fd5-0004-46c0-a944-929644ac9263\") "
Sep 07 12:15:35.917241 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:15:35.916944    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f12e7fd5-0004-46c0-a944-929644ac9263-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "f12e7fd5-0004-46c0-a944-929644ac9263" (UID: "f12e7fd5-0004-46c0-a944-929644ac9263"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 12:15:35.917241 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:15:35.917100    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f12e7fd5-0004-46c0-a944-929644ac9263-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "f12e7fd5-0004-46c0-a944-929644ac9263" (UID: "f12e7fd5-0004-46c0-a944-929644ac9263"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 12:15:35.917241 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:15:35.917210    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f12e7fd5-0004-46c0-a944-929644ac9263-istio-data" (OuterVolumeSpecName: "istio-data") pod "f12e7fd5-0004-46c0-a944-929644ac9263" (UID: "f12e7fd5-0004-46c0-a944-929644ac9263"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 12:15:35.917408 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:15:35.917271    2563 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f12e7fd5-0004-46c0-a944-929644ac9263-workload-socket\") on node \"ip-10-0-135-173.ec2.internal\" DevicePath \"\""
Sep 07 12:15:35.917408 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:15:35.917290    2563 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f12e7fd5-0004-46c0-a944-929644ac9263-workload-certs\") on node \"ip-10-0-135-173.ec2.internal\" DevicePath \"\""
Sep 07 12:15:35.917810 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:15:35.917625    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f12e7fd5-0004-46c0-a944-929644ac9263-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "f12e7fd5-0004-46c0-a944-929644ac9263" (UID: "f12e7fd5-0004-46c0-a944-929644ac9263"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 12:15:35.917810 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:15:35.917774    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f12e7fd5-0004-46c0-a944-929644ac9263-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "f12e7fd5-0004-46c0-a944-929644ac9263" (UID: "f12e7fd5-0004-46c0-a944-929644ac9263"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 07 12:15:35.919266 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:15:35.919218    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f12e7fd5-0004-46c0-a944-929644ac9263-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "f12e7fd5-0004-46c0-a944-929644ac9263" (UID: "f12e7fd5-0004-46c0-a944-929644ac9263"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 12:15:35.919596 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:15:35.919572    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f12e7fd5-0004-46c0-a944-929644ac9263-kube-api-access-nh6mv" (OuterVolumeSpecName: "kube-api-access-nh6mv") pod "f12e7fd5-0004-46c0-a944-929644ac9263" (UID: "f12e7fd5-0004-46c0-a944-929644ac9263"). InnerVolumeSpecName "kube-api-access-nh6mv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 12:15:35.919702 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:15:35.919574    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/f12e7fd5-0004-46c0-a944-929644ac9263-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "f12e7fd5-0004-46c0-a944-929644ac9263" (UID: "f12e7fd5-0004-46c0-a944-929644ac9263"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 07 12:15:35.919702 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:15:35.919675    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f12e7fd5-0004-46c0-a944-929644ac9263-istio-token" (OuterVolumeSpecName: "istio-token") pod "f12e7fd5-0004-46c0-a944-929644ac9263" (UID: "f12e7fd5-0004-46c0-a944-929644ac9263"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 12:15:35.921662 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:15:35.921631    2563 generic.go:356] "Generic (PLEG): container finished" podID="f12e7fd5-0004-46c0-a944-929644ac9263" containerID="b90362221b21765ac92610531b145ec53a1740836b2f027502ae22e4363f1f28" exitCode=0
Sep 07 12:15:35.921776 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:15:35.921733    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-f7f063-openshift-default-766d6644cb-fdxmb"
Sep 07 12:15:35.921776 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:15:35.921762    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-f7f063-openshift-default-766d6644cb-fdxmb" event={"ID":"f12e7fd5-0004-46c0-a944-929644ac9263","Type":"ContainerDied","Data":"b90362221b21765ac92610531b145ec53a1740836b2f027502ae22e4363f1f28"}
Sep 07 12:15:35.921889 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:15:35.921817    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-f7f063-openshift-default-766d6644cb-fdxmb" event={"ID":"f12e7fd5-0004-46c0-a944-929644ac9263","Type":"ContainerDied","Data":"426619fc2e4e9b3c6fc4627dd1c769333ad956350614f0eba6dd56554afaa514"}
Sep 07 12:15:35.921889 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:15:35.921841    2563 scope.go:117] "RemoveContainer" containerID="b90362221b21765ac92610531b145ec53a1740836b2f027502ae22e4363f1f28"
Sep 07 12:15:35.944130 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:15:35.942805    2563 scope.go:117] "RemoveContainer" containerID="874710699c087b1616f29ffd9b09f2bdef7c9e3ac2b6cdfb1c37c5321f7e1ff4"
Sep 07 12:15:35.952943 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:15:35.952922    2563 scope.go:117] "RemoveContainer" containerID="b90362221b21765ac92610531b145ec53a1740836b2f027502ae22e4363f1f28"
Sep 07 12:15:35.953329 ip-10-0-135-173 kubenswrapper[2563]: E0907 12:15:35.953300    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b90362221b21765ac92610531b145ec53a1740836b2f027502ae22e4363f1f28\": container with ID starting with b90362221b21765ac92610531b145ec53a1740836b2f027502ae22e4363f1f28 not found: ID does not exist" containerID="b90362221b21765ac92610531b145ec53a1740836b2f027502ae22e4363f1f28"
Sep 07 12:15:35.953436 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:15:35.953340    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b90362221b21765ac92610531b145ec53a1740836b2f027502ae22e4363f1f28"} err="failed to get container status \"b90362221b21765ac92610531b145ec53a1740836b2f027502ae22e4363f1f28\": rpc error: code = NotFound desc = could not find container \"b90362221b21765ac92610531b145ec53a1740836b2f027502ae22e4363f1f28\": container with ID starting with b90362221b21765ac92610531b145ec53a1740836b2f027502ae22e4363f1f28 not found: ID does not exist"
Sep 07 12:15:35.953436 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:15:35.953369    2563 scope.go:117] "RemoveContainer" containerID="874710699c087b1616f29ffd9b09f2bdef7c9e3ac2b6cdfb1c37c5321f7e1ff4"
Sep 07 12:15:35.953704 ip-10-0-135-173 kubenswrapper[2563]: E0907 12:15:35.953683    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"874710699c087b1616f29ffd9b09f2bdef7c9e3ac2b6cdfb1c37c5321f7e1ff4\": container with ID starting with 874710699c087b1616f29ffd9b09f2bdef7c9e3ac2b6cdfb1c37c5321f7e1ff4 not found: ID does not exist" containerID="874710699c087b1616f29ffd9b09f2bdef7c9e3ac2b6cdfb1c37c5321f7e1ff4"
Sep 07 12:15:35.953754 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:15:35.953710    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"874710699c087b1616f29ffd9b09f2bdef7c9e3ac2b6cdfb1c37c5321f7e1ff4"} err="failed to get container status \"874710699c087b1616f29ffd9b09f2bdef7c9e3ac2b6cdfb1c37c5321f7e1ff4\": rpc error: code = NotFound desc = could not find container \"874710699c087b1616f29ffd9b09f2bdef7c9e3ac2b6cdfb1c37c5321f7e1ff4\": container with ID starting with 874710699c087b1616f29ffd9b09f2bdef7c9e3ac2b6cdfb1c37c5321f7e1ff4 not found: ID does not exist"
Sep 07 12:15:35.954996 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:15:35.954977    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-f7f063-openshift-default-766d6644cb-fdxmb"]
Sep 07 12:15:35.965025 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:15:35.965007    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-f7f063-openshift-default-766d6644cb-fdxmb"]
Sep 07 12:15:36.018169 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:15:36.018131    2563 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f12e7fd5-0004-46c0-a944-929644ac9263-istio-podinfo\") on node \"ip-10-0-135-173.ec2.internal\" DevicePath \"\""
Sep 07 12:15:36.018169 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:15:36.018160    2563 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f12e7fd5-0004-46c0-a944-929644ac9263-istiod-ca-cert\") on node \"ip-10-0-135-173.ec2.internal\" DevicePath \"\""
Sep 07 12:15:36.018169 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:15:36.018170    2563 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f12e7fd5-0004-46c0-a944-929644ac9263-istio-data\") on node \"ip-10-0-135-173.ec2.internal\" DevicePath \"\""
Sep 07 12:15:36.018169 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:15:36.018178    2563 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f12e7fd5-0004-46c0-a944-929644ac9263-istio-token\") on node \"ip-10-0-135-173.ec2.internal\" DevicePath \"\""
Sep 07 12:15:36.018533 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:15:36.018185    2563 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f12e7fd5-0004-46c0-a944-929644ac9263-istio-envoy\") on node \"ip-10-0-135-173.ec2.internal\" DevicePath \"\""
Sep 07 12:15:36.018533 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:15:36.018194    2563 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f12e7fd5-0004-46c0-a944-929644ac9263-credential-socket\") on node \"ip-10-0-135-173.ec2.internal\" DevicePath \"\""
Sep 07 12:15:36.018533 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:15:36.018203    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nh6mv\" (UniqueName: \"kubernetes.io/projected/f12e7fd5-0004-46c0-a944-929644ac9263-kube-api-access-nh6mv\") on node \"ip-10-0-135-173.ec2.internal\" DevicePath \"\""
Sep 07 12:15:37.041289 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:15:37.041251    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f12e7fd5-0004-46c0-a944-929644ac9263" path="/var/lib/kubelet/pods/f12e7fd5-0004-46c0-a944-929644ac9263/volumes"
Sep 07 12:15:38.719668 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:15:38.719634    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-w3-491296/rate-test-model-491296-kserve-8489bd84d7-64c8v"
Sep 07 12:15:38.720037 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:15:38.719677    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-w3-491296/rate-test-model-491296-kserve-8489bd84d7-64c8v"
Sep 07 12:15:38.732276 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:15:38.732254    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-w3-491296/rate-test-model-491296-kserve-8489bd84d7-64c8v"
Sep 07 12:15:38.949845 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:15:38.949819    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-w3-491296/rate-test-model-491296-kserve-8489bd84d7-64c8v"
Sep 07 12:15:46.437673 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:15:46.437619    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:15:46.537238 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:15:46.537205    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:15:46.637232 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:15:46.637196    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:15:46.690358 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:15:46.690278    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:15:47.006283 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:15:47.006249    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:15:49.601521 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:15:49.601490    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-p4l4w"]
Sep 07 12:15:52.691020 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:15:52.690981    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:15:52.750797 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:15:52.750770    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:15:52.928557 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:15:52.928522    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-9e422b-openshift-default-6bd67bb6-dqsxv"]
Sep 07 12:15:52.928809 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:15:52.928782    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-9e422b-openshift-default-6bd67bb6-dqsxv" podUID="0ac41394-bb87-4c27-9ab9-6b0d7988cf3a" containerName="istio-proxy" containerID="cri-o://a22bbbd043ce357e65f09cb4ca8e043b2d90ee088e35f8534330484eae955614" gracePeriod=30
Sep 07 12:15:52.948934 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:15:52.948902    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:15:53.432656 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:15:53.432563    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:15:54.715120 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:15:54.715090    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-w3-613886/rate-test-model-613886-kserve-68dbd4d899-jcpsb"]
Sep 07 12:15:54.715508 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:15:54.715493    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f12e7fd5-0004-46c0-a944-929644ac9263" containerName="istio-proxy"
Sep 07 12:15:54.715556 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:15:54.715510    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="f12e7fd5-0004-46c0-a944-929644ac9263" containerName="istio-proxy"
Sep 07 12:15:54.715556 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:15:54.715518    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f12e7fd5-0004-46c0-a944-929644ac9263" containerName="istio-proxy"
Sep 07 12:15:54.715556 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:15:54.715523    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="f12e7fd5-0004-46c0-a944-929644ac9263" containerName="istio-proxy"
Sep 07 12:15:54.715654 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:15:54.715592    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="f12e7fd5-0004-46c0-a944-929644ac9263" containerName="istio-proxy"
Sep 07 12:15:54.718792 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:15:54.718772    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-613886/rate-test-model-613886-kserve-68dbd4d899-jcpsb"
Sep 07 12:15:54.720906 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:15:54.720887    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-613886\"/\"openshift-service-ca.crt\""
Sep 07 12:15:54.721048 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:15:54.720944    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-613886\"/\"kube-root-ca.crt\""
Sep 07 12:15:54.721048 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:15:54.720979    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-613886\"/\"rate-test-model-613886-kserve-self-signed-certs\""
Sep 07 12:15:54.721048 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:15:54.720891    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-613886\"/\"default-dockercfg-9r7cs\""
Sep 07 12:15:54.729342 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:15:54.729317    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-613886/rate-test-model-613886-kserve-68dbd4d899-jcpsb"]
Sep 07 12:15:54.744752 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:15:54.744726    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:15:54.801112 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:15:54.801089    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/174107f3-6240-40de-a11b-f56c2f51a57c-model-cache\") pod \"rate-test-model-613886-kserve-68dbd4d899-jcpsb\" (UID: \"174107f3-6240-40de-a11b-f56c2f51a57c\") " pod="ai-tenant-e2e-shared-b-w3-613886/rate-test-model-613886-kserve-68dbd4d899-jcpsb"
Sep 07 12:15:54.801218 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:15:54.801119    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/174107f3-6240-40de-a11b-f56c2f51a57c-dshm\") pod \"rate-test-model-613886-kserve-68dbd4d899-jcpsb\" (UID: \"174107f3-6240-40de-a11b-f56c2f51a57c\") " pod="ai-tenant-e2e-shared-b-w3-613886/rate-test-model-613886-kserve-68dbd4d899-jcpsb"
Sep 07 12:15:54.801218 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:15:54.801155    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-np82c\" (UniqueName: \"kubernetes.io/projected/174107f3-6240-40de-a11b-f56c2f51a57c-kube-api-access-np82c\") pod \"rate-test-model-613886-kserve-68dbd4d899-jcpsb\" (UID: \"174107f3-6240-40de-a11b-f56c2f51a57c\") " pod="ai-tenant-e2e-shared-b-w3-613886/rate-test-model-613886-kserve-68dbd4d899-jcpsb"
Sep 07 12:15:54.801306 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:15:54.801222    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/174107f3-6240-40de-a11b-f56c2f51a57c-tmp-dir\") pod \"rate-test-model-613886-kserve-68dbd4d899-jcpsb\" (UID: \"174107f3-6240-40de-a11b-f56c2f51a57c\") " pod="ai-tenant-e2e-shared-b-w3-613886/rate-test-model-613886-kserve-68dbd4d899-jcpsb"
Sep 07 12:15:54.801306 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:15:54.801288    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/174107f3-6240-40de-a11b-f56c2f51a57c-tls-certs\") pod \"rate-test-model-613886-kserve-68dbd4d899-jcpsb\" (UID: \"174107f3-6240-40de-a11b-f56c2f51a57c\") " pod="ai-tenant-e2e-shared-b-w3-613886/rate-test-model-613886-kserve-68dbd4d899-jcpsb"
Sep 07 12:15:54.801388 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:15:54.801323    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/174107f3-6240-40de-a11b-f56c2f51a57c-home\") pod \"rate-test-model-613886-kserve-68dbd4d899-jcpsb\" (UID: \"174107f3-6240-40de-a11b-f56c2f51a57c\") " pod="ai-tenant-e2e-shared-b-w3-613886/rate-test-model-613886-kserve-68dbd4d899-jcpsb"
Sep 07 12:15:54.901957 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:15:54.901927    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-np82c\" (UniqueName: \"kubernetes.io/projected/174107f3-6240-40de-a11b-f56c2f51a57c-kube-api-access-np82c\") pod \"rate-test-model-613886-kserve-68dbd4d899-jcpsb\" (UID: \"174107f3-6240-40de-a11b-f56c2f51a57c\") " pod="ai-tenant-e2e-shared-b-w3-613886/rate-test-model-613886-kserve-68dbd4d899-jcpsb"
Sep 07 12:15:54.902110 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:15:54.901964    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/174107f3-6240-40de-a11b-f56c2f51a57c-tmp-dir\") pod \"rate-test-model-613886-kserve-68dbd4d899-jcpsb\" (UID: \"174107f3-6240-40de-a11b-f56c2f51a57c\") " pod="ai-tenant-e2e-shared-b-w3-613886/rate-test-model-613886-kserve-68dbd4d899-jcpsb"
Sep 07 12:15:54.902110 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:15:54.902006    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/174107f3-6240-40de-a11b-f56c2f51a57c-tls-certs\") pod \"rate-test-model-613886-kserve-68dbd4d899-jcpsb\" (UID: \"174107f3-6240-40de-a11b-f56c2f51a57c\") " pod="ai-tenant-e2e-shared-b-w3-613886/rate-test-model-613886-kserve-68dbd4d899-jcpsb"
Sep 07 12:15:54.902110 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:15:54.902040    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/174107f3-6240-40de-a11b-f56c2f51a57c-home\") pod \"rate-test-model-613886-kserve-68dbd4d899-jcpsb\" (UID: \"174107f3-6240-40de-a11b-f56c2f51a57c\") " pod="ai-tenant-e2e-shared-b-w3-613886/rate-test-model-613886-kserve-68dbd4d899-jcpsb"
Sep 07 12:15:54.902110 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:15:54.902088    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/174107f3-6240-40de-a11b-f56c2f51a57c-model-cache\") pod \"rate-test-model-613886-kserve-68dbd4d899-jcpsb\" (UID: \"174107f3-6240-40de-a11b-f56c2f51a57c\") " pod="ai-tenant-e2e-shared-b-w3-613886/rate-test-model-613886-kserve-68dbd4d899-jcpsb"
Sep 07 12:15:54.902320 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:15:54.902204    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/174107f3-6240-40de-a11b-f56c2f51a57c-dshm\") pod \"rate-test-model-613886-kserve-68dbd4d899-jcpsb\" (UID: \"174107f3-6240-40de-a11b-f56c2f51a57c\") " pod="ai-tenant-e2e-shared-b-w3-613886/rate-test-model-613886-kserve-68dbd4d899-jcpsb"
Sep 07 12:15:54.902368 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:15:54.902336    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/174107f3-6240-40de-a11b-f56c2f51a57c-tmp-dir\") pod \"rate-test-model-613886-kserve-68dbd4d899-jcpsb\" (UID: \"174107f3-6240-40de-a11b-f56c2f51a57c\") " pod="ai-tenant-e2e-shared-b-w3-613886/rate-test-model-613886-kserve-68dbd4d899-jcpsb"
Sep 07 12:15:54.902404 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:15:54.902381    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/174107f3-6240-40de-a11b-f56c2f51a57c-model-cache\") pod \"rate-test-model-613886-kserve-68dbd4d899-jcpsb\" (UID: \"174107f3-6240-40de-a11b-f56c2f51a57c\") " pod="ai-tenant-e2e-shared-b-w3-613886/rate-test-model-613886-kserve-68dbd4d899-jcpsb"
Sep 07 12:15:54.902482 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:15:54.902461    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/174107f3-6240-40de-a11b-f56c2f51a57c-home\") pod \"rate-test-model-613886-kserve-68dbd4d899-jcpsb\" (UID: \"174107f3-6240-40de-a11b-f56c2f51a57c\") " pod="ai-tenant-e2e-shared-b-w3-613886/rate-test-model-613886-kserve-68dbd4d899-jcpsb"
Sep 07 12:15:54.904355 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:15:54.904328    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/174107f3-6240-40de-a11b-f56c2f51a57c-dshm\") pod \"rate-test-model-613886-kserve-68dbd4d899-jcpsb\" (UID: \"174107f3-6240-40de-a11b-f56c2f51a57c\") " pod="ai-tenant-e2e-shared-b-w3-613886/rate-test-model-613886-kserve-68dbd4d899-jcpsb"
Sep 07 12:15:54.904579 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:15:54.904563    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/174107f3-6240-40de-a11b-f56c2f51a57c-tls-certs\") pod \"rate-test-model-613886-kserve-68dbd4d899-jcpsb\" (UID: \"174107f3-6240-40de-a11b-f56c2f51a57c\") " pod="ai-tenant-e2e-shared-b-w3-613886/rate-test-model-613886-kserve-68dbd4d899-jcpsb"
Sep 07 12:15:54.909117 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:15:54.909094    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-np82c\" (UniqueName: \"kubernetes.io/projected/174107f3-6240-40de-a11b-f56c2f51a57c-kube-api-access-np82c\") pod \"rate-test-model-613886-kserve-68dbd4d899-jcpsb\" (UID: \"174107f3-6240-40de-a11b-f56c2f51a57c\") " pod="ai-tenant-e2e-shared-b-w3-613886/rate-test-model-613886-kserve-68dbd4d899-jcpsb"
Sep 07 12:15:55.028237 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:15:55.028151    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-613886/rate-test-model-613886-kserve-68dbd4d899-jcpsb"
Sep 07 12:15:55.299041 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:15:55.298954    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-613886/rate-test-model-613886-kserve-68dbd4d899-jcpsb"]
Sep 07 12:15:55.306308 ip-10-0-135-173 kubenswrapper[2563]: W0907 12:15:55.306284    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod174107f3_6240_40de_a11b_f56c2f51a57c.slice/crio-d14a73eba64c96c5dae4ed0b4f2e9004958dfa28a8bbafd8ece3d82128902672 WatchSource:0}: Error finding container d14a73eba64c96c5dae4ed0b4f2e9004958dfa28a8bbafd8ece3d82128902672: Status 404 returned error can't find the container with id d14a73eba64c96c5dae4ed0b4f2e9004958dfa28a8bbafd8ece3d82128902672
Sep 07 12:15:56.027986 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:15:56.027952    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-613886/rate-test-model-613886-kserve-68dbd4d899-jcpsb" event={"ID":"174107f3-6240-40de-a11b-f56c2f51a57c","Type":"ContainerStarted","Data":"761266d5fe48dbaa96872ed2ab45e48bdad416dac56e833e4cf63dc5df22e5ce"}
Sep 07 12:15:56.027986 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:15:56.027991    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-613886/rate-test-model-613886-kserve-68dbd4d899-jcpsb" event={"ID":"174107f3-6240-40de-a11b-f56c2f51a57c","Type":"ContainerStarted","Data":"d14a73eba64c96c5dae4ed0b4f2e9004958dfa28a8bbafd8ece3d82128902672"}
Sep 07 12:15:56.042958 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:15:56.042911    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-w3-613886/rate-test-model-613886-kserve-68dbd4d899-jcpsb" podStartSLOduration=2.042896918 podStartE2EDuration="2.042896918s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 12:15:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 12:15:56.041253904 +0000 UTC m=+1727.559985717" watchObservedRunningTime="2026-09-07 12:15:56.042896918 +0000 UTC m=+1727.561628725"
Sep 07 12:15:56.311825 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:15:56.311735    2563 patch_prober.go:28] interesting pod/e2e-api-b-9e422b-openshift-default-6bd67bb6-dqsxv container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 07 12:15:56.311825 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:15:56.311796    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-b-9e422b-openshift-default-6bd67bb6-dqsxv" podUID="0ac41394-bb87-4c27-9ab9-6b0d7988cf3a" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 07 12:15:58.220521 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:15:58.220493    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-9e422b-openshift-default-6bd67bb6-dqsxv"
Sep 07 12:15:58.334649 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:15:58.334610    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tftqn\" (UniqueName: \"kubernetes.io/projected/0ac41394-bb87-4c27-9ab9-6b0d7988cf3a-kube-api-access-tftqn\") pod \"0ac41394-bb87-4c27-9ab9-6b0d7988cf3a\" (UID: \"0ac41394-bb87-4c27-9ab9-6b0d7988cf3a\") "
Sep 07 12:15:58.334830 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:15:58.334686    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0ac41394-bb87-4c27-9ab9-6b0d7988cf3a-istiod-ca-cert\") pod \"0ac41394-bb87-4c27-9ab9-6b0d7988cf3a\" (UID: \"0ac41394-bb87-4c27-9ab9-6b0d7988cf3a\") "
Sep 07 12:15:58.334830 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:15:58.334723    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0ac41394-bb87-4c27-9ab9-6b0d7988cf3a-istio-data\") pod \"0ac41394-bb87-4c27-9ab9-6b0d7988cf3a\" (UID: \"0ac41394-bb87-4c27-9ab9-6b0d7988cf3a\") "
Sep 07 12:15:58.334830 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:15:58.334755    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0ac41394-bb87-4c27-9ab9-6b0d7988cf3a-istio-token\") pod \"0ac41394-bb87-4c27-9ab9-6b0d7988cf3a\" (UID: \"0ac41394-bb87-4c27-9ab9-6b0d7988cf3a\") "
Sep 07 12:15:58.334830 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:15:58.334778    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0ac41394-bb87-4c27-9ab9-6b0d7988cf3a-workload-socket\") pod \"0ac41394-bb87-4c27-9ab9-6b0d7988cf3a\" (UID: \"0ac41394-bb87-4c27-9ab9-6b0d7988cf3a\") "
Sep 07 12:15:58.334830 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:15:58.334809    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0ac41394-bb87-4c27-9ab9-6b0d7988cf3a-credential-socket\") pod \"0ac41394-bb87-4c27-9ab9-6b0d7988cf3a\" (UID: \"0ac41394-bb87-4c27-9ab9-6b0d7988cf3a\") "
Sep 07 12:15:58.335108 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:15:58.334864    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0ac41394-bb87-4c27-9ab9-6b0d7988cf3a-workload-certs\") pod \"0ac41394-bb87-4c27-9ab9-6b0d7988cf3a\" (UID: \"0ac41394-bb87-4c27-9ab9-6b0d7988cf3a\") "
Sep 07 12:15:58.335108 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:15:58.334903    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0ac41394-bb87-4c27-9ab9-6b0d7988cf3a-istio-podinfo\") pod \"0ac41394-bb87-4c27-9ab9-6b0d7988cf3a\" (UID: \"0ac41394-bb87-4c27-9ab9-6b0d7988cf3a\") "
Sep 07 12:15:58.335108 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:15:58.334956    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0ac41394-bb87-4c27-9ab9-6b0d7988cf3a-istio-envoy\") pod \"0ac41394-bb87-4c27-9ab9-6b0d7988cf3a\" (UID: \"0ac41394-bb87-4c27-9ab9-6b0d7988cf3a\") "
Sep 07 12:15:58.335108 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:15:58.335043    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0ac41394-bb87-4c27-9ab9-6b0d7988cf3a-istio-data" (OuterVolumeSpecName: "istio-data") pod "0ac41394-bb87-4c27-9ab9-6b0d7988cf3a" (UID: "0ac41394-bb87-4c27-9ab9-6b0d7988cf3a"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 12:15:58.335108 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:15:58.335055    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0ac41394-bb87-4c27-9ab9-6b0d7988cf3a-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "0ac41394-bb87-4c27-9ab9-6b0d7988cf3a" (UID: "0ac41394-bb87-4c27-9ab9-6b0d7988cf3a"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 12:15:58.335108 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:15:58.335081    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0ac41394-bb87-4c27-9ab9-6b0d7988cf3a-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "0ac41394-bb87-4c27-9ab9-6b0d7988cf3a" (UID: "0ac41394-bb87-4c27-9ab9-6b0d7988cf3a"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 07 12:15:58.335409 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:15:58.335212    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0ac41394-bb87-4c27-9ab9-6b0d7988cf3a-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "0ac41394-bb87-4c27-9ab9-6b0d7988cf3a" (UID: "0ac41394-bb87-4c27-9ab9-6b0d7988cf3a"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 12:15:58.335409 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:15:58.335326    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0ac41394-bb87-4c27-9ab9-6b0d7988cf3a-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "0ac41394-bb87-4c27-9ab9-6b0d7988cf3a" (UID: "0ac41394-bb87-4c27-9ab9-6b0d7988cf3a"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 12:15:58.335409 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:15:58.335337    2563 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0ac41394-bb87-4c27-9ab9-6b0d7988cf3a-istio-data\") on node \"ip-10-0-135-173.ec2.internal\" DevicePath \"\""
Sep 07 12:15:58.335409 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:15:58.335356    2563 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0ac41394-bb87-4c27-9ab9-6b0d7988cf3a-workload-socket\") on node \"ip-10-0-135-173.ec2.internal\" DevicePath \"\""
Sep 07 12:15:58.335409 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:15:58.335373    2563 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0ac41394-bb87-4c27-9ab9-6b0d7988cf3a-workload-certs\") on node \"ip-10-0-135-173.ec2.internal\" DevicePath \"\""
Sep 07 12:15:58.335409 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:15:58.335388    2563 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0ac41394-bb87-4c27-9ab9-6b0d7988cf3a-istiod-ca-cert\") on node \"ip-10-0-135-173.ec2.internal\" DevicePath \"\""
Sep 07 12:15:58.337018 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:15:58.336988    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0ac41394-bb87-4c27-9ab9-6b0d7988cf3a-kube-api-access-tftqn" (OuterVolumeSpecName: "kube-api-access-tftqn") pod "0ac41394-bb87-4c27-9ab9-6b0d7988cf3a" (UID: "0ac41394-bb87-4c27-9ab9-6b0d7988cf3a"). InnerVolumeSpecName "kube-api-access-tftqn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 12:15:58.337018 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:15:58.336988    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0ac41394-bb87-4c27-9ab9-6b0d7988cf3a-istio-token" (OuterVolumeSpecName: "istio-token") pod "0ac41394-bb87-4c27-9ab9-6b0d7988cf3a" (UID: "0ac41394-bb87-4c27-9ab9-6b0d7988cf3a"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 12:15:58.337393 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:15:58.337376    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0ac41394-bb87-4c27-9ab9-6b0d7988cf3a-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "0ac41394-bb87-4c27-9ab9-6b0d7988cf3a" (UID: "0ac41394-bb87-4c27-9ab9-6b0d7988cf3a"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 12:15:58.337458 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:15:58.337407    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/0ac41394-bb87-4c27-9ab9-6b0d7988cf3a-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "0ac41394-bb87-4c27-9ab9-6b0d7988cf3a" (UID: "0ac41394-bb87-4c27-9ab9-6b0d7988cf3a"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 07 12:15:58.436743 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:15:58.436708    2563 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0ac41394-bb87-4c27-9ab9-6b0d7988cf3a-istio-podinfo\") on node \"ip-10-0-135-173.ec2.internal\" DevicePath \"\""
Sep 07 12:15:58.436743 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:15:58.436737    2563 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0ac41394-bb87-4c27-9ab9-6b0d7988cf3a-istio-envoy\") on node \"ip-10-0-135-173.ec2.internal\" DevicePath \"\""
Sep 07 12:15:58.436743 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:15:58.436749    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tftqn\" (UniqueName: \"kubernetes.io/projected/0ac41394-bb87-4c27-9ab9-6b0d7988cf3a-kube-api-access-tftqn\") on node \"ip-10-0-135-173.ec2.internal\" DevicePath \"\""
Sep 07 12:15:58.436962 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:15:58.436764    2563 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0ac41394-bb87-4c27-9ab9-6b0d7988cf3a-istio-token\") on node \"ip-10-0-135-173.ec2.internal\" DevicePath \"\""
Sep 07 12:15:58.436962 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:15:58.436775    2563 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0ac41394-bb87-4c27-9ab9-6b0d7988cf3a-credential-socket\") on node \"ip-10-0-135-173.ec2.internal\" DevicePath \"\""
Sep 07 12:15:59.047770 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:15:59.047738    2563 generic.go:356] "Generic (PLEG): container finished" podID="0ac41394-bb87-4c27-9ab9-6b0d7988cf3a" containerID="a22bbbd043ce357e65f09cb4ca8e043b2d90ee088e35f8534330484eae955614" exitCode=0
Sep 07 12:15:59.047927 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:15:59.047783    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-9e422b-openshift-default-6bd67bb6-dqsxv"
Sep 07 12:15:59.047927 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:15:59.047812    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-9e422b-openshift-default-6bd67bb6-dqsxv" event={"ID":"0ac41394-bb87-4c27-9ab9-6b0d7988cf3a","Type":"ContainerDied","Data":"a22bbbd043ce357e65f09cb4ca8e043b2d90ee088e35f8534330484eae955614"}
Sep 07 12:15:59.047927 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:15:59.047858    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-9e422b-openshift-default-6bd67bb6-dqsxv" event={"ID":"0ac41394-bb87-4c27-9ab9-6b0d7988cf3a","Type":"ContainerDied","Data":"d44b9d1bb58e1be15f5c5ca7fc584497f7112e5995962a745fa6f0ed08ddbd7a"}
Sep 07 12:15:59.047927 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:15:59.047883    2563 scope.go:117] "RemoveContainer" containerID="a22bbbd043ce357e65f09cb4ca8e043b2d90ee088e35f8534330484eae955614"
Sep 07 12:15:59.057520 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:15:59.057320    2563 scope.go:117] "RemoveContainer" containerID="b4566c9e92267f1ace947cdba29f97dfc2c8057cc114a8b07ff041f0779283ac"
Sep 07 12:15:59.063386 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:15:59.063364    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-9e422b-openshift-default-6bd67bb6-dqsxv"]
Sep 07 12:15:59.066918 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:15:59.066898    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-9e422b-openshift-default-6bd67bb6-dqsxv"]
Sep 07 12:15:59.068982 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:15:59.068966    2563 scope.go:117] "RemoveContainer" containerID="a22bbbd043ce357e65f09cb4ca8e043b2d90ee088e35f8534330484eae955614"
Sep 07 12:15:59.069263 ip-10-0-135-173 kubenswrapper[2563]: E0907 12:15:59.069233    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a22bbbd043ce357e65f09cb4ca8e043b2d90ee088e35f8534330484eae955614\": container with ID starting with a22bbbd043ce357e65f09cb4ca8e043b2d90ee088e35f8534330484eae955614 not found: ID does not exist" containerID="a22bbbd043ce357e65f09cb4ca8e043b2d90ee088e35f8534330484eae955614"
Sep 07 12:15:59.069303 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:15:59.069269    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a22bbbd043ce357e65f09cb4ca8e043b2d90ee088e35f8534330484eae955614"} err="failed to get container status \"a22bbbd043ce357e65f09cb4ca8e043b2d90ee088e35f8534330484eae955614\": rpc error: code = NotFound desc = could not find container \"a22bbbd043ce357e65f09cb4ca8e043b2d90ee088e35f8534330484eae955614\": container with ID starting with a22bbbd043ce357e65f09cb4ca8e043b2d90ee088e35f8534330484eae955614 not found: ID does not exist"
Sep 07 12:15:59.069303 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:15:59.069285    2563 scope.go:117] "RemoveContainer" containerID="b4566c9e92267f1ace947cdba29f97dfc2c8057cc114a8b07ff041f0779283ac"
Sep 07 12:15:59.069540 ip-10-0-135-173 kubenswrapper[2563]: E0907 12:15:59.069525    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b4566c9e92267f1ace947cdba29f97dfc2c8057cc114a8b07ff041f0779283ac\": container with ID starting with b4566c9e92267f1ace947cdba29f97dfc2c8057cc114a8b07ff041f0779283ac not found: ID does not exist" containerID="b4566c9e92267f1ace947cdba29f97dfc2c8057cc114a8b07ff041f0779283ac"
Sep 07 12:15:59.069588 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:15:59.069544    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b4566c9e92267f1ace947cdba29f97dfc2c8057cc114a8b07ff041f0779283ac"} err="failed to get container status \"b4566c9e92267f1ace947cdba29f97dfc2c8057cc114a8b07ff041f0779283ac\": rpc error: code = NotFound desc = could not find container \"b4566c9e92267f1ace947cdba29f97dfc2c8057cc114a8b07ff041f0779283ac\": container with ID starting with b4566c9e92267f1ace947cdba29f97dfc2c8057cc114a8b07ff041f0779283ac not found: ID does not exist"
Sep 07 12:16:01.040980 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:16:01.040945    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0ac41394-bb87-4c27-9ab9-6b0d7988cf3a" path="/var/lib/kubelet/pods/0ac41394-bb87-4c27-9ab9-6b0d7988cf3a/volumes"
Sep 07 12:16:03.877103 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:16:03.877073    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-3a0bb2-767f497fdc-ck8sf"]
Sep 07 12:16:03.877577 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:16:03.877303    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-3a0bb2-767f497fdc-ck8sf" podUID="9e938f27-d2a4-405f-ba3b-caa5f2b01d13" containerName="maas-api" containerID="cri-o://9e13e152be884f80b5576478b26632622c4f160532d6a69962496cdd5102975f" gracePeriod=30
Sep 07 12:16:03.904862 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:16:03.904834    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:16:03.963041 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:16:03.963001    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:16:04.025291 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:16:04.025236    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:16:04.084959 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:16:04.084930    2563 generic.go:356] "Generic (PLEG): container finished" podID="9e938f27-d2a4-405f-ba3b-caa5f2b01d13" containerID="9e13e152be884f80b5576478b26632622c4f160532d6a69962496cdd5102975f" exitCode=0
Sep 07 12:16:04.085096 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:16:04.084979    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-3a0bb2-767f497fdc-ck8sf" event={"ID":"9e938f27-d2a4-405f-ba3b-caa5f2b01d13","Type":"ContainerDied","Data":"9e13e152be884f80b5576478b26632622c4f160532d6a69962496cdd5102975f"}
Sep 07 12:16:04.102830 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:16:04.102803    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:16:04.125203 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:16:04.125182    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-3a0bb2-767f497fdc-ck8sf"
Sep 07 12:16:04.161077 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:16:04.160986    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:16:04.285788 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:16:04.285753    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9e938f27-d2a4-405f-ba3b-caa5f2b01d13-metrics-certs\") pod \"9e938f27-d2a4-405f-ba3b-caa5f2b01d13\" (UID: \"9e938f27-d2a4-405f-ba3b-caa5f2b01d13\") "
Sep 07 12:16:04.285961 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:16:04.285869    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jvx27\" (UniqueName: \"kubernetes.io/projected/9e938f27-d2a4-405f-ba3b-caa5f2b01d13-kube-api-access-jvx27\") pod \"9e938f27-d2a4-405f-ba3b-caa5f2b01d13\" (UID: \"9e938f27-d2a4-405f-ba3b-caa5f2b01d13\") "
Sep 07 12:16:04.285961 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:16:04.285905    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9e938f27-d2a4-405f-ba3b-caa5f2b01d13-maas-api-tls\") pod \"9e938f27-d2a4-405f-ba3b-caa5f2b01d13\" (UID: \"9e938f27-d2a4-405f-ba3b-caa5f2b01d13\") "
Sep 07 12:16:04.286085 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:16:04.286018    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9e938f27-d2a4-405f-ba3b-caa5f2b01d13-openshift-service-ca\") pod \"9e938f27-d2a4-405f-ba3b-caa5f2b01d13\" (UID: \"9e938f27-d2a4-405f-ba3b-caa5f2b01d13\") "
Sep 07 12:16:04.286474 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:16:04.286441    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9e938f27-d2a4-405f-ba3b-caa5f2b01d13-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "9e938f27-d2a4-405f-ba3b-caa5f2b01d13" (UID: "9e938f27-d2a4-405f-ba3b-caa5f2b01d13"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 07 12:16:04.287990 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:16:04.287961    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9e938f27-d2a4-405f-ba3b-caa5f2b01d13-kube-api-access-jvx27" (OuterVolumeSpecName: "kube-api-access-jvx27") pod "9e938f27-d2a4-405f-ba3b-caa5f2b01d13" (UID: "9e938f27-d2a4-405f-ba3b-caa5f2b01d13"). InnerVolumeSpecName "kube-api-access-jvx27". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 12:16:04.288087 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:16:04.287966    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9e938f27-d2a4-405f-ba3b-caa5f2b01d13-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "9e938f27-d2a4-405f-ba3b-caa5f2b01d13" (UID: "9e938f27-d2a4-405f-ba3b-caa5f2b01d13"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 07 12:16:04.288087 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:16:04.287987    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9e938f27-d2a4-405f-ba3b-caa5f2b01d13-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "9e938f27-d2a4-405f-ba3b-caa5f2b01d13" (UID: "9e938f27-d2a4-405f-ba3b-caa5f2b01d13"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 07 12:16:04.387167 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:16:04.387139    2563 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9e938f27-d2a4-405f-ba3b-caa5f2b01d13-openshift-service-ca\") on node \"ip-10-0-135-173.ec2.internal\" DevicePath \"\""
Sep 07 12:16:04.387167 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:16:04.387169    2563 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9e938f27-d2a4-405f-ba3b-caa5f2b01d13-metrics-certs\") on node \"ip-10-0-135-173.ec2.internal\" DevicePath \"\""
Sep 07 12:16:04.387338 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:16:04.387182    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jvx27\" (UniqueName: \"kubernetes.io/projected/9e938f27-d2a4-405f-ba3b-caa5f2b01d13-kube-api-access-jvx27\") on node \"ip-10-0-135-173.ec2.internal\" DevicePath \"\""
Sep 07 12:16:04.387338 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:16:04.387196    2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9e938f27-d2a4-405f-ba3b-caa5f2b01d13-maas-api-tls\") on node \"ip-10-0-135-173.ec2.internal\" DevicePath \"\""
Sep 07 12:16:04.456193 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:16:04.456156    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:16:05.029085 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:16:05.029054    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-w3-613886/rate-test-model-613886-kserve-68dbd4d899-jcpsb"
Sep 07 12:16:05.029085 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:16:05.029092    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-w3-613886/rate-test-model-613886-kserve-68dbd4d899-jcpsb"
Sep 07 12:16:05.042590 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:16:05.042565    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-w3-613886/rate-test-model-613886-kserve-68dbd4d899-jcpsb"
Sep 07 12:16:05.090884 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:16:05.090863    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-3a0bb2-767f497fdc-ck8sf"
Sep 07 12:16:05.091009 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:16:05.090910    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-3a0bb2-767f497fdc-ck8sf" event={"ID":"9e938f27-d2a4-405f-ba3b-caa5f2b01d13","Type":"ContainerDied","Data":"feb53f1608d398841e0bfc3519669df88728988a10101f63a7bd2fd0ebfa42a5"}
Sep 07 12:16:05.091009 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:16:05.090957    2563 scope.go:117] "RemoveContainer" containerID="9e13e152be884f80b5576478b26632622c4f160532d6a69962496cdd5102975f"
Sep 07 12:16:05.103110 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:16:05.103084    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-w3-613886/rate-test-model-613886-kserve-68dbd4d899-jcpsb"
Sep 07 12:16:05.104391 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:16:05.104367    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-3a0bb2-767f497fdc-ck8sf"]
Sep 07 12:16:05.106479 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:16:05.106453    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-3a0bb2-767f497fdc-ck8sf"]
Sep 07 12:16:07.040831 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:16:07.040795    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9e938f27-d2a4-405f-ba3b-caa5f2b01d13" path="/var/lib/kubelet/pods/9e938f27-d2a4-405f-ba3b-caa5f2b01d13/volumes"
Sep 07 12:16:07.302942 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:16:07.302858    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-p4l4w"]
Sep 07 12:16:10.402746 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:16:10.402713    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:16:10.503803 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:16:10.503772    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:16:10.662768 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:16:10.662690    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:16:11.120401 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:16:11.120367    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:16:20.302127 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:16:20.302092    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-p4l4w"]
Sep 07 12:16:21.520293 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:16:21.520257    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:16:21.583158 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:16:21.583119    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:16:21.684407 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:16:21.684377    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:16:21.737881 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:16:21.737855    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:16:22.042831 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:16:22.042788    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:16:23.948815 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:16:23.948776    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:16:27.951503 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:16:27.951453    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:16:28.051512 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:16:28.051480    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:16:28.199450 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:16:28.199396    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-bff0c9-openshift-default-5ccb878dbd-vv8g2"]
Sep 07 12:16:28.199665 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:16:28.199628    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-w2-bff0c9-openshift-default-5ccb878dbd-vv8g2" podUID="42e3b8e0-8d8e-4d30-b7b0-ce97fd72afed" containerName="istio-proxy" containerID="cri-o://7f45739fad4c828f8c0716e0d25c50ab2dfc0384310b8e70fdff9e8e58ff20a2" gracePeriod=30
Sep 07 12:16:28.208914 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:16:28.208847    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:16:33.443094 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:16:33.443072    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w2-bff0c9-openshift-default-5ccb878dbd-vv8g2"
Sep 07 12:16:33.555220 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:16:33.555153    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/42e3b8e0-8d8e-4d30-b7b0-ce97fd72afed-credential-socket\") pod \"42e3b8e0-8d8e-4d30-b7b0-ce97fd72afed\" (UID: \"42e3b8e0-8d8e-4d30-b7b0-ce97fd72afed\") "
Sep 07 12:16:33.555220 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:16:33.555199    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/42e3b8e0-8d8e-4d30-b7b0-ce97fd72afed-workload-certs\") pod \"42e3b8e0-8d8e-4d30-b7b0-ce97fd72afed\" (UID: \"42e3b8e0-8d8e-4d30-b7b0-ce97fd72afed\") "
Sep 07 12:16:33.555448 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:16:33.555224    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/42e3b8e0-8d8e-4d30-b7b0-ce97fd72afed-istio-data\") pod \"42e3b8e0-8d8e-4d30-b7b0-ce97fd72afed\" (UID: \"42e3b8e0-8d8e-4d30-b7b0-ce97fd72afed\") "
Sep 07 12:16:33.555448 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:16:33.555276    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/42e3b8e0-8d8e-4d30-b7b0-ce97fd72afed-workload-socket\") pod \"42e3b8e0-8d8e-4d30-b7b0-ce97fd72afed\" (UID: \"42e3b8e0-8d8e-4d30-b7b0-ce97fd72afed\") "
Sep 07 12:16:33.555448 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:16:33.555299    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/42e3b8e0-8d8e-4d30-b7b0-ce97fd72afed-istiod-ca-cert\") pod \"42e3b8e0-8d8e-4d30-b7b0-ce97fd72afed\" (UID: \"42e3b8e0-8d8e-4d30-b7b0-ce97fd72afed\") "
Sep 07 12:16:33.555448 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:16:33.555365    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nq7bm\" (UniqueName: \"kubernetes.io/projected/42e3b8e0-8d8e-4d30-b7b0-ce97fd72afed-kube-api-access-nq7bm\") pod \"42e3b8e0-8d8e-4d30-b7b0-ce97fd72afed\" (UID: \"42e3b8e0-8d8e-4d30-b7b0-ce97fd72afed\") "
Sep 07 12:16:33.555448 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:16:33.555396    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/42e3b8e0-8d8e-4d30-b7b0-ce97fd72afed-istio-token\") pod \"42e3b8e0-8d8e-4d30-b7b0-ce97fd72afed\" (UID: \"42e3b8e0-8d8e-4d30-b7b0-ce97fd72afed\") "
Sep 07 12:16:33.555698 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:16:33.555439    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/42e3b8e0-8d8e-4d30-b7b0-ce97fd72afed-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "42e3b8e0-8d8e-4d30-b7b0-ce97fd72afed" (UID: "42e3b8e0-8d8e-4d30-b7b0-ce97fd72afed"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 12:16:33.555698 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:16:33.555452    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/42e3b8e0-8d8e-4d30-b7b0-ce97fd72afed-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "42e3b8e0-8d8e-4d30-b7b0-ce97fd72afed" (UID: "42e3b8e0-8d8e-4d30-b7b0-ce97fd72afed"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 12:16:33.555698 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:16:33.555458    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/42e3b8e0-8d8e-4d30-b7b0-ce97fd72afed-istio-envoy\") pod \"42e3b8e0-8d8e-4d30-b7b0-ce97fd72afed\" (UID: \"42e3b8e0-8d8e-4d30-b7b0-ce97fd72afed\") "
Sep 07 12:16:33.555698 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:16:33.555537    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/42e3b8e0-8d8e-4d30-b7b0-ce97fd72afed-istio-podinfo\") pod \"42e3b8e0-8d8e-4d30-b7b0-ce97fd72afed\" (UID: \"42e3b8e0-8d8e-4d30-b7b0-ce97fd72afed\") "
Sep 07 12:16:33.555698 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:16:33.555550    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/42e3b8e0-8d8e-4d30-b7b0-ce97fd72afed-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "42e3b8e0-8d8e-4d30-b7b0-ce97fd72afed" (UID: "42e3b8e0-8d8e-4d30-b7b0-ce97fd72afed"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 12:16:33.555698 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:16:33.555569    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/42e3b8e0-8d8e-4d30-b7b0-ce97fd72afed-istio-data" (OuterVolumeSpecName: "istio-data") pod "42e3b8e0-8d8e-4d30-b7b0-ce97fd72afed" (UID: "42e3b8e0-8d8e-4d30-b7b0-ce97fd72afed"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 12:16:33.555967 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:16:33.555761    2563 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/42e3b8e0-8d8e-4d30-b7b0-ce97fd72afed-workload-socket\") on node \"ip-10-0-135-173.ec2.internal\" DevicePath \"\""
Sep 07 12:16:33.555967 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:16:33.555777    2563 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/42e3b8e0-8d8e-4d30-b7b0-ce97fd72afed-credential-socket\") on node \"ip-10-0-135-173.ec2.internal\" DevicePath \"\""
Sep 07 12:16:33.555967 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:16:33.555788    2563 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/42e3b8e0-8d8e-4d30-b7b0-ce97fd72afed-workload-certs\") on node \"ip-10-0-135-173.ec2.internal\" DevicePath \"\""
Sep 07 12:16:33.555967 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:16:33.555797    2563 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/42e3b8e0-8d8e-4d30-b7b0-ce97fd72afed-istio-data\") on node \"ip-10-0-135-173.ec2.internal\" DevicePath \"\""
Sep 07 12:16:33.556173 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:16:33.556037    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/42e3b8e0-8d8e-4d30-b7b0-ce97fd72afed-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "42e3b8e0-8d8e-4d30-b7b0-ce97fd72afed" (UID: "42e3b8e0-8d8e-4d30-b7b0-ce97fd72afed"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 07 12:16:33.557641 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:16:33.557616    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/42e3b8e0-8d8e-4d30-b7b0-ce97fd72afed-istio-token" (OuterVolumeSpecName: "istio-token") pod "42e3b8e0-8d8e-4d30-b7b0-ce97fd72afed" (UID: "42e3b8e0-8d8e-4d30-b7b0-ce97fd72afed"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 12:16:33.557744 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:16:33.557667    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/42e3b8e0-8d8e-4d30-b7b0-ce97fd72afed-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "42e3b8e0-8d8e-4d30-b7b0-ce97fd72afed" (UID: "42e3b8e0-8d8e-4d30-b7b0-ce97fd72afed"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 07 12:16:33.557744 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:16:33.557671    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/42e3b8e0-8d8e-4d30-b7b0-ce97fd72afed-kube-api-access-nq7bm" (OuterVolumeSpecName: "kube-api-access-nq7bm") pod "42e3b8e0-8d8e-4d30-b7b0-ce97fd72afed" (UID: "42e3b8e0-8d8e-4d30-b7b0-ce97fd72afed"). InnerVolumeSpecName "kube-api-access-nq7bm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 12:16:33.557744 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:16:33.557693    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/42e3b8e0-8d8e-4d30-b7b0-ce97fd72afed-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "42e3b8e0-8d8e-4d30-b7b0-ce97fd72afed" (UID: "42e3b8e0-8d8e-4d30-b7b0-ce97fd72afed"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 12:16:33.656358 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:16:33.656335    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nq7bm\" (UniqueName: \"kubernetes.io/projected/42e3b8e0-8d8e-4d30-b7b0-ce97fd72afed-kube-api-access-nq7bm\") on node \"ip-10-0-135-173.ec2.internal\" DevicePath \"\""
Sep 07 12:16:33.656358 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:16:33.656357    2563 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/42e3b8e0-8d8e-4d30-b7b0-ce97fd72afed-istio-token\") on node \"ip-10-0-135-173.ec2.internal\" DevicePath \"\""
Sep 07 12:16:33.656532 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:16:33.656365    2563 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/42e3b8e0-8d8e-4d30-b7b0-ce97fd72afed-istio-envoy\") on node \"ip-10-0-135-173.ec2.internal\" DevicePath \"\""
Sep 07 12:16:33.656532 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:16:33.656374    2563 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/42e3b8e0-8d8e-4d30-b7b0-ce97fd72afed-istio-podinfo\") on node \"ip-10-0-135-173.ec2.internal\" DevicePath \"\""
Sep 07 12:16:33.656532 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:16:33.656382    2563 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/42e3b8e0-8d8e-4d30-b7b0-ce97fd72afed-istiod-ca-cert\") on node \"ip-10-0-135-173.ec2.internal\" DevicePath \"\""
Sep 07 12:16:34.236004 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:16:34.235968    2563 generic.go:356] "Generic (PLEG): container finished" podID="42e3b8e0-8d8e-4d30-b7b0-ce97fd72afed" containerID="7f45739fad4c828f8c0716e0d25c50ab2dfc0384310b8e70fdff9e8e58ff20a2" exitCode=0
Sep 07 12:16:34.236187 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:16:34.236035    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w2-bff0c9-openshift-default-5ccb878dbd-vv8g2"
Sep 07 12:16:34.236187 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:16:34.236058    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-bff0c9-openshift-default-5ccb878dbd-vv8g2" event={"ID":"42e3b8e0-8d8e-4d30-b7b0-ce97fd72afed","Type":"ContainerDied","Data":"7f45739fad4c828f8c0716e0d25c50ab2dfc0384310b8e70fdff9e8e58ff20a2"}
Sep 07 12:16:34.236187 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:16:34.236117    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-bff0c9-openshift-default-5ccb878dbd-vv8g2" event={"ID":"42e3b8e0-8d8e-4d30-b7b0-ce97fd72afed","Type":"ContainerDied","Data":"d76a94836595e035b6381c82999334574e08c9eae4189ad0b3fd2bbb56490db0"}
Sep 07 12:16:34.236187 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:16:34.236148    2563 scope.go:117] "RemoveContainer" containerID="7f45739fad4c828f8c0716e0d25c50ab2dfc0384310b8e70fdff9e8e58ff20a2"
Sep 07 12:16:34.245435 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:16:34.245397    2563 scope.go:117] "RemoveContainer" containerID="f11d690d4306100794e2365a5a0237e13f4b54f60cc44b49ac1b5152e78d180a"
Sep 07 12:16:34.253453 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:16:34.253437    2563 scope.go:117] "RemoveContainer" containerID="7f45739fad4c828f8c0716e0d25c50ab2dfc0384310b8e70fdff9e8e58ff20a2"
Sep 07 12:16:34.253717 ip-10-0-135-173 kubenswrapper[2563]: E0907 12:16:34.253700    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7f45739fad4c828f8c0716e0d25c50ab2dfc0384310b8e70fdff9e8e58ff20a2\": container with ID starting with 7f45739fad4c828f8c0716e0d25c50ab2dfc0384310b8e70fdff9e8e58ff20a2 not found: ID does not exist" containerID="7f45739fad4c828f8c0716e0d25c50ab2dfc0384310b8e70fdff9e8e58ff20a2"
Sep 07 12:16:34.253784 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:16:34.253762    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7f45739fad4c828f8c0716e0d25c50ab2dfc0384310b8e70fdff9e8e58ff20a2"} err="failed to get container status \"7f45739fad4c828f8c0716e0d25c50ab2dfc0384310b8e70fdff9e8e58ff20a2\": rpc error: code = NotFound desc = could not find container \"7f45739fad4c828f8c0716e0d25c50ab2dfc0384310b8e70fdff9e8e58ff20a2\": container with ID starting with 7f45739fad4c828f8c0716e0d25c50ab2dfc0384310b8e70fdff9e8e58ff20a2 not found: ID does not exist"
Sep 07 12:16:34.253850 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:16:34.253812    2563 scope.go:117] "RemoveContainer" containerID="f11d690d4306100794e2365a5a0237e13f4b54f60cc44b49ac1b5152e78d180a"
Sep 07 12:16:34.254211 ip-10-0-135-173 kubenswrapper[2563]: E0907 12:16:34.254165    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f11d690d4306100794e2365a5a0237e13f4b54f60cc44b49ac1b5152e78d180a\": container with ID starting with f11d690d4306100794e2365a5a0237e13f4b54f60cc44b49ac1b5152e78d180a not found: ID does not exist" containerID="f11d690d4306100794e2365a5a0237e13f4b54f60cc44b49ac1b5152e78d180a"
Sep 07 12:16:34.254347 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:16:34.254211    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f11d690d4306100794e2365a5a0237e13f4b54f60cc44b49ac1b5152e78d180a"} err="failed to get container status \"f11d690d4306100794e2365a5a0237e13f4b54f60cc44b49ac1b5152e78d180a\": rpc error: code = NotFound desc = could not find container \"f11d690d4306100794e2365a5a0237e13f4b54f60cc44b49ac1b5152e78d180a\": container with ID starting with f11d690d4306100794e2365a5a0237e13f4b54f60cc44b49ac1b5152e78d180a not found: ID does not exist"
Sep 07 12:16:34.256448 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:16:34.256411    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-bff0c9-openshift-default-5ccb878dbd-vv8g2"]
Sep 07 12:16:34.259863 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:16:34.259844    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-bff0c9-openshift-default-5ccb878dbd-vv8g2"]
Sep 07 12:16:35.041112 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:16:35.041077    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="42e3b8e0-8d8e-4d30-b7b0-ce97fd72afed" path="/var/lib/kubelet/pods/42e3b8e0-8d8e-4d30-b7b0-ce97fd72afed/volumes"
Sep 07 12:16:37.505482 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:16:37.505451    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-p4l4w"]
Sep 07 12:16:41.700801 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:16:41.700770    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-491296/rate-test-model-491296-kserve-8489bd84d7-64c8v"]
Sep 07 12:16:41.723120 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:16:41.723088    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:16:42.724735 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:16:42.724705    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-613886/rate-test-model-613886-kserve-68dbd4d899-jcpsb"]
Sep 07 12:16:42.743250 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:16:42.743215    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:16:43.151009 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:16:43.150928    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:16:43.425948 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:16:43.425868    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-a-aab734-openshift-default-8558845d6b-lq2x8"]
Sep 07 12:16:43.426378 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:16:43.426356    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9e938f27-d2a4-405f-ba3b-caa5f2b01d13" containerName="maas-api"
Sep 07 12:16:43.426509 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:16:43.426382    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="9e938f27-d2a4-405f-ba3b-caa5f2b01d13" containerName="maas-api"
Sep 07 12:16:43.426509 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:16:43.426406    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="42e3b8e0-8d8e-4d30-b7b0-ce97fd72afed" containerName="istio-proxy"
Sep 07 12:16:43.426509 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:16:43.426438    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="42e3b8e0-8d8e-4d30-b7b0-ce97fd72afed" containerName="istio-proxy"
Sep 07 12:16:43.426509 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:16:43.426454    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="42e3b8e0-8d8e-4d30-b7b0-ce97fd72afed" containerName="istio-proxy"
Sep 07 12:16:43.426509 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:16:43.426460    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="42e3b8e0-8d8e-4d30-b7b0-ce97fd72afed" containerName="istio-proxy"
Sep 07 12:16:43.426509 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:16:43.426466    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0ac41394-bb87-4c27-9ab9-6b0d7988cf3a" containerName="istio-proxy"
Sep 07 12:16:43.426509 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:16:43.426472    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="0ac41394-bb87-4c27-9ab9-6b0d7988cf3a" containerName="istio-proxy"
Sep 07 12:16:43.426509 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:16:43.426493    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0ac41394-bb87-4c27-9ab9-6b0d7988cf3a" containerName="istio-proxy"
Sep 07 12:16:43.426509 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:16:43.426500    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="0ac41394-bb87-4c27-9ab9-6b0d7988cf3a" containerName="istio-proxy"
Sep 07 12:16:43.427109 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:16:43.426577    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="0ac41394-bb87-4c27-9ab9-6b0d7988cf3a" containerName="istio-proxy"
Sep 07 12:16:43.427109 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:16:43.426586    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="f12e7fd5-0004-46c0-a944-929644ac9263" containerName="istio-proxy"
Sep 07 12:16:43.427109 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:16:43.426593    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="0ac41394-bb87-4c27-9ab9-6b0d7988cf3a" containerName="istio-proxy"
Sep 07 12:16:43.427109 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:16:43.426598    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="42e3b8e0-8d8e-4d30-b7b0-ce97fd72afed" containerName="istio-proxy"
Sep 07 12:16:43.427109 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:16:43.426603    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="42e3b8e0-8d8e-4d30-b7b0-ce97fd72afed" containerName="istio-proxy"
Sep 07 12:16:43.427109 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:16:43.426609    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="9e938f27-d2a4-405f-ba3b-caa5f2b01d13" containerName="maas-api"
Sep 07 12:16:43.429663 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:16:43.429643    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-aab734-openshift-default-8558845d6b-lq2x8"
Sep 07 12:16:43.431410 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:16:43.431387    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Sep 07 12:16:43.431539 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:16:43.431453    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-a-aab734-openshift-default-dockercfg-qbw6r\""
Sep 07 12:16:43.441533 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:16:43.441513    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-a-aab734-openshift-default-8558845d6b-lq2x8"]
Sep 07 12:16:43.458274 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:16:43.458251    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:16:43.550432 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:16:43.550383    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a0c5913e-5a31-47c6-b62d-cca418880291-istio-token\") pod \"e2e-ipp-a-aab734-openshift-default-8558845d6b-lq2x8\" (UID: \"a0c5913e-5a31-47c6-b62d-cca418880291\") " pod="openshift-ingress/e2e-ipp-a-aab734-openshift-default-8558845d6b-lq2x8"
Sep 07 12:16:43.550561 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:16:43.550433    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vmdp5\" (UniqueName: \"kubernetes.io/projected/a0c5913e-5a31-47c6-b62d-cca418880291-kube-api-access-vmdp5\") pod \"e2e-ipp-a-aab734-openshift-default-8558845d6b-lq2x8\" (UID: \"a0c5913e-5a31-47c6-b62d-cca418880291\") " pod="openshift-ingress/e2e-ipp-a-aab734-openshift-default-8558845d6b-lq2x8"
Sep 07 12:16:43.550561 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:16:43.550464    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a0c5913e-5a31-47c6-b62d-cca418880291-istio-data\") pod \"e2e-ipp-a-aab734-openshift-default-8558845d6b-lq2x8\" (UID: \"a0c5913e-5a31-47c6-b62d-cca418880291\") " pod="openshift-ingress/e2e-ipp-a-aab734-openshift-default-8558845d6b-lq2x8"
Sep 07 12:16:43.550561 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:16:43.550508    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a0c5913e-5a31-47c6-b62d-cca418880291-istio-envoy\") pod \"e2e-ipp-a-aab734-openshift-default-8558845d6b-lq2x8\" (UID: \"a0c5913e-5a31-47c6-b62d-cca418880291\") " pod="openshift-ingress/e2e-ipp-a-aab734-openshift-default-8558845d6b-lq2x8"
Sep 07 12:16:43.550561 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:16:43.550531    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a0c5913e-5a31-47c6-b62d-cca418880291-istio-podinfo\") pod \"e2e-ipp-a-aab734-openshift-default-8558845d6b-lq2x8\" (UID: \"a0c5913e-5a31-47c6-b62d-cca418880291\") " pod="openshift-ingress/e2e-ipp-a-aab734-openshift-default-8558845d6b-lq2x8"
Sep 07 12:16:43.550561 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:16:43.550550    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a0c5913e-5a31-47c6-b62d-cca418880291-workload-socket\") pod \"e2e-ipp-a-aab734-openshift-default-8558845d6b-lq2x8\" (UID: \"a0c5913e-5a31-47c6-b62d-cca418880291\") " pod="openshift-ingress/e2e-ipp-a-aab734-openshift-default-8558845d6b-lq2x8"
Sep 07 12:16:43.550561 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:16:43.550564    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a0c5913e-5a31-47c6-b62d-cca418880291-credential-socket\") pod \"e2e-ipp-a-aab734-openshift-default-8558845d6b-lq2x8\" (UID: \"a0c5913e-5a31-47c6-b62d-cca418880291\") " pod="openshift-ingress/e2e-ipp-a-aab734-openshift-default-8558845d6b-lq2x8"
Sep 07 12:16:43.550894 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:16:43.550583    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a0c5913e-5a31-47c6-b62d-cca418880291-istiod-ca-cert\") pod \"e2e-ipp-a-aab734-openshift-default-8558845d6b-lq2x8\" (UID: \"a0c5913e-5a31-47c6-b62d-cca418880291\") " pod="openshift-ingress/e2e-ipp-a-aab734-openshift-default-8558845d6b-lq2x8"
Sep 07 12:16:43.550894 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:16:43.550666    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a0c5913e-5a31-47c6-b62d-cca418880291-workload-certs\") pod \"e2e-ipp-a-aab734-openshift-default-8558845d6b-lq2x8\" (UID: \"a0c5913e-5a31-47c6-b62d-cca418880291\") " pod="openshift-ingress/e2e-ipp-a-aab734-openshift-default-8558845d6b-lq2x8"
Sep 07 12:16:43.651881 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:16:43.651857    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a0c5913e-5a31-47c6-b62d-cca418880291-istio-data\") pod \"e2e-ipp-a-aab734-openshift-default-8558845d6b-lq2x8\" (UID: \"a0c5913e-5a31-47c6-b62d-cca418880291\") " pod="openshift-ingress/e2e-ipp-a-aab734-openshift-default-8558845d6b-lq2x8"
Sep 07 12:16:43.652011 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:16:43.651897    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a0c5913e-5a31-47c6-b62d-cca418880291-istio-envoy\") pod \"e2e-ipp-a-aab734-openshift-default-8558845d6b-lq2x8\" (UID: \"a0c5913e-5a31-47c6-b62d-cca418880291\") " pod="openshift-ingress/e2e-ipp-a-aab734-openshift-default-8558845d6b-lq2x8"
Sep 07 12:16:43.652011 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:16:43.651921    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a0c5913e-5a31-47c6-b62d-cca418880291-istio-podinfo\") pod \"e2e-ipp-a-aab734-openshift-default-8558845d6b-lq2x8\" (UID: \"a0c5913e-5a31-47c6-b62d-cca418880291\") " pod="openshift-ingress/e2e-ipp-a-aab734-openshift-default-8558845d6b-lq2x8"
Sep 07 12:16:43.652011 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:16:43.651939    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a0c5913e-5a31-47c6-b62d-cca418880291-workload-socket\") pod \"e2e-ipp-a-aab734-openshift-default-8558845d6b-lq2x8\" (UID: \"a0c5913e-5a31-47c6-b62d-cca418880291\") " pod="openshift-ingress/e2e-ipp-a-aab734-openshift-default-8558845d6b-lq2x8"
Sep 07 12:16:43.652011 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:16:43.651974    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a0c5913e-5a31-47c6-b62d-cca418880291-credential-socket\") pod \"e2e-ipp-a-aab734-openshift-default-8558845d6b-lq2x8\" (UID: \"a0c5913e-5a31-47c6-b62d-cca418880291\") " pod="openshift-ingress/e2e-ipp-a-aab734-openshift-default-8558845d6b-lq2x8"
Sep 07 12:16:43.652011 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:16:43.652004    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a0c5913e-5a31-47c6-b62d-cca418880291-istiod-ca-cert\") pod \"e2e-ipp-a-aab734-openshift-default-8558845d6b-lq2x8\" (UID: \"a0c5913e-5a31-47c6-b62d-cca418880291\") " pod="openshift-ingress/e2e-ipp-a-aab734-openshift-default-8558845d6b-lq2x8"
Sep 07 12:16:43.652297 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:16:43.652124    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a0c5913e-5a31-47c6-b62d-cca418880291-workload-certs\") pod \"e2e-ipp-a-aab734-openshift-default-8558845d6b-lq2x8\" (UID: \"a0c5913e-5a31-47c6-b62d-cca418880291\") " pod="openshift-ingress/e2e-ipp-a-aab734-openshift-default-8558845d6b-lq2x8"
Sep 07 12:16:43.652297 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:16:43.652237    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a0c5913e-5a31-47c6-b62d-cca418880291-istio-token\") pod \"e2e-ipp-a-aab734-openshift-default-8558845d6b-lq2x8\" (UID: \"a0c5913e-5a31-47c6-b62d-cca418880291\") " pod="openshift-ingress/e2e-ipp-a-aab734-openshift-default-8558845d6b-lq2x8"
Sep 07 12:16:43.652297 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:16:43.652267    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vmdp5\" (UniqueName: \"kubernetes.io/projected/a0c5913e-5a31-47c6-b62d-cca418880291-kube-api-access-vmdp5\") pod \"e2e-ipp-a-aab734-openshift-default-8558845d6b-lq2x8\" (UID: \"a0c5913e-5a31-47c6-b62d-cca418880291\") " pod="openshift-ingress/e2e-ipp-a-aab734-openshift-default-8558845d6b-lq2x8"
Sep 07 12:16:43.652475 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:16:43.652300    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a0c5913e-5a31-47c6-b62d-cca418880291-istio-data\") pod \"e2e-ipp-a-aab734-openshift-default-8558845d6b-lq2x8\" (UID: \"a0c5913e-5a31-47c6-b62d-cca418880291\") " pod="openshift-ingress/e2e-ipp-a-aab734-openshift-default-8558845d6b-lq2x8"
Sep 07 12:16:43.652475 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:16:43.652319    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a0c5913e-5a31-47c6-b62d-cca418880291-workload-socket\") pod \"e2e-ipp-a-aab734-openshift-default-8558845d6b-lq2x8\" (UID: \"a0c5913e-5a31-47c6-b62d-cca418880291\") " pod="openshift-ingress/e2e-ipp-a-aab734-openshift-default-8558845d6b-lq2x8"
Sep 07 12:16:43.652475 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:16:43.652391    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a0c5913e-5a31-47c6-b62d-cca418880291-credential-socket\") pod \"e2e-ipp-a-aab734-openshift-default-8558845d6b-lq2x8\" (UID: \"a0c5913e-5a31-47c6-b62d-cca418880291\") " pod="openshift-ingress/e2e-ipp-a-aab734-openshift-default-8558845d6b-lq2x8"
Sep 07 12:16:43.652644 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:16:43.652573    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a0c5913e-5a31-47c6-b62d-cca418880291-workload-certs\") pod \"e2e-ipp-a-aab734-openshift-default-8558845d6b-lq2x8\" (UID: \"a0c5913e-5a31-47c6-b62d-cca418880291\") " pod="openshift-ingress/e2e-ipp-a-aab734-openshift-default-8558845d6b-lq2x8"
Sep 07 12:16:43.653153 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:16:43.653130    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a0c5913e-5a31-47c6-b62d-cca418880291-istiod-ca-cert\") pod \"e2e-ipp-a-aab734-openshift-default-8558845d6b-lq2x8\" (UID: \"a0c5913e-5a31-47c6-b62d-cca418880291\") " pod="openshift-ingress/e2e-ipp-a-aab734-openshift-default-8558845d6b-lq2x8"
Sep 07 12:16:43.654459 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:16:43.654442    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a0c5913e-5a31-47c6-b62d-cca418880291-istio-envoy\") pod \"e2e-ipp-a-aab734-openshift-default-8558845d6b-lq2x8\" (UID: \"a0c5913e-5a31-47c6-b62d-cca418880291\") " pod="openshift-ingress/e2e-ipp-a-aab734-openshift-default-8558845d6b-lq2x8"
Sep 07 12:16:43.654774 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:16:43.654754    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a0c5913e-5a31-47c6-b62d-cca418880291-istio-podinfo\") pod \"e2e-ipp-a-aab734-openshift-default-8558845d6b-lq2x8\" (UID: \"a0c5913e-5a31-47c6-b62d-cca418880291\") " pod="openshift-ingress/e2e-ipp-a-aab734-openshift-default-8558845d6b-lq2x8"
Sep 07 12:16:43.659136 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:16:43.659117    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a0c5913e-5a31-47c6-b62d-cca418880291-istio-token\") pod \"e2e-ipp-a-aab734-openshift-default-8558845d6b-lq2x8\" (UID: \"a0c5913e-5a31-47c6-b62d-cca418880291\") " pod="openshift-ingress/e2e-ipp-a-aab734-openshift-default-8558845d6b-lq2x8"
Sep 07 12:16:43.659308 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:16:43.659288    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vmdp5\" (UniqueName: \"kubernetes.io/projected/a0c5913e-5a31-47c6-b62d-cca418880291-kube-api-access-vmdp5\") pod \"e2e-ipp-a-aab734-openshift-default-8558845d6b-lq2x8\" (UID: \"a0c5913e-5a31-47c6-b62d-cca418880291\") " pod="openshift-ingress/e2e-ipp-a-aab734-openshift-default-8558845d6b-lq2x8"
Sep 07 12:16:43.740300 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:16:43.740277    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-aab734-openshift-default-8558845d6b-lq2x8"
Sep 07 12:16:43.783918 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:16:43.783875    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:16:44.013018 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:16:44.012937    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-a-aab734-openshift-default-8558845d6b-lq2x8"]
Sep 07 12:16:44.016783 ip-10-0-135-173 kubenswrapper[2563]: W0907 12:16:44.016756    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda0c5913e_5a31_47c6_b62d_cca418880291.slice/crio-577b559ed9c54877646fc638392c172d39d093064bc62514bfcb6d90eb0a34f7 WatchSource:0}: Error finding container 577b559ed9c54877646fc638392c172d39d093064bc62514bfcb6d90eb0a34f7: Status 404 returned error can't find the container with id 577b559ed9c54877646fc638392c172d39d093064bc62514bfcb6d90eb0a34f7
Sep 07 12:16:44.019108 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:16:44.019078    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 07 12:16:44.019198 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:16:44.019149    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 07 12:16:44.019198 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:16:44.019176    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 07 12:16:44.287666 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:16:44.287585    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-aab734-openshift-default-8558845d6b-lq2x8" event={"ID":"a0c5913e-5a31-47c6-b62d-cca418880291","Type":"ContainerStarted","Data":"5cd87e26d7bb0f226c83cd8b7c0356c4ffc56c721a91f951b499953d363218a3"}
Sep 07 12:16:44.287666 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:16:44.287643    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-aab734-openshift-default-8558845d6b-lq2x8" event={"ID":"a0c5913e-5a31-47c6-b62d-cca418880291","Type":"ContainerStarted","Data":"577b559ed9c54877646fc638392c172d39d093064bc62514bfcb6d90eb0a34f7"}
Sep 07 12:16:44.307177 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:16:44.307116    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-a-aab734-openshift-default-8558845d6b-lq2x8" podStartSLOduration=1.307095669 podStartE2EDuration="1.307095669s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 12:16:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 12:16:44.303656149 +0000 UTC m=+1775.822387961" watchObservedRunningTime="2026-09-07 12:16:44.307095669 +0000 UTC m=+1775.825827484"
Sep 07 12:16:44.741166 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:16:44.741136    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-a-aab734-openshift-default-8558845d6b-lq2x8"
Sep 07 12:16:45.745537 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:16:45.745507    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-a-aab734-openshift-default-8558845d6b-lq2x8"
Sep 07 12:16:46.173067 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:16:46.172983    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:16:46.264528 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:16:46.264494    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:16:46.302701 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:16:46.302672    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-a-aab734-openshift-default-8558845d6b-lq2x8"
Sep 07 12:16:46.303888 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:16:46.303864    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-a-aab734-openshift-default-8558845d6b-lq2x8"
Sep 07 12:16:47.446375 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:16:47.446344    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:16:47.545590 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:16:47.545555    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:16:47.664008 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:16:47.663980    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:16:47.935689 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:16:47.935608    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:16:48.011408 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:16:48.011375    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:16:51.006138 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:16:51.006104    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-p4l4w"]
Sep 07 12:16:56.722510 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:16:56.722446    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-w3-491296/rate-test-model-491296-kserve-8489bd84d7-64c8v" podUID="9ee646c3-95cd-4f77-a713-675c4cd3ffc3" containerName="main" containerID="cri-o://0335c2e1ecc41e47836c1018a8a1aaadf9810bb38dc8337ae175d16f280b0964" gracePeriod=45
Sep 07 12:16:57.000185 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:16:57.000158    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-491296/rate-test-model-491296-kserve-8489bd84d7-64c8v"
Sep 07 12:16:57.180133 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:16:57.180097    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9ee646c3-95cd-4f77-a713-675c4cd3ffc3-dshm\") pod \"9ee646c3-95cd-4f77-a713-675c4cd3ffc3\" (UID: \"9ee646c3-95cd-4f77-a713-675c4cd3ffc3\") "
Sep 07 12:16:57.180297 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:16:57.180217    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9ee646c3-95cd-4f77-a713-675c4cd3ffc3-model-cache\") pod \"9ee646c3-95cd-4f77-a713-675c4cd3ffc3\" (UID: \"9ee646c3-95cd-4f77-a713-675c4cd3ffc3\") "
Sep 07 12:16:57.180297 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:16:57.180246    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9ee646c3-95cd-4f77-a713-675c4cd3ffc3-tmp-dir\") pod \"9ee646c3-95cd-4f77-a713-675c4cd3ffc3\" (UID: \"9ee646c3-95cd-4f77-a713-675c4cd3ffc3\") "
Sep 07 12:16:57.180371 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:16:57.180298    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wlz66\" (UniqueName: \"kubernetes.io/projected/9ee646c3-95cd-4f77-a713-675c4cd3ffc3-kube-api-access-wlz66\") pod \"9ee646c3-95cd-4f77-a713-675c4cd3ffc3\" (UID: \"9ee646c3-95cd-4f77-a713-675c4cd3ffc3\") "
Sep 07 12:16:57.180371 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:16:57.180335    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9ee646c3-95cd-4f77-a713-675c4cd3ffc3-home\") pod \"9ee646c3-95cd-4f77-a713-675c4cd3ffc3\" (UID: \"9ee646c3-95cd-4f77-a713-675c4cd3ffc3\") "
Sep 07 12:16:57.180463 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:16:57.180393    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9ee646c3-95cd-4f77-a713-675c4cd3ffc3-tls-certs\") pod \"9ee646c3-95cd-4f77-a713-675c4cd3ffc3\" (UID: \"9ee646c3-95cd-4f77-a713-675c4cd3ffc3\") "
Sep 07 12:16:57.180533 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:16:57.180489    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9ee646c3-95cd-4f77-a713-675c4cd3ffc3-model-cache" (OuterVolumeSpecName: "model-cache") pod "9ee646c3-95cd-4f77-a713-675c4cd3ffc3" (UID: "9ee646c3-95cd-4f77-a713-675c4cd3ffc3"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 12:16:57.180585 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:16:57.180555    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9ee646c3-95cd-4f77-a713-675c4cd3ffc3-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "9ee646c3-95cd-4f77-a713-675c4cd3ffc3" (UID: "9ee646c3-95cd-4f77-a713-675c4cd3ffc3"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 12:16:57.180643 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:16:57.180618    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9ee646c3-95cd-4f77-a713-675c4cd3ffc3-home" (OuterVolumeSpecName: "home") pod "9ee646c3-95cd-4f77-a713-675c4cd3ffc3" (UID: "9ee646c3-95cd-4f77-a713-675c4cd3ffc3"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 12:16:57.180794 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:16:57.180773    2563 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9ee646c3-95cd-4f77-a713-675c4cd3ffc3-model-cache\") on node \"ip-10-0-135-173.ec2.internal\" DevicePath \"\""
Sep 07 12:16:57.180794 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:16:57.180796    2563 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9ee646c3-95cd-4f77-a713-675c4cd3ffc3-tmp-dir\") on node \"ip-10-0-135-173.ec2.internal\" DevicePath \"\""
Sep 07 12:16:57.180977 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:16:57.180807    2563 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9ee646c3-95cd-4f77-a713-675c4cd3ffc3-home\") on node \"ip-10-0-135-173.ec2.internal\" DevicePath \"\""
Sep 07 12:16:57.182372 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:16:57.182350    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9ee646c3-95cd-4f77-a713-675c4cd3ffc3-kube-api-access-wlz66" (OuterVolumeSpecName: "kube-api-access-wlz66") pod "9ee646c3-95cd-4f77-a713-675c4cd3ffc3" (UID: "9ee646c3-95cd-4f77-a713-675c4cd3ffc3"). InnerVolumeSpecName "kube-api-access-wlz66". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 12:16:57.182472 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:16:57.182408    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9ee646c3-95cd-4f77-a713-675c4cd3ffc3-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "9ee646c3-95cd-4f77-a713-675c4cd3ffc3" (UID: "9ee646c3-95cd-4f77-a713-675c4cd3ffc3"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 07 12:16:57.182536 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:16:57.182519    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9ee646c3-95cd-4f77-a713-675c4cd3ffc3-dshm" (OuterVolumeSpecName: "dshm") pod "9ee646c3-95cd-4f77-a713-675c4cd3ffc3" (UID: "9ee646c3-95cd-4f77-a713-675c4cd3ffc3"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 12:16:57.282045 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:16:57.281960    2563 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9ee646c3-95cd-4f77-a713-675c4cd3ffc3-tls-certs\") on node \"ip-10-0-135-173.ec2.internal\" DevicePath \"\""
Sep 07 12:16:57.282045 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:16:57.281997    2563 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9ee646c3-95cd-4f77-a713-675c4cd3ffc3-dshm\") on node \"ip-10-0-135-173.ec2.internal\" DevicePath \"\""
Sep 07 12:16:57.282045 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:16:57.282010    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wlz66\" (UniqueName: \"kubernetes.io/projected/9ee646c3-95cd-4f77-a713-675c4cd3ffc3-kube-api-access-wlz66\") on node \"ip-10-0-135-173.ec2.internal\" DevicePath \"\""
Sep 07 12:16:57.370523 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:16:57.370483    2563 generic.go:356] "Generic (PLEG): container finished" podID="9ee646c3-95cd-4f77-a713-675c4cd3ffc3" containerID="0335c2e1ecc41e47836c1018a8a1aaadf9810bb38dc8337ae175d16f280b0964" exitCode=0
Sep 07 12:16:57.370709 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:16:57.370537    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-491296/rate-test-model-491296-kserve-8489bd84d7-64c8v" event={"ID":"9ee646c3-95cd-4f77-a713-675c4cd3ffc3","Type":"ContainerDied","Data":"0335c2e1ecc41e47836c1018a8a1aaadf9810bb38dc8337ae175d16f280b0964"}
Sep 07 12:16:57.370709 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:16:57.370574    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-491296/rate-test-model-491296-kserve-8489bd84d7-64c8v" event={"ID":"9ee646c3-95cd-4f77-a713-675c4cd3ffc3","Type":"ContainerDied","Data":"85563b5c06ea8332a173cb0ee9756a03aec8ca93b99397982eac9964b7deec00"}
Sep 07 12:16:57.370709 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:16:57.370601    2563 scope.go:117] "RemoveContainer" containerID="0335c2e1ecc41e47836c1018a8a1aaadf9810bb38dc8337ae175d16f280b0964"
Sep 07 12:16:57.370892 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:16:57.370758    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-491296/rate-test-model-491296-kserve-8489bd84d7-64c8v"
Sep 07 12:16:57.383364 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:16:57.383343    2563 scope.go:117] "RemoveContainer" containerID="0335c2e1ecc41e47836c1018a8a1aaadf9810bb38dc8337ae175d16f280b0964"
Sep 07 12:16:57.383659 ip-10-0-135-173 kubenswrapper[2563]: E0907 12:16:57.383635    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0335c2e1ecc41e47836c1018a8a1aaadf9810bb38dc8337ae175d16f280b0964\": container with ID starting with 0335c2e1ecc41e47836c1018a8a1aaadf9810bb38dc8337ae175d16f280b0964 not found: ID does not exist" containerID="0335c2e1ecc41e47836c1018a8a1aaadf9810bb38dc8337ae175d16f280b0964"
Sep 07 12:16:57.383731 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:16:57.383672    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0335c2e1ecc41e47836c1018a8a1aaadf9810bb38dc8337ae175d16f280b0964"} err="failed to get container status \"0335c2e1ecc41e47836c1018a8a1aaadf9810bb38dc8337ae175d16f280b0964\": rpc error: code = NotFound desc = could not find container \"0335c2e1ecc41e47836c1018a8a1aaadf9810bb38dc8337ae175d16f280b0964\": container with ID starting with 0335c2e1ecc41e47836c1018a8a1aaadf9810bb38dc8337ae175d16f280b0964 not found: ID does not exist"
Sep 07 12:16:57.391837 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:16:57.391815    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-491296/rate-test-model-491296-kserve-8489bd84d7-64c8v"]
Sep 07 12:16:57.396116 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:16:57.396087    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-w3-491296/rate-test-model-491296-kserve-8489bd84d7-64c8v"]
Sep 07 12:16:57.744538 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:16:57.744486    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-w3-613886/rate-test-model-613886-kserve-68dbd4d899-jcpsb" podUID="174107f3-6240-40de-a11b-f56c2f51a57c" containerName="main" containerID="cri-o://761266d5fe48dbaa96872ed2ab45e48bdad416dac56e833e4cf63dc5df22e5ce" gracePeriod=45
Sep 07 12:16:58.017808 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:16:58.017787    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-613886/rate-test-model-613886-kserve-68dbd4d899-jcpsb"
Sep 07 12:16:58.189341 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:16:58.189291    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/174107f3-6240-40de-a11b-f56c2f51a57c-tmp-dir\") pod \"174107f3-6240-40de-a11b-f56c2f51a57c\" (UID: \"174107f3-6240-40de-a11b-f56c2f51a57c\") "
Sep 07 12:16:58.189341 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:16:58.189346    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/174107f3-6240-40de-a11b-f56c2f51a57c-tls-certs\") pod \"174107f3-6240-40de-a11b-f56c2f51a57c\" (UID: \"174107f3-6240-40de-a11b-f56c2f51a57c\") "
Sep 07 12:16:58.189602 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:16:58.189389    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/174107f3-6240-40de-a11b-f56c2f51a57c-model-cache\") pod \"174107f3-6240-40de-a11b-f56c2f51a57c\" (UID: \"174107f3-6240-40de-a11b-f56c2f51a57c\") "
Sep 07 12:16:58.189602 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:16:58.189427    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-np82c\" (UniqueName: \"kubernetes.io/projected/174107f3-6240-40de-a11b-f56c2f51a57c-kube-api-access-np82c\") pod \"174107f3-6240-40de-a11b-f56c2f51a57c\" (UID: \"174107f3-6240-40de-a11b-f56c2f51a57c\") "
Sep 07 12:16:58.189602 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:16:58.189478    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/174107f3-6240-40de-a11b-f56c2f51a57c-dshm\") pod \"174107f3-6240-40de-a11b-f56c2f51a57c\" (UID: \"174107f3-6240-40de-a11b-f56c2f51a57c\") "
Sep 07 12:16:58.189602 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:16:58.189520    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/174107f3-6240-40de-a11b-f56c2f51a57c-home\") pod \"174107f3-6240-40de-a11b-f56c2f51a57c\" (UID: \"174107f3-6240-40de-a11b-f56c2f51a57c\") "
Sep 07 12:16:58.189810 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:16:58.189614    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/174107f3-6240-40de-a11b-f56c2f51a57c-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "174107f3-6240-40de-a11b-f56c2f51a57c" (UID: "174107f3-6240-40de-a11b-f56c2f51a57c"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 12:16:58.189810 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:16:58.189633    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/174107f3-6240-40de-a11b-f56c2f51a57c-model-cache" (OuterVolumeSpecName: "model-cache") pod "174107f3-6240-40de-a11b-f56c2f51a57c" (UID: "174107f3-6240-40de-a11b-f56c2f51a57c"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 12:16:58.189938 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:16:58.189802    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/174107f3-6240-40de-a11b-f56c2f51a57c-home" (OuterVolumeSpecName: "home") pod "174107f3-6240-40de-a11b-f56c2f51a57c" (UID: "174107f3-6240-40de-a11b-f56c2f51a57c"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 12:16:58.189938 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:16:58.189860    2563 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/174107f3-6240-40de-a11b-f56c2f51a57c-tmp-dir\") on node \"ip-10-0-135-173.ec2.internal\" DevicePath \"\""
Sep 07 12:16:58.189938 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:16:58.189878    2563 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/174107f3-6240-40de-a11b-f56c2f51a57c-model-cache\") on node \"ip-10-0-135-173.ec2.internal\" DevicePath \"\""
Sep 07 12:16:58.191534 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:16:58.191503    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/174107f3-6240-40de-a11b-f56c2f51a57c-dshm" (OuterVolumeSpecName: "dshm") pod "174107f3-6240-40de-a11b-f56c2f51a57c" (UID: "174107f3-6240-40de-a11b-f56c2f51a57c"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 12:16:58.191627 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:16:58.191563    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/174107f3-6240-40de-a11b-f56c2f51a57c-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "174107f3-6240-40de-a11b-f56c2f51a57c" (UID: "174107f3-6240-40de-a11b-f56c2f51a57c"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 07 12:16:58.191627 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:16:58.191580    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/174107f3-6240-40de-a11b-f56c2f51a57c-kube-api-access-np82c" (OuterVolumeSpecName: "kube-api-access-np82c") pod "174107f3-6240-40de-a11b-f56c2f51a57c" (UID: "174107f3-6240-40de-a11b-f56c2f51a57c"). InnerVolumeSpecName "kube-api-access-np82c". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 12:16:58.291060 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:16:58.290980    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-np82c\" (UniqueName: \"kubernetes.io/projected/174107f3-6240-40de-a11b-f56c2f51a57c-kube-api-access-np82c\") on node \"ip-10-0-135-173.ec2.internal\" DevicePath \"\""
Sep 07 12:16:58.291060 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:16:58.291009    2563 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/174107f3-6240-40de-a11b-f56c2f51a57c-dshm\") on node \"ip-10-0-135-173.ec2.internal\" DevicePath \"\""
Sep 07 12:16:58.291060 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:16:58.291019    2563 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/174107f3-6240-40de-a11b-f56c2f51a57c-home\") on node \"ip-10-0-135-173.ec2.internal\" DevicePath \"\""
Sep 07 12:16:58.291060 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:16:58.291028    2563 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/174107f3-6240-40de-a11b-f56c2f51a57c-tls-certs\") on node \"ip-10-0-135-173.ec2.internal\" DevicePath \"\""
Sep 07 12:16:58.378690 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:16:58.378658    2563 generic.go:356] "Generic (PLEG): container finished" podID="174107f3-6240-40de-a11b-f56c2f51a57c" containerID="761266d5fe48dbaa96872ed2ab45e48bdad416dac56e833e4cf63dc5df22e5ce" exitCode=0
Sep 07 12:16:58.378845 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:16:58.378699    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-613886/rate-test-model-613886-kserve-68dbd4d899-jcpsb"
Sep 07 12:16:58.378845 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:16:58.378736    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-613886/rate-test-model-613886-kserve-68dbd4d899-jcpsb" event={"ID":"174107f3-6240-40de-a11b-f56c2f51a57c","Type":"ContainerDied","Data":"761266d5fe48dbaa96872ed2ab45e48bdad416dac56e833e4cf63dc5df22e5ce"}
Sep 07 12:16:58.378845 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:16:58.378778    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-613886/rate-test-model-613886-kserve-68dbd4d899-jcpsb" event={"ID":"174107f3-6240-40de-a11b-f56c2f51a57c","Type":"ContainerDied","Data":"d14a73eba64c96c5dae4ed0b4f2e9004958dfa28a8bbafd8ece3d82128902672"}
Sep 07 12:16:58.379359 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:16:58.379173    2563 scope.go:117] "RemoveContainer" containerID="761266d5fe48dbaa96872ed2ab45e48bdad416dac56e833e4cf63dc5df22e5ce"
Sep 07 12:16:58.396892 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:16:58.396342    2563 scope.go:117] "RemoveContainer" containerID="761266d5fe48dbaa96872ed2ab45e48bdad416dac56e833e4cf63dc5df22e5ce"
Sep 07 12:16:58.397084 ip-10-0-135-173 kubenswrapper[2563]: E0907 12:16:58.397064    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"761266d5fe48dbaa96872ed2ab45e48bdad416dac56e833e4cf63dc5df22e5ce\": container with ID starting with 761266d5fe48dbaa96872ed2ab45e48bdad416dac56e833e4cf63dc5df22e5ce not found: ID does not exist" containerID="761266d5fe48dbaa96872ed2ab45e48bdad416dac56e833e4cf63dc5df22e5ce"
Sep 07 12:16:58.397130 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:16:58.397092    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"761266d5fe48dbaa96872ed2ab45e48bdad416dac56e833e4cf63dc5df22e5ce"} err="failed to get container status \"761266d5fe48dbaa96872ed2ab45e48bdad416dac56e833e4cf63dc5df22e5ce\": rpc error: code = NotFound desc = could not find container \"761266d5fe48dbaa96872ed2ab45e48bdad416dac56e833e4cf63dc5df22e5ce\": container with ID starting with 761266d5fe48dbaa96872ed2ab45e48bdad416dac56e833e4cf63dc5df22e5ce not found: ID does not exist"
Sep 07 12:16:58.404356 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:16:58.404312    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-613886/rate-test-model-613886-kserve-68dbd4d899-jcpsb"]
Sep 07 12:16:58.406641 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:16:58.406616    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-w3-613886/rate-test-model-613886-kserve-68dbd4d899-jcpsb"]
Sep 07 12:16:59.040361 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:16:59.040333    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="174107f3-6240-40de-a11b-f56c2f51a57c" path="/var/lib/kubelet/pods/174107f3-6240-40de-a11b-f56c2f51a57c/volumes"
Sep 07 12:16:59.040766 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:16:59.040752    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9ee646c3-95cd-4f77-a713-675c4cd3ffc3" path="/var/lib/kubelet/pods/9ee646c3-95cd-4f77-a713-675c4cd3ffc3/volumes"
Sep 07 12:17:01.962245 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:17:01.961782    2563 patch_prober.go:28] interesting pod/e2e-ipp-a-aab734-openshift-default-8558845d6b-lq2x8 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.62:15021/healthz/ready\": read tcp 10.133.0.2:40702->10.133.0.62:15021: read: connection reset by peer" start-of-body=
Sep 07 12:17:01.962245 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:17:01.961856    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ipp-a-aab734-openshift-default-8558845d6b-lq2x8" podUID="a0c5913e-5a31-47c6-b62d-cca418880291" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.62:15021/healthz/ready\": read tcp 10.133.0.2:40702->10.133.0.62:15021: read: connection reset by peer"
Sep 07 12:17:02.402508 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:17:02.402412    2563 generic.go:356] "Generic (PLEG): container finished" podID="a0c5913e-5a31-47c6-b62d-cca418880291" containerID="5cd87e26d7bb0f226c83cd8b7c0356c4ffc56c721a91f951b499953d363218a3" exitCode=0
Sep 07 12:17:02.402508 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:17:02.402486    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-aab734-openshift-default-8558845d6b-lq2x8" event={"ID":"a0c5913e-5a31-47c6-b62d-cca418880291","Type":"ContainerDied","Data":"5cd87e26d7bb0f226c83cd8b7c0356c4ffc56c721a91f951b499953d363218a3"}
Sep 07 12:17:02.402880 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:17:02.402866    2563 scope.go:117] "RemoveContainer" containerID="5cd87e26d7bb0f226c83cd8b7c0356c4ffc56c721a91f951b499953d363218a3"
Sep 07 12:17:02.403920 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:17:02.403893    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 07 12:17:02.404024 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:17:02.403973    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 07 12:17:02.404024 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:17:02.404018    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 07 12:17:03.409794 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:17:03.409762    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-aab734-openshift-default-8558845d6b-lq2x8" event={"ID":"a0c5913e-5a31-47c6-b62d-cca418880291","Type":"ContainerStarted","Data":"6bf6915193b230d9c907eba0dd7cc1d37b67092d0b5f91b68ed7ec0e4a798d20"}
Sep 07 12:17:03.740757 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:17:03.740718    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-a-aab734-openshift-default-8558845d6b-lq2x8"
Sep 07 12:17:03.740984 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:17:03.740960    2563 patch_prober.go:28] interesting pod/e2e-ipp-a-aab734-openshift-default-8558845d6b-lq2x8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused" start-of-body=
Sep 07 12:17:03.741032 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:17:03.741010    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-aab734-openshift-default-8558845d6b-lq2x8" podUID="a0c5913e-5a31-47c6-b62d-cca418880291" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused"
Sep 07 12:17:04.740856 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:17:04.740814    2563 patch_prober.go:28] interesting pod/e2e-ipp-a-aab734-openshift-default-8558845d6b-lq2x8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused" start-of-body=
Sep 07 12:17:04.741313 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:17:04.740876    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-aab734-openshift-default-8558845d6b-lq2x8" podUID="a0c5913e-5a31-47c6-b62d-cca418880291" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused"
Sep 07 12:17:05.741472 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:17:05.741405    2563 patch_prober.go:28] interesting pod/e2e-ipp-a-aab734-openshift-default-8558845d6b-lq2x8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused" start-of-body=
Sep 07 12:17:05.741839 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:17:05.741487    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-aab734-openshift-default-8558845d6b-lq2x8" podUID="a0c5913e-5a31-47c6-b62d-cca418880291" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused"
Sep 07 12:17:06.740969 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:17:06.740931    2563 patch_prober.go:28] interesting pod/e2e-ipp-a-aab734-openshift-default-8558845d6b-lq2x8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused" start-of-body=
Sep 07 12:17:06.741150 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:17:06.740991    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-aab734-openshift-default-8558845d6b-lq2x8" podUID="a0c5913e-5a31-47c6-b62d-cca418880291" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused"
Sep 07 12:17:07.740724 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:17:07.740692    2563 patch_prober.go:28] interesting pod/e2e-ipp-a-aab734-openshift-default-8558845d6b-lq2x8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused" start-of-body=
Sep 07 12:17:07.741091 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:17:07.740761    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-aab734-openshift-default-8558845d6b-lq2x8" podUID="a0c5913e-5a31-47c6-b62d-cca418880291" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused"
Sep 07 12:17:08.747496 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:17:08.747465    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-a-aab734-openshift-default-8558845d6b-lq2x8"
Sep 07 12:17:08.747881 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:17:08.747805    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-a-aab734-openshift-default-8558845d6b-lq2x8"
Sep 07 12:17:08.748444 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:17:08.748408    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-a-aab734-openshift-default-8558845d6b-lq2x8"
Sep 07 12:17:09.242197 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:17:09.242169    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-4nfbc_b34ca85d-ac51-4724-a508-1026106b3f3a/console-operator/2.log"
Sep 07 12:17:09.246128 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:17:09.246104    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-4nfbc_b34ca85d-ac51-4724-a508-1026106b3f3a/console-operator/2.log"
Sep 07 12:17:11.131175 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:17:11.131144    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:17:12.040084 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:17:12.040047    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-b-3e7ca1-openshift-default-7f74dcbfc6-f79fh"]
Sep 07 12:17:12.040463 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:17:12.040443    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9ee646c3-95cd-4f77-a713-675c4cd3ffc3" containerName="main"
Sep 07 12:17:12.040463 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:17:12.040459    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="9ee646c3-95cd-4f77-a713-675c4cd3ffc3" containerName="main"
Sep 07 12:17:12.040668 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:17:12.040474    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="174107f3-6240-40de-a11b-f56c2f51a57c" containerName="main"
Sep 07 12:17:12.040668 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:17:12.040480    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="174107f3-6240-40de-a11b-f56c2f51a57c" containerName="main"
Sep 07 12:17:12.040668 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:17:12.040546    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="9ee646c3-95cd-4f77-a713-675c4cd3ffc3" containerName="main"
Sep 07 12:17:12.040668 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:17:12.040565    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="174107f3-6240-40de-a11b-f56c2f51a57c" containerName="main"
Sep 07 12:17:12.043620 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:17:12.043599    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-3e7ca1-openshift-default-7f74dcbfc6-f79fh"
Sep 07 12:17:12.045460 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:17:12.045436    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-b-3e7ca1-openshift-default-dockercfg-62vzn\""
Sep 07 12:17:12.053177 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:17:12.053155    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-b-3e7ca1-openshift-default-7f74dcbfc6-f79fh"]
Sep 07 12:17:12.115301 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:17:12.115266    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ada380e3-be47-48af-9fde-9d422ac172ed-workload-socket\") pod \"e2e-ipp-b-3e7ca1-openshift-default-7f74dcbfc6-f79fh\" (UID: \"ada380e3-be47-48af-9fde-9d422ac172ed\") " pod="openshift-ingress/e2e-ipp-b-3e7ca1-openshift-default-7f74dcbfc6-f79fh"
Sep 07 12:17:12.115510 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:17:12.115318    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gnmcz\" (UniqueName: \"kubernetes.io/projected/ada380e3-be47-48af-9fde-9d422ac172ed-kube-api-access-gnmcz\") pod \"e2e-ipp-b-3e7ca1-openshift-default-7f74dcbfc6-f79fh\" (UID: \"ada380e3-be47-48af-9fde-9d422ac172ed\") " pod="openshift-ingress/e2e-ipp-b-3e7ca1-openshift-default-7f74dcbfc6-f79fh"
Sep 07 12:17:12.115510 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:17:12.115380    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ada380e3-be47-48af-9fde-9d422ac172ed-istio-token\") pod \"e2e-ipp-b-3e7ca1-openshift-default-7f74dcbfc6-f79fh\" (UID: \"ada380e3-be47-48af-9fde-9d422ac172ed\") " pod="openshift-ingress/e2e-ipp-b-3e7ca1-openshift-default-7f74dcbfc6-f79fh"
Sep 07 12:17:12.115510 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:17:12.115450    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ada380e3-be47-48af-9fde-9d422ac172ed-credential-socket\") pod \"e2e-ipp-b-3e7ca1-openshift-default-7f74dcbfc6-f79fh\" (UID: \"ada380e3-be47-48af-9fde-9d422ac172ed\") " pod="openshift-ingress/e2e-ipp-b-3e7ca1-openshift-default-7f74dcbfc6-f79fh"
Sep 07 12:17:12.115510 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:17:12.115475    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ada380e3-be47-48af-9fde-9d422ac172ed-istio-envoy\") pod \"e2e-ipp-b-3e7ca1-openshift-default-7f74dcbfc6-f79fh\" (UID: \"ada380e3-be47-48af-9fde-9d422ac172ed\") " pod="openshift-ingress/e2e-ipp-b-3e7ca1-openshift-default-7f74dcbfc6-f79fh"
Sep 07 12:17:12.115510 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:17:12.115499    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ada380e3-be47-48af-9fde-9d422ac172ed-istio-podinfo\") pod \"e2e-ipp-b-3e7ca1-openshift-default-7f74dcbfc6-f79fh\" (UID: \"ada380e3-be47-48af-9fde-9d422ac172ed\") " pod="openshift-ingress/e2e-ipp-b-3e7ca1-openshift-default-7f74dcbfc6-f79fh"
Sep 07 12:17:12.115722 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:17:12.115538    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ada380e3-be47-48af-9fde-9d422ac172ed-istiod-ca-cert\") pod \"e2e-ipp-b-3e7ca1-openshift-default-7f74dcbfc6-f79fh\" (UID: \"ada380e3-be47-48af-9fde-9d422ac172ed\") " pod="openshift-ingress/e2e-ipp-b-3e7ca1-openshift-default-7f74dcbfc6-f79fh"
Sep 07 12:17:12.115722 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:17:12.115615    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ada380e3-be47-48af-9fde-9d422ac172ed-istio-data\") pod \"e2e-ipp-b-3e7ca1-openshift-default-7f74dcbfc6-f79fh\" (UID: \"ada380e3-be47-48af-9fde-9d422ac172ed\") " pod="openshift-ingress/e2e-ipp-b-3e7ca1-openshift-default-7f74dcbfc6-f79fh"
Sep 07 12:17:12.115722 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:17:12.115661    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ada380e3-be47-48af-9fde-9d422ac172ed-workload-certs\") pod \"e2e-ipp-b-3e7ca1-openshift-default-7f74dcbfc6-f79fh\" (UID: \"ada380e3-be47-48af-9fde-9d422ac172ed\") " pod="openshift-ingress/e2e-ipp-b-3e7ca1-openshift-default-7f74dcbfc6-f79fh"
Sep 07 12:17:12.216762 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:17:12.216727    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ada380e3-be47-48af-9fde-9d422ac172ed-workload-certs\") pod \"e2e-ipp-b-3e7ca1-openshift-default-7f74dcbfc6-f79fh\" (UID: \"ada380e3-be47-48af-9fde-9d422ac172ed\") " pod="openshift-ingress/e2e-ipp-b-3e7ca1-openshift-default-7f74dcbfc6-f79fh"
Sep 07 12:17:12.217207 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:17:12.216785    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ada380e3-be47-48af-9fde-9d422ac172ed-workload-socket\") pod \"e2e-ipp-b-3e7ca1-openshift-default-7f74dcbfc6-f79fh\" (UID: \"ada380e3-be47-48af-9fde-9d422ac172ed\") " pod="openshift-ingress/e2e-ipp-b-3e7ca1-openshift-default-7f74dcbfc6-f79fh"
Sep 07 12:17:12.217207 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:17:12.216807    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gnmcz\" (UniqueName: \"kubernetes.io/projected/ada380e3-be47-48af-9fde-9d422ac172ed-kube-api-access-gnmcz\") pod \"e2e-ipp-b-3e7ca1-openshift-default-7f74dcbfc6-f79fh\" (UID: \"ada380e3-be47-48af-9fde-9d422ac172ed\") " pod="openshift-ingress/e2e-ipp-b-3e7ca1-openshift-default-7f74dcbfc6-f79fh"
Sep 07 12:17:12.217207 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:17:12.216834    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ada380e3-be47-48af-9fde-9d422ac172ed-istio-token\") pod \"e2e-ipp-b-3e7ca1-openshift-default-7f74dcbfc6-f79fh\" (UID: \"ada380e3-be47-48af-9fde-9d422ac172ed\") " pod="openshift-ingress/e2e-ipp-b-3e7ca1-openshift-default-7f74dcbfc6-f79fh"
Sep 07 12:17:12.217207 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:17:12.216866    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ada380e3-be47-48af-9fde-9d422ac172ed-credential-socket\") pod \"e2e-ipp-b-3e7ca1-openshift-default-7f74dcbfc6-f79fh\" (UID: \"ada380e3-be47-48af-9fde-9d422ac172ed\") " pod="openshift-ingress/e2e-ipp-b-3e7ca1-openshift-default-7f74dcbfc6-f79fh"
Sep 07 12:17:12.217207 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:17:12.216883    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ada380e3-be47-48af-9fde-9d422ac172ed-istio-envoy\") pod \"e2e-ipp-b-3e7ca1-openshift-default-7f74dcbfc6-f79fh\" (UID: \"ada380e3-be47-48af-9fde-9d422ac172ed\") " pod="openshift-ingress/e2e-ipp-b-3e7ca1-openshift-default-7f74dcbfc6-f79fh"
Sep 07 12:17:12.217207 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:17:12.217005    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ada380e3-be47-48af-9fde-9d422ac172ed-istio-podinfo\") pod \"e2e-ipp-b-3e7ca1-openshift-default-7f74dcbfc6-f79fh\" (UID: \"ada380e3-be47-48af-9fde-9d422ac172ed\") " pod="openshift-ingress/e2e-ipp-b-3e7ca1-openshift-default-7f74dcbfc6-f79fh"
Sep 07 12:17:12.217207 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:17:12.217111    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ada380e3-be47-48af-9fde-9d422ac172ed-istiod-ca-cert\") pod \"e2e-ipp-b-3e7ca1-openshift-default-7f74dcbfc6-f79fh\" (UID: \"ada380e3-be47-48af-9fde-9d422ac172ed\") " pod="openshift-ingress/e2e-ipp-b-3e7ca1-openshift-default-7f74dcbfc6-f79fh"
Sep 07 12:17:12.217207 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:17:12.217151    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ada380e3-be47-48af-9fde-9d422ac172ed-workload-certs\") pod \"e2e-ipp-b-3e7ca1-openshift-default-7f74dcbfc6-f79fh\" (UID: \"ada380e3-be47-48af-9fde-9d422ac172ed\") " pod="openshift-ingress/e2e-ipp-b-3e7ca1-openshift-default-7f74dcbfc6-f79fh"
Sep 07 12:17:12.217207 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:17:12.217168    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ada380e3-be47-48af-9fde-9d422ac172ed-istio-data\") pod \"e2e-ipp-b-3e7ca1-openshift-default-7f74dcbfc6-f79fh\" (UID: \"ada380e3-be47-48af-9fde-9d422ac172ed\") " pod="openshift-ingress/e2e-ipp-b-3e7ca1-openshift-default-7f74dcbfc6-f79fh"
Sep 07 12:17:12.217207 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:17:12.217203    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ada380e3-be47-48af-9fde-9d422ac172ed-workload-socket\") pod \"e2e-ipp-b-3e7ca1-openshift-default-7f74dcbfc6-f79fh\" (UID: \"ada380e3-be47-48af-9fde-9d422ac172ed\") " pod="openshift-ingress/e2e-ipp-b-3e7ca1-openshift-default-7f74dcbfc6-f79fh"
Sep 07 12:17:12.217718 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:17:12.217246    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ada380e3-be47-48af-9fde-9d422ac172ed-credential-socket\") pod \"e2e-ipp-b-3e7ca1-openshift-default-7f74dcbfc6-f79fh\" (UID: \"ada380e3-be47-48af-9fde-9d422ac172ed\") " pod="openshift-ingress/e2e-ipp-b-3e7ca1-openshift-default-7f74dcbfc6-f79fh"
Sep 07 12:17:12.217718 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:17:12.217569    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ada380e3-be47-48af-9fde-9d422ac172ed-istio-data\") pod \"e2e-ipp-b-3e7ca1-openshift-default-7f74dcbfc6-f79fh\" (UID: \"ada380e3-be47-48af-9fde-9d422ac172ed\") " pod="openshift-ingress/e2e-ipp-b-3e7ca1-openshift-default-7f74dcbfc6-f79fh"
Sep 07 12:17:12.217796 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:17:12.217756    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ada380e3-be47-48af-9fde-9d422ac172ed-istiod-ca-cert\") pod \"e2e-ipp-b-3e7ca1-openshift-default-7f74dcbfc6-f79fh\" (UID: \"ada380e3-be47-48af-9fde-9d422ac172ed\") " pod="openshift-ingress/e2e-ipp-b-3e7ca1-openshift-default-7f74dcbfc6-f79fh"
Sep 07 12:17:12.219306 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:17:12.219285    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ada380e3-be47-48af-9fde-9d422ac172ed-istio-envoy\") pod \"e2e-ipp-b-3e7ca1-openshift-default-7f74dcbfc6-f79fh\" (UID: \"ada380e3-be47-48af-9fde-9d422ac172ed\") " pod="openshift-ingress/e2e-ipp-b-3e7ca1-openshift-default-7f74dcbfc6-f79fh"
Sep 07 12:17:12.219519 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:17:12.219501    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ada380e3-be47-48af-9fde-9d422ac172ed-istio-podinfo\") pod \"e2e-ipp-b-3e7ca1-openshift-default-7f74dcbfc6-f79fh\" (UID: \"ada380e3-be47-48af-9fde-9d422ac172ed\") " pod="openshift-ingress/e2e-ipp-b-3e7ca1-openshift-default-7f74dcbfc6-f79fh"
Sep 07 12:17:12.223906 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:17:12.223882    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ada380e3-be47-48af-9fde-9d422ac172ed-istio-token\") pod \"e2e-ipp-b-3e7ca1-openshift-default-7f74dcbfc6-f79fh\" (UID: \"ada380e3-be47-48af-9fde-9d422ac172ed\") " pod="openshift-ingress/e2e-ipp-b-3e7ca1-openshift-default-7f74dcbfc6-f79fh"
Sep 07 12:17:12.224362 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:17:12.224342    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gnmcz\" (UniqueName: \"kubernetes.io/projected/ada380e3-be47-48af-9fde-9d422ac172ed-kube-api-access-gnmcz\") pod \"e2e-ipp-b-3e7ca1-openshift-default-7f74dcbfc6-f79fh\" (UID: \"ada380e3-be47-48af-9fde-9d422ac172ed\") " pod="openshift-ingress/e2e-ipp-b-3e7ca1-openshift-default-7f74dcbfc6-f79fh"
Sep 07 12:17:12.321027 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:17:12.320945    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:17:12.355493 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:17:12.355464    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-3e7ca1-openshift-default-7f74dcbfc6-f79fh"
Sep 07 12:17:12.627262 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:17:12.627235    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-b-3e7ca1-openshift-default-7f74dcbfc6-f79fh"]
Sep 07 12:17:12.629242 ip-10-0-135-173 kubenswrapper[2563]: W0907 12:17:12.629219    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podada380e3_be47_48af_9fde_9d422ac172ed.slice/crio-1aa79100e8303b6cd2ec8a00c3e167a383feb99842dd8680cb7ceb84d5c6431e WatchSource:0}: Error finding container 1aa79100e8303b6cd2ec8a00c3e167a383feb99842dd8680cb7ceb84d5c6431e: Status 404 returned error can't find the container with id 1aa79100e8303b6cd2ec8a00c3e167a383feb99842dd8680cb7ceb84d5c6431e
Sep 07 12:17:12.631970 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:17:12.631940    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 07 12:17:12.632038 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:17:12.631998    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 07 12:17:12.632038 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:17:12.632031    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 07 12:17:13.461146 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:17:13.461107    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-3e7ca1-openshift-default-7f74dcbfc6-f79fh" event={"ID":"ada380e3-be47-48af-9fde-9d422ac172ed","Type":"ContainerStarted","Data":"012c5190c29ca3d7ef18f97592291166385bef3c634621f92ef1aecc25d9689d"}
Sep 07 12:17:13.461631 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:17:13.461186    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-3e7ca1-openshift-default-7f74dcbfc6-f79fh" event={"ID":"ada380e3-be47-48af-9fde-9d422ac172ed","Type":"ContainerStarted","Data":"1aa79100e8303b6cd2ec8a00c3e167a383feb99842dd8680cb7ceb84d5c6431e"}
Sep 07 12:17:13.481019 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:17:13.480968    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-b-3e7ca1-openshift-default-7f74dcbfc6-f79fh" podStartSLOduration=1.480951662 podStartE2EDuration="1.480951662s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 12:17:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 12:17:13.478330732 +0000 UTC m=+1804.997062545" watchObservedRunningTime="2026-09-07 12:17:13.480951662 +0000 UTC m=+1804.999683468"
Sep 07 12:17:14.356482 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:17:14.356447    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-b-3e7ca1-openshift-default-7f74dcbfc6-f79fh"
Sep 07 12:17:14.361180 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:17:14.361153    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-b-3e7ca1-openshift-default-7f74dcbfc6-f79fh"
Sep 07 12:17:14.466423 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:17:14.466378    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-b-3e7ca1-openshift-default-7f74dcbfc6-f79fh"
Sep 07 12:17:14.467315 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:17:14.467290    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-b-3e7ca1-openshift-default-7f74dcbfc6-f79fh"
Sep 07 12:17:15.340216 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:17:15.340165    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:17:15.439165 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:17:15.439125    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:17:16.511942 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:17:16.511908    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:17:16.612235 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:17:16.612198    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:17:16.805185 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:17:16.805097    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:17:17.090209 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:17:17.090125    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:17:17.191159 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:17:17.191126    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:17:21.606652 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:17:21.606615    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-p4l4w"]
Sep 07 12:17:30.508577 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:17:30.508534    2563 patch_prober.go:28] interesting pod/e2e-ipp-b-3e7ca1-openshift-default-7f74dcbfc6-f79fh container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.63:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Sep 07 12:17:30.508972 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:17:30.508599    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ipp-b-3e7ca1-openshift-default-7f74dcbfc6-f79fh" podUID="ada380e3-be47-48af-9fde-9d422ac172ed" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.63:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Sep 07 12:17:31.548095 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:17:31.548061    2563 generic.go:356] "Generic (PLEG): container finished" podID="ada380e3-be47-48af-9fde-9d422ac172ed" containerID="012c5190c29ca3d7ef18f97592291166385bef3c634621f92ef1aecc25d9689d" exitCode=0
Sep 07 12:17:31.548515 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:17:31.548133    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-3e7ca1-openshift-default-7f74dcbfc6-f79fh" event={"ID":"ada380e3-be47-48af-9fde-9d422ac172ed","Type":"ContainerDied","Data":"012c5190c29ca3d7ef18f97592291166385bef3c634621f92ef1aecc25d9689d"}
Sep 07 12:17:31.548565 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:17:31.548523    2563 scope.go:117] "RemoveContainer" containerID="012c5190c29ca3d7ef18f97592291166385bef3c634621f92ef1aecc25d9689d"
Sep 07 12:17:31.549600 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:17:31.549576    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 07 12:17:31.549653 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:17:31.549644    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 07 12:17:31.549690 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:17:31.549678    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 07 12:17:32.559820 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:17:32.559776    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-3e7ca1-openshift-default-7f74dcbfc6-f79fh" event={"ID":"ada380e3-be47-48af-9fde-9d422ac172ed","Type":"ContainerStarted","Data":"5dc5864fd2f0165bedfd7908ef41892b3c31f5206dad42bfd32ab0c8e18b3474"}
Sep 07 12:17:33.356875 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:17:33.356832    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-b-3e7ca1-openshift-default-7f74dcbfc6-f79fh"
Sep 07 12:17:33.357087 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:17:33.356989    2563 patch_prober.go:28] interesting pod/e2e-ipp-b-3e7ca1-openshift-default-7f74dcbfc6-f79fh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused" start-of-body=
Sep 07 12:17:33.357087 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:17:33.357034    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-3e7ca1-openshift-default-7f74dcbfc6-f79fh" podUID="ada380e3-be47-48af-9fde-9d422ac172ed" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused"
Sep 07 12:17:34.356856 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:17:34.356818    2563 patch_prober.go:28] interesting pod/e2e-ipp-b-3e7ca1-openshift-default-7f74dcbfc6-f79fh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused" start-of-body=
Sep 07 12:17:34.357464 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:17:34.357410    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-3e7ca1-openshift-default-7f74dcbfc6-f79fh" podUID="ada380e3-be47-48af-9fde-9d422ac172ed" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused"
Sep 07 12:17:35.356009 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:17:35.355968    2563 patch_prober.go:28] interesting pod/e2e-ipp-b-3e7ca1-openshift-default-7f74dcbfc6-f79fh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused" start-of-body=
Sep 07 12:17:35.356293 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:17:35.356263    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-3e7ca1-openshift-default-7f74dcbfc6-f79fh" podUID="ada380e3-be47-48af-9fde-9d422ac172ed" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused"
Sep 07 12:17:36.356682 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:17:36.356637    2563 patch_prober.go:28] interesting pod/e2e-ipp-b-3e7ca1-openshift-default-7f74dcbfc6-f79fh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused" start-of-body=
Sep 07 12:17:36.357195 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:17:36.356712    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-3e7ca1-openshift-default-7f74dcbfc6-f79fh" podUID="ada380e3-be47-48af-9fde-9d422ac172ed" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused"
Sep 07 12:17:37.356908 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:17:37.356858    2563 patch_prober.go:28] interesting pod/e2e-ipp-b-3e7ca1-openshift-default-7f74dcbfc6-f79fh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused" start-of-body=
Sep 07 12:17:37.357394 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:17:37.356934    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-3e7ca1-openshift-default-7f74dcbfc6-f79fh" podUID="ada380e3-be47-48af-9fde-9d422ac172ed" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused"
Sep 07 12:17:38.359453 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:17:38.359411    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-b-3e7ca1-openshift-default-7f74dcbfc6-f79fh"
Sep 07 12:17:38.359827 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:17:38.359717    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-b-3e7ca1-openshift-default-7f74dcbfc6-f79fh"
Sep 07 12:17:38.360306 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:17:38.360289    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-b-3e7ca1-openshift-default-7f74dcbfc6-f79fh"
Sep 07 12:17:40.304995 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:17:40.304958    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:17:56.405478 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:17:56.405438    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-ipp-a-aab734/ipp-route-aab734-kserve-6fb79867c5-hcfd8"]
Sep 07 12:17:56.409025 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:17:56.409008    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-aab734/ipp-route-aab734-kserve-6fb79867c5-hcfd8"
Sep 07 12:17:56.410750 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:17:56.410726    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-ipp-a-aab734\"/\"kube-root-ca.crt\""
Sep 07 12:17:56.410900 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:17:56.410768    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-ipp-a-aab734\"/\"default-dockercfg-2xt8v\""
Sep 07 12:17:56.411133 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:17:56.411115    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-ipp-a-aab734\"/\"ipp-route-aab734-kserve-self-signed-certs\""
Sep 07 12:17:56.411231 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:17:56.411131    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-ipp-a-aab734\"/\"openshift-service-ca.crt\""
Sep 07 12:17:56.416020 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:17:56.415997    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-ipp-a-aab734/ipp-route-aab734-kserve-6fb79867c5-hcfd8"]
Sep 07 12:17:56.438773 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:17:56.438743    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fd792abf-4178-43c0-b4f0-94df7367bff3-dshm\") pod \"ipp-route-aab734-kserve-6fb79867c5-hcfd8\" (UID: \"fd792abf-4178-43c0-b4f0-94df7367bff3\") " pod="ai-tenant-e2e-ipp-a-aab734/ipp-route-aab734-kserve-6fb79867c5-hcfd8"
Sep 07 12:17:56.438874 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:17:56.438791    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fd792abf-4178-43c0-b4f0-94df7367bff3-home\") pod \"ipp-route-aab734-kserve-6fb79867c5-hcfd8\" (UID: \"fd792abf-4178-43c0-b4f0-94df7367bff3\") " pod="ai-tenant-e2e-ipp-a-aab734/ipp-route-aab734-kserve-6fb79867c5-hcfd8"
Sep 07 12:17:56.438934 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:17:56.438896    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pqpxt\" (UniqueName: \"kubernetes.io/projected/fd792abf-4178-43c0-b4f0-94df7367bff3-kube-api-access-pqpxt\") pod \"ipp-route-aab734-kserve-6fb79867c5-hcfd8\" (UID: \"fd792abf-4178-43c0-b4f0-94df7367bff3\") " pod="ai-tenant-e2e-ipp-a-aab734/ipp-route-aab734-kserve-6fb79867c5-hcfd8"
Sep 07 12:17:56.438989 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:17:56.438972    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fd792abf-4178-43c0-b4f0-94df7367bff3-model-cache\") pod \"ipp-route-aab734-kserve-6fb79867c5-hcfd8\" (UID: \"fd792abf-4178-43c0-b4f0-94df7367bff3\") " pod="ai-tenant-e2e-ipp-a-aab734/ipp-route-aab734-kserve-6fb79867c5-hcfd8"
Sep 07 12:17:56.439031 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:17:56.439009    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fd792abf-4178-43c0-b4f0-94df7367bff3-tmp-dir\") pod \"ipp-route-aab734-kserve-6fb79867c5-hcfd8\" (UID: \"fd792abf-4178-43c0-b4f0-94df7367bff3\") " pod="ai-tenant-e2e-ipp-a-aab734/ipp-route-aab734-kserve-6fb79867c5-hcfd8"
Sep 07 12:17:56.439075 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:17:56.439043    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fd792abf-4178-43c0-b4f0-94df7367bff3-tls-certs\") pod \"ipp-route-aab734-kserve-6fb79867c5-hcfd8\" (UID: \"fd792abf-4178-43c0-b4f0-94df7367bff3\") " pod="ai-tenant-e2e-ipp-a-aab734/ipp-route-aab734-kserve-6fb79867c5-hcfd8"
Sep 07 12:17:56.510290 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:17:56.510253    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:17:56.539797 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:17:56.539772    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fd792abf-4178-43c0-b4f0-94df7367bff3-dshm\") pod \"ipp-route-aab734-kserve-6fb79867c5-hcfd8\" (UID: \"fd792abf-4178-43c0-b4f0-94df7367bff3\") " pod="ai-tenant-e2e-ipp-a-aab734/ipp-route-aab734-kserve-6fb79867c5-hcfd8"
Sep 07 12:17:56.539897 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:17:56.539818    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fd792abf-4178-43c0-b4f0-94df7367bff3-home\") pod \"ipp-route-aab734-kserve-6fb79867c5-hcfd8\" (UID: \"fd792abf-4178-43c0-b4f0-94df7367bff3\") " pod="ai-tenant-e2e-ipp-a-aab734/ipp-route-aab734-kserve-6fb79867c5-hcfd8"
Sep 07 12:17:56.539897 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:17:56.539865    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pqpxt\" (UniqueName: \"kubernetes.io/projected/fd792abf-4178-43c0-b4f0-94df7367bff3-kube-api-access-pqpxt\") pod \"ipp-route-aab734-kserve-6fb79867c5-hcfd8\" (UID: \"fd792abf-4178-43c0-b4f0-94df7367bff3\") " pod="ai-tenant-e2e-ipp-a-aab734/ipp-route-aab734-kserve-6fb79867c5-hcfd8"
Sep 07 12:17:56.539998 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:17:56.539967    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fd792abf-4178-43c0-b4f0-94df7367bff3-model-cache\") pod \"ipp-route-aab734-kserve-6fb79867c5-hcfd8\" (UID: \"fd792abf-4178-43c0-b4f0-94df7367bff3\") " pod="ai-tenant-e2e-ipp-a-aab734/ipp-route-aab734-kserve-6fb79867c5-hcfd8"
Sep 07 12:17:56.540060 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:17:56.540018    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fd792abf-4178-43c0-b4f0-94df7367bff3-tmp-dir\") pod \"ipp-route-aab734-kserve-6fb79867c5-hcfd8\" (UID: \"fd792abf-4178-43c0-b4f0-94df7367bff3\") " pod="ai-tenant-e2e-ipp-a-aab734/ipp-route-aab734-kserve-6fb79867c5-hcfd8"
Sep 07 12:17:56.540113 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:17:56.540056    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fd792abf-4178-43c0-b4f0-94df7367bff3-tls-certs\") pod \"ipp-route-aab734-kserve-6fb79867c5-hcfd8\" (UID: \"fd792abf-4178-43c0-b4f0-94df7367bff3\") " pod="ai-tenant-e2e-ipp-a-aab734/ipp-route-aab734-kserve-6fb79867c5-hcfd8"
Sep 07 12:17:56.540240 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:17:56.540220    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fd792abf-4178-43c0-b4f0-94df7367bff3-home\") pod \"ipp-route-aab734-kserve-6fb79867c5-hcfd8\" (UID: \"fd792abf-4178-43c0-b4f0-94df7367bff3\") " pod="ai-tenant-e2e-ipp-a-aab734/ipp-route-aab734-kserve-6fb79867c5-hcfd8"
Sep 07 12:17:56.540310 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:17:56.540269    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fd792abf-4178-43c0-b4f0-94df7367bff3-model-cache\") pod \"ipp-route-aab734-kserve-6fb79867c5-hcfd8\" (UID: \"fd792abf-4178-43c0-b4f0-94df7367bff3\") " pod="ai-tenant-e2e-ipp-a-aab734/ipp-route-aab734-kserve-6fb79867c5-hcfd8"
Sep 07 12:17:56.540368 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:17:56.540348    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fd792abf-4178-43c0-b4f0-94df7367bff3-tmp-dir\") pod \"ipp-route-aab734-kserve-6fb79867c5-hcfd8\" (UID: \"fd792abf-4178-43c0-b4f0-94df7367bff3\") " pod="ai-tenant-e2e-ipp-a-aab734/ipp-route-aab734-kserve-6fb79867c5-hcfd8"
Sep 07 12:17:56.542023 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:17:56.541997    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fd792abf-4178-43c0-b4f0-94df7367bff3-dshm\") pod \"ipp-route-aab734-kserve-6fb79867c5-hcfd8\" (UID: \"fd792abf-4178-43c0-b4f0-94df7367bff3\") " pod="ai-tenant-e2e-ipp-a-aab734/ipp-route-aab734-kserve-6fb79867c5-hcfd8"
Sep 07 12:17:56.542395 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:17:56.542378    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fd792abf-4178-43c0-b4f0-94df7367bff3-tls-certs\") pod \"ipp-route-aab734-kserve-6fb79867c5-hcfd8\" (UID: \"fd792abf-4178-43c0-b4f0-94df7367bff3\") " pod="ai-tenant-e2e-ipp-a-aab734/ipp-route-aab734-kserve-6fb79867c5-hcfd8"
Sep 07 12:17:56.546510 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:17:56.546493    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pqpxt\" (UniqueName: \"kubernetes.io/projected/fd792abf-4178-43c0-b4f0-94df7367bff3-kube-api-access-pqpxt\") pod \"ipp-route-aab734-kserve-6fb79867c5-hcfd8\" (UID: \"fd792abf-4178-43c0-b4f0-94df7367bff3\") " pod="ai-tenant-e2e-ipp-a-aab734/ipp-route-aab734-kserve-6fb79867c5-hcfd8"
Sep 07 12:17:56.719334 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:17:56.719304    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-aab734/ipp-route-aab734-kserve-6fb79867c5-hcfd8"
Sep 07 12:17:56.995030 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:17:56.995004    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-ipp-a-aab734/ipp-route-aab734-kserve-6fb79867c5-hcfd8"]
Sep 07 12:17:56.997196 ip-10-0-135-173 kubenswrapper[2563]: W0907 12:17:56.997171    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podfd792abf_4178_43c0_b4f0_94df7367bff3.slice/crio-66869743fb5a8ae190c3e97b4da21d6d332f1b7ebe9ca7787d6f8552b41d5537 WatchSource:0}: Error finding container 66869743fb5a8ae190c3e97b4da21d6d332f1b7ebe9ca7787d6f8552b41d5537: Status 404 returned error can't find the container with id 66869743fb5a8ae190c3e97b4da21d6d332f1b7ebe9ca7787d6f8552b41d5537
Sep 07 12:17:57.700228 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:17:57.700193    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-aab734/ipp-route-aab734-kserve-6fb79867c5-hcfd8" event={"ID":"fd792abf-4178-43c0-b4f0-94df7367bff3","Type":"ContainerStarted","Data":"ef9691c0ab2332d8c969e5786ebccbff335a299b0091044040fa219b20e33bc9"}
Sep 07 12:17:57.700726 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:17:57.700238    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-aab734/ipp-route-aab734-kserve-6fb79867c5-hcfd8" event={"ID":"fd792abf-4178-43c0-b4f0-94df7367bff3","Type":"ContainerStarted","Data":"66869743fb5a8ae190c3e97b4da21d6d332f1b7ebe9ca7787d6f8552b41d5537"}
Sep 07 12:17:57.724835 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:17:57.724787    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-ipp-a-aab734/ipp-route-aab734-kserve-6fb79867c5-hcfd8" podStartSLOduration=1.724771993 podStartE2EDuration="1.724771993s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 12:17:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 12:17:57.72336417 +0000 UTC m=+1849.242095982" watchObservedRunningTime="2026-09-07 12:17:57.724771993 +0000 UTC m=+1849.243503843"
Sep 07 12:18:00.310523 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:18:00.310476    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-p4l4w"]
Sep 07 12:18:06.719619 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:18:06.719587    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-ipp-a-aab734/ipp-route-aab734-kserve-6fb79867c5-hcfd8"
Sep 07 12:18:06.720101 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:18:06.719665    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-ipp-a-aab734/ipp-route-aab734-kserve-6fb79867c5-hcfd8"
Sep 07 12:18:06.732187 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:18:06.732159    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-ipp-a-aab734/ipp-route-aab734-kserve-6fb79867c5-hcfd8"
Sep 07 12:18:06.763364 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:18:06.763344    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-ipp-a-aab734/ipp-route-aab734-kserve-6fb79867c5-hcfd8"
Sep 07 12:18:16.505441 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:18:16.501252    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-p4l4w"]
Sep 07 12:18:23.495884 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:18:23.495845    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:18:23.769757 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:18:23.769681    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-845d36-openshift-default-64d7978b84-kkzds"]
Sep 07 12:18:23.773976 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:18:23.773950    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-845d36-openshift-default-64d7978b84-kkzds"
Sep 07 12:18:23.775821 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:18:23.775797    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-cleanup-845d36-openshift-default-dockercfg-4ljtp\""
Sep 07 12:18:23.782916 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:18:23.782892    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-845d36-openshift-default-64d7978b84-kkzds"]
Sep 07 12:18:23.805021 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:18:23.804993    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:18:23.888060 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:18:23.888030    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f2486ed1-45b8-49ec-ad3c-82e6094aeee4-istio-token\") pod \"e2e-ipp-cleanup-845d36-openshift-default-64d7978b84-kkzds\" (UID: \"f2486ed1-45b8-49ec-ad3c-82e6094aeee4\") " pod="openshift-ingress/e2e-ipp-cleanup-845d36-openshift-default-64d7978b84-kkzds"
Sep 07 12:18:23.888192 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:18:23.888066    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f2486ed1-45b8-49ec-ad3c-82e6094aeee4-workload-socket\") pod \"e2e-ipp-cleanup-845d36-openshift-default-64d7978b84-kkzds\" (UID: \"f2486ed1-45b8-49ec-ad3c-82e6094aeee4\") " pod="openshift-ingress/e2e-ipp-cleanup-845d36-openshift-default-64d7978b84-kkzds"
Sep 07 12:18:23.888192 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:18:23.888112    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f2486ed1-45b8-49ec-ad3c-82e6094aeee4-istio-podinfo\") pod \"e2e-ipp-cleanup-845d36-openshift-default-64d7978b84-kkzds\" (UID: \"f2486ed1-45b8-49ec-ad3c-82e6094aeee4\") " pod="openshift-ingress/e2e-ipp-cleanup-845d36-openshift-default-64d7978b84-kkzds"
Sep 07 12:18:23.888192 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:18:23.888163    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f2486ed1-45b8-49ec-ad3c-82e6094aeee4-credential-socket\") pod \"e2e-ipp-cleanup-845d36-openshift-default-64d7978b84-kkzds\" (UID: \"f2486ed1-45b8-49ec-ad3c-82e6094aeee4\") " pod="openshift-ingress/e2e-ipp-cleanup-845d36-openshift-default-64d7978b84-kkzds"
Sep 07 12:18:23.888331 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:18:23.888204    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f2486ed1-45b8-49ec-ad3c-82e6094aeee4-workload-certs\") pod \"e2e-ipp-cleanup-845d36-openshift-default-64d7978b84-kkzds\" (UID: \"f2486ed1-45b8-49ec-ad3c-82e6094aeee4\") " pod="openshift-ingress/e2e-ipp-cleanup-845d36-openshift-default-64d7978b84-kkzds"
Sep 07 12:18:23.888331 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:18:23.888251    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f2486ed1-45b8-49ec-ad3c-82e6094aeee4-istio-envoy\") pod \"e2e-ipp-cleanup-845d36-openshift-default-64d7978b84-kkzds\" (UID: \"f2486ed1-45b8-49ec-ad3c-82e6094aeee4\") " pod="openshift-ingress/e2e-ipp-cleanup-845d36-openshift-default-64d7978b84-kkzds"
Sep 07 12:18:23.888331 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:18:23.888271    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f2486ed1-45b8-49ec-ad3c-82e6094aeee4-istiod-ca-cert\") pod \"e2e-ipp-cleanup-845d36-openshift-default-64d7978b84-kkzds\" (UID: \"f2486ed1-45b8-49ec-ad3c-82e6094aeee4\") " pod="openshift-ingress/e2e-ipp-cleanup-845d36-openshift-default-64d7978b84-kkzds"
Sep 07 12:18:23.888331 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:18:23.888325    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f2486ed1-45b8-49ec-ad3c-82e6094aeee4-istio-data\") pod \"e2e-ipp-cleanup-845d36-openshift-default-64d7978b84-kkzds\" (UID: \"f2486ed1-45b8-49ec-ad3c-82e6094aeee4\") " pod="openshift-ingress/e2e-ipp-cleanup-845d36-openshift-default-64d7978b84-kkzds"
Sep 07 12:18:23.888521 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:18:23.888354    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2fmsh\" (UniqueName: \"kubernetes.io/projected/f2486ed1-45b8-49ec-ad3c-82e6094aeee4-kube-api-access-2fmsh\") pod \"e2e-ipp-cleanup-845d36-openshift-default-64d7978b84-kkzds\" (UID: \"f2486ed1-45b8-49ec-ad3c-82e6094aeee4\") " pod="openshift-ingress/e2e-ipp-cleanup-845d36-openshift-default-64d7978b84-kkzds"
Sep 07 12:18:23.897597 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:18:23.897554    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:18:23.989315 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:18:23.989283    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f2486ed1-45b8-49ec-ad3c-82e6094aeee4-credential-socket\") pod \"e2e-ipp-cleanup-845d36-openshift-default-64d7978b84-kkzds\" (UID: \"f2486ed1-45b8-49ec-ad3c-82e6094aeee4\") " pod="openshift-ingress/e2e-ipp-cleanup-845d36-openshift-default-64d7978b84-kkzds"
Sep 07 12:18:23.989315 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:18:23.989316    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f2486ed1-45b8-49ec-ad3c-82e6094aeee4-workload-certs\") pod \"e2e-ipp-cleanup-845d36-openshift-default-64d7978b84-kkzds\" (UID: \"f2486ed1-45b8-49ec-ad3c-82e6094aeee4\") " pod="openshift-ingress/e2e-ipp-cleanup-845d36-openshift-default-64d7978b84-kkzds"
Sep 07 12:18:23.989581 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:18:23.989339    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f2486ed1-45b8-49ec-ad3c-82e6094aeee4-istio-envoy\") pod \"e2e-ipp-cleanup-845d36-openshift-default-64d7978b84-kkzds\" (UID: \"f2486ed1-45b8-49ec-ad3c-82e6094aeee4\") " pod="openshift-ingress/e2e-ipp-cleanup-845d36-openshift-default-64d7978b84-kkzds"
Sep 07 12:18:23.989581 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:18:23.989471    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f2486ed1-45b8-49ec-ad3c-82e6094aeee4-istiod-ca-cert\") pod \"e2e-ipp-cleanup-845d36-openshift-default-64d7978b84-kkzds\" (UID: \"f2486ed1-45b8-49ec-ad3c-82e6094aeee4\") " pod="openshift-ingress/e2e-ipp-cleanup-845d36-openshift-default-64d7978b84-kkzds"
Sep 07 12:18:23.989581 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:18:23.989560    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f2486ed1-45b8-49ec-ad3c-82e6094aeee4-istio-data\") pod \"e2e-ipp-cleanup-845d36-openshift-default-64d7978b84-kkzds\" (UID: \"f2486ed1-45b8-49ec-ad3c-82e6094aeee4\") " pod="openshift-ingress/e2e-ipp-cleanup-845d36-openshift-default-64d7978b84-kkzds"
Sep 07 12:18:23.989746 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:18:23.989604    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2fmsh\" (UniqueName: \"kubernetes.io/projected/f2486ed1-45b8-49ec-ad3c-82e6094aeee4-kube-api-access-2fmsh\") pod \"e2e-ipp-cleanup-845d36-openshift-default-64d7978b84-kkzds\" (UID: \"f2486ed1-45b8-49ec-ad3c-82e6094aeee4\") " pod="openshift-ingress/e2e-ipp-cleanup-845d36-openshift-default-64d7978b84-kkzds"
Sep 07 12:18:23.989746 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:18:23.989700    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f2486ed1-45b8-49ec-ad3c-82e6094aeee4-istio-token\") pod \"e2e-ipp-cleanup-845d36-openshift-default-64d7978b84-kkzds\" (UID: \"f2486ed1-45b8-49ec-ad3c-82e6094aeee4\") " pod="openshift-ingress/e2e-ipp-cleanup-845d36-openshift-default-64d7978b84-kkzds"
Sep 07 12:18:23.989746 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:18:23.989732    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f2486ed1-45b8-49ec-ad3c-82e6094aeee4-workload-socket\") pod \"e2e-ipp-cleanup-845d36-openshift-default-64d7978b84-kkzds\" (UID: \"f2486ed1-45b8-49ec-ad3c-82e6094aeee4\") " pod="openshift-ingress/e2e-ipp-cleanup-845d36-openshift-default-64d7978b84-kkzds"
Sep 07 12:18:23.989912 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:18:23.989790    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f2486ed1-45b8-49ec-ad3c-82e6094aeee4-workload-certs\") pod \"e2e-ipp-cleanup-845d36-openshift-default-64d7978b84-kkzds\" (UID: \"f2486ed1-45b8-49ec-ad3c-82e6094aeee4\") " pod="openshift-ingress/e2e-ipp-cleanup-845d36-openshift-default-64d7978b84-kkzds"
Sep 07 12:18:23.989912 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:18:23.989814    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f2486ed1-45b8-49ec-ad3c-82e6094aeee4-credential-socket\") pod \"e2e-ipp-cleanup-845d36-openshift-default-64d7978b84-kkzds\" (UID: \"f2486ed1-45b8-49ec-ad3c-82e6094aeee4\") " pod="openshift-ingress/e2e-ipp-cleanup-845d36-openshift-default-64d7978b84-kkzds"
Sep 07 12:18:23.989912 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:18:23.989908    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f2486ed1-45b8-49ec-ad3c-82e6094aeee4-istio-podinfo\") pod \"e2e-ipp-cleanup-845d36-openshift-default-64d7978b84-kkzds\" (UID: \"f2486ed1-45b8-49ec-ad3c-82e6094aeee4\") " pod="openshift-ingress/e2e-ipp-cleanup-845d36-openshift-default-64d7978b84-kkzds"
Sep 07 12:18:23.990076 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:18:23.989942    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f2486ed1-45b8-49ec-ad3c-82e6094aeee4-istio-data\") pod \"e2e-ipp-cleanup-845d36-openshift-default-64d7978b84-kkzds\" (UID: \"f2486ed1-45b8-49ec-ad3c-82e6094aeee4\") " pod="openshift-ingress/e2e-ipp-cleanup-845d36-openshift-default-64d7978b84-kkzds"
Sep 07 12:18:23.990076 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:18:23.990053    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f2486ed1-45b8-49ec-ad3c-82e6094aeee4-workload-socket\") pod \"e2e-ipp-cleanup-845d36-openshift-default-64d7978b84-kkzds\" (UID: \"f2486ed1-45b8-49ec-ad3c-82e6094aeee4\") " pod="openshift-ingress/e2e-ipp-cleanup-845d36-openshift-default-64d7978b84-kkzds"
Sep 07 12:18:23.990214 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:18:23.990192    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f2486ed1-45b8-49ec-ad3c-82e6094aeee4-istiod-ca-cert\") pod \"e2e-ipp-cleanup-845d36-openshift-default-64d7978b84-kkzds\" (UID: \"f2486ed1-45b8-49ec-ad3c-82e6094aeee4\") " pod="openshift-ingress/e2e-ipp-cleanup-845d36-openshift-default-64d7978b84-kkzds"
Sep 07 12:18:23.991817 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:18:23.991796    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f2486ed1-45b8-49ec-ad3c-82e6094aeee4-istio-envoy\") pod \"e2e-ipp-cleanup-845d36-openshift-default-64d7978b84-kkzds\" (UID: \"f2486ed1-45b8-49ec-ad3c-82e6094aeee4\") " pod="openshift-ingress/e2e-ipp-cleanup-845d36-openshift-default-64d7978b84-kkzds"
Sep 07 12:18:23.992123 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:18:23.992107    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f2486ed1-45b8-49ec-ad3c-82e6094aeee4-istio-podinfo\") pod \"e2e-ipp-cleanup-845d36-openshift-default-64d7978b84-kkzds\" (UID: \"f2486ed1-45b8-49ec-ad3c-82e6094aeee4\") " pod="openshift-ingress/e2e-ipp-cleanup-845d36-openshift-default-64d7978b84-kkzds"
Sep 07 12:18:23.998317 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:18:23.998292    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f2486ed1-45b8-49ec-ad3c-82e6094aeee4-istio-token\") pod \"e2e-ipp-cleanup-845d36-openshift-default-64d7978b84-kkzds\" (UID: \"f2486ed1-45b8-49ec-ad3c-82e6094aeee4\") " pod="openshift-ingress/e2e-ipp-cleanup-845d36-openshift-default-64d7978b84-kkzds"
Sep 07 12:18:23.998990 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:18:23.998972    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2fmsh\" (UniqueName: \"kubernetes.io/projected/f2486ed1-45b8-49ec-ad3c-82e6094aeee4-kube-api-access-2fmsh\") pod \"e2e-ipp-cleanup-845d36-openshift-default-64d7978b84-kkzds\" (UID: \"f2486ed1-45b8-49ec-ad3c-82e6094aeee4\") " pod="openshift-ingress/e2e-ipp-cleanup-845d36-openshift-default-64d7978b84-kkzds"
Sep 07 12:18:24.085335 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:18:24.085277    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-845d36-openshift-default-64d7978b84-kkzds"
Sep 07 12:18:24.358950 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:18:24.358923    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-845d36-openshift-default-64d7978b84-kkzds"]
Sep 07 12:18:24.360593 ip-10-0-135-173 kubenswrapper[2563]: W0907 12:18:24.360561    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf2486ed1_45b8_49ec_ad3c_82e6094aeee4.slice/crio-03802548ab09a1d6cd07b9a3d452f57e7131d1447b2fe112015d27f682c7ea09 WatchSource:0}: Error finding container 03802548ab09a1d6cd07b9a3d452f57e7131d1447b2fe112015d27f682c7ea09: Status 404 returned error can't find the container with id 03802548ab09a1d6cd07b9a3d452f57e7131d1447b2fe112015d27f682c7ea09
Sep 07 12:18:24.362887 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:18:24.362846    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 07 12:18:24.362970 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:18:24.362953    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 07 12:18:24.363011 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:18:24.362999    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 07 12:18:24.842705 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:18:24.842671    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-845d36-openshift-default-64d7978b84-kkzds" event={"ID":"f2486ed1-45b8-49ec-ad3c-82e6094aeee4","Type":"ContainerStarted","Data":"a6ca9c841b8cdf2180a3474ec48e9fbb6e26f8ec6029d18b825844dda76d9359"}
Sep 07 12:18:24.843086 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:18:24.842722    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-845d36-openshift-default-64d7978b84-kkzds" event={"ID":"f2486ed1-45b8-49ec-ad3c-82e6094aeee4","Type":"ContainerStarted","Data":"03802548ab09a1d6cd07b9a3d452f57e7131d1447b2fe112015d27f682c7ea09"}
Sep 07 12:18:24.861681 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:18:24.861633    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-cleanup-845d36-openshift-default-64d7978b84-kkzds" podStartSLOduration=1.861617458 podStartE2EDuration="1.861617458s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 12:18:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 12:18:24.859117922 +0000 UTC m=+1876.377849748" watchObservedRunningTime="2026-09-07 12:18:24.861617458 +0000 UTC m=+1876.380349270"
Sep 07 12:18:25.086232 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:18:25.086201    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-cleanup-845d36-openshift-default-64d7978b84-kkzds"
Sep 07 12:18:25.091728 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:18:25.091703    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-cleanup-845d36-openshift-default-64d7978b84-kkzds"
Sep 07 12:18:25.847429 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:18:25.847396    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-cleanup-845d36-openshift-default-64d7978b84-kkzds"
Sep 07 12:18:25.848383 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:18:25.848367    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-cleanup-845d36-openshift-default-64d7978b84-kkzds"
Sep 07 12:18:26.745076 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:18:26.745043    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:18:26.839938 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:18:26.839906    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:18:27.857103 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:18:27.857071    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-845d36-69c46b5f8d-2kjgn"]
Sep 07 12:18:27.861031 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:18:27.861011    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-845d36-69c46b5f8d-2kjgn"
Sep 07 12:18:27.862820 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:18:27.862743    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-cleanup-845d36-serving-cert\""
Sep 07 12:18:27.872206 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:18:27.871566    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-845d36-69c46b5f8d-2kjgn"]
Sep 07 12:18:27.908169 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:18:27.908130    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:18:28.007001 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:18:28.006962    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:18:28.030812 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:18:28.030783    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5f741d59-c73c-48b8-a6d5-887ee0467484-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-845d36-69c46b5f8d-2kjgn\" (UID: \"5f741d59-c73c-48b8-a6d5-887ee0467484\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-845d36-69c46b5f8d-2kjgn"
Sep 07 12:18:28.030962 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:18:28.030945    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5f741d59-c73c-48b8-a6d5-887ee0467484-metrics-certs\") pod \"maas-api-e2e-ipp-cleanup-845d36-69c46b5f8d-2kjgn\" (UID: \"5f741d59-c73c-48b8-a6d5-887ee0467484\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-845d36-69c46b5f8d-2kjgn"
Sep 07 12:18:28.031029 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:18:28.030981    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vcrml\" (UniqueName: \"kubernetes.io/projected/5f741d59-c73c-48b8-a6d5-887ee0467484-kube-api-access-vcrml\") pod \"maas-api-e2e-ipp-cleanup-845d36-69c46b5f8d-2kjgn\" (UID: \"5f741d59-c73c-48b8-a6d5-887ee0467484\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-845d36-69c46b5f8d-2kjgn"
Sep 07 12:18:28.031098 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:18:28.031043    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5f741d59-c73c-48b8-a6d5-887ee0467484-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-845d36-69c46b5f8d-2kjgn\" (UID: \"5f741d59-c73c-48b8-a6d5-887ee0467484\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-845d36-69c46b5f8d-2kjgn"
Sep 07 12:18:28.132064 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:18:28.132001    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5f741d59-c73c-48b8-a6d5-887ee0467484-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-845d36-69c46b5f8d-2kjgn\" (UID: \"5f741d59-c73c-48b8-a6d5-887ee0467484\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-845d36-69c46b5f8d-2kjgn"
Sep 07 12:18:28.132175 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:18:28.132098    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5f741d59-c73c-48b8-a6d5-887ee0467484-metrics-certs\") pod \"maas-api-e2e-ipp-cleanup-845d36-69c46b5f8d-2kjgn\" (UID: \"5f741d59-c73c-48b8-a6d5-887ee0467484\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-845d36-69c46b5f8d-2kjgn"
Sep 07 12:18:28.132175 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:18:28.132132    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vcrml\" (UniqueName: \"kubernetes.io/projected/5f741d59-c73c-48b8-a6d5-887ee0467484-kube-api-access-vcrml\") pod \"maas-api-e2e-ipp-cleanup-845d36-69c46b5f8d-2kjgn\" (UID: \"5f741d59-c73c-48b8-a6d5-887ee0467484\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-845d36-69c46b5f8d-2kjgn"
Sep 07 12:18:28.132175 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:18:28.132165    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5f741d59-c73c-48b8-a6d5-887ee0467484-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-845d36-69c46b5f8d-2kjgn\" (UID: \"5f741d59-c73c-48b8-a6d5-887ee0467484\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-845d36-69c46b5f8d-2kjgn"
Sep 07 12:18:28.132320 ip-10-0-135-173 kubenswrapper[2563]: E0907 12:18:28.132303    2563 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-845d36-serving-cert: secret "maas-api-e2e-ipp-cleanup-845d36-serving-cert" not found
Sep 07 12:18:28.132392 ip-10-0-135-173 kubenswrapper[2563]: E0907 12:18:28.132382    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5f741d59-c73c-48b8-a6d5-887ee0467484-maas-api-tls podName:5f741d59-c73c-48b8-a6d5-887ee0467484 nodeName:}" failed. No retries permitted until 2026-09-07 12:18:28.632361959 +0000 UTC m=+1880.151093751 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/5f741d59-c73c-48b8-a6d5-887ee0467484-maas-api-tls") pod "maas-api-e2e-ipp-cleanup-845d36-69c46b5f8d-2kjgn" (UID: "5f741d59-c73c-48b8-a6d5-887ee0467484") : secret "maas-api-e2e-ipp-cleanup-845d36-serving-cert" not found
Sep 07 12:18:28.132788 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:18:28.132761    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5f741d59-c73c-48b8-a6d5-887ee0467484-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-845d36-69c46b5f8d-2kjgn\" (UID: \"5f741d59-c73c-48b8-a6d5-887ee0467484\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-845d36-69c46b5f8d-2kjgn"
Sep 07 12:18:28.134639 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:18:28.134614    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5f741d59-c73c-48b8-a6d5-887ee0467484-metrics-certs\") pod \"maas-api-e2e-ipp-cleanup-845d36-69c46b5f8d-2kjgn\" (UID: \"5f741d59-c73c-48b8-a6d5-887ee0467484\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-845d36-69c46b5f8d-2kjgn"
Sep 07 12:18:28.140160 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:18:28.140143    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vcrml\" (UniqueName: \"kubernetes.io/projected/5f741d59-c73c-48b8-a6d5-887ee0467484-kube-api-access-vcrml\") pod \"maas-api-e2e-ipp-cleanup-845d36-69c46b5f8d-2kjgn\" (UID: \"5f741d59-c73c-48b8-a6d5-887ee0467484\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-845d36-69c46b5f8d-2kjgn"
Sep 07 12:18:28.386226 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:18:28.386140    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:18:28.494483 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:18:28.494441    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:18:28.589327 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:18:28.589288    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:18:28.636660 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:18:28.636588    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5f741d59-c73c-48b8-a6d5-887ee0467484-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-845d36-69c46b5f8d-2kjgn\" (UID: \"5f741d59-c73c-48b8-a6d5-887ee0467484\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-845d36-69c46b5f8d-2kjgn"
Sep 07 12:18:28.639038 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:18:28.639017    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5f741d59-c73c-48b8-a6d5-887ee0467484-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-845d36-69c46b5f8d-2kjgn\" (UID: \"5f741d59-c73c-48b8-a6d5-887ee0467484\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-845d36-69c46b5f8d-2kjgn"
Sep 07 12:18:28.775905 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:18:28.775877    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-845d36-69c46b5f8d-2kjgn"
Sep 07 12:18:29.046181 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:18:29.046156    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-845d36-69c46b5f8d-2kjgn"]
Sep 07 12:18:29.048268 ip-10-0-135-173 kubenswrapper[2563]: W0907 12:18:29.048242    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5f741d59_c73c_48b8_a6d5_887ee0467484.slice/crio-1a492c2fe98f21f19191e1015db231f6bc830d9f1d50a1b9fbaeeec8f4f6bff4 WatchSource:0}: Error finding container 1a492c2fe98f21f19191e1015db231f6bc830d9f1d50a1b9fbaeeec8f4f6bff4: Status 404 returned error can't find the container with id 1a492c2fe98f21f19191e1015db231f6bc830d9f1d50a1b9fbaeeec8f4f6bff4
Sep 07 12:18:29.873008 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:18:29.872912    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-845d36-69c46b5f8d-2kjgn" event={"ID":"5f741d59-c73c-48b8-a6d5-887ee0467484","Type":"ContainerStarted","Data":"24c0944c552c41ae4c4ef23848f104a828d1b1c30fa2a614573dd5e6970971dd"}
Sep 07 12:18:29.873008 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:18:29.872971    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-845d36-69c46b5f8d-2kjgn" event={"ID":"5f741d59-c73c-48b8-a6d5-887ee0467484","Type":"ContainerStarted","Data":"1a492c2fe98f21f19191e1015db231f6bc830d9f1d50a1b9fbaeeec8f4f6bff4"}
Sep 07 12:18:29.873199 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:18:29.873013    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-845d36-69c46b5f8d-2kjgn"
Sep 07 12:18:29.887806 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:18:29.887761    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-845d36-69c46b5f8d-2kjgn" podStartSLOduration=2.447920137 podStartE2EDuration="2.887745704s" totalImagesPullingTime="439.825567ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 12:18:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 12:18:29.886476479 +0000 UTC m=+1881.405208292" watchObservedRunningTime="2026-09-07 12:18:29.887745704 +0000 UTC m=+1881.406477516"
Sep 07 12:18:35.883718 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:18:35.883686    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-845d36-69c46b5f8d-2kjgn"
Sep 07 12:18:36.511024 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:18:36.510983    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-p4l4w"]
Sep 07 12:18:41.905691 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:18:41.905647    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:18:46.963092 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:18:46.963056    2563 generic.go:356] "Generic (PLEG): container finished" podID="f2486ed1-45b8-49ec-ad3c-82e6094aeee4" containerID="a6ca9c841b8cdf2180a3474ec48e9fbb6e26f8ec6029d18b825844dda76d9359" exitCode=0
Sep 07 12:18:46.963540 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:18:46.963130    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-845d36-openshift-default-64d7978b84-kkzds" event={"ID":"f2486ed1-45b8-49ec-ad3c-82e6094aeee4","Type":"ContainerDied","Data":"a6ca9c841b8cdf2180a3474ec48e9fbb6e26f8ec6029d18b825844dda76d9359"}
Sep 07 12:18:46.963588 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:18:46.963558    2563 scope.go:117] "RemoveContainer" containerID="a6ca9c841b8cdf2180a3474ec48e9fbb6e26f8ec6029d18b825844dda76d9359"
Sep 07 12:18:46.964735 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:18:46.964709    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 07 12:18:46.964818 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:18:46.964777    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 07 12:18:46.964818 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:18:46.964808    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 07 12:18:47.977184 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:18:47.977149    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-845d36-openshift-default-64d7978b84-kkzds" event={"ID":"f2486ed1-45b8-49ec-ad3c-82e6094aeee4","Type":"ContainerStarted","Data":"3b369678548808b7c7e4467fc7db6e4e7fa02d8124001d5f65570255227d463d"}
Sep 07 12:18:48.086126 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:18:48.086100    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-cleanup-845d36-openshift-default-64d7978b84-kkzds"
Sep 07 12:18:48.086344 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:18:48.086323    2563 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-845d36-openshift-default-64d7978b84-kkzds container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused" start-of-body=
Sep 07 12:18:48.086436 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:18:48.086395    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-845d36-openshift-default-64d7978b84-kkzds" podUID="f2486ed1-45b8-49ec-ad3c-82e6094aeee4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused"
Sep 07 12:18:49.086845 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:18:49.086796    2563 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-845d36-openshift-default-64d7978b84-kkzds container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused" start-of-body=
Sep 07 12:18:49.087335 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:18:49.086908    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-845d36-openshift-default-64d7978b84-kkzds" podUID="f2486ed1-45b8-49ec-ad3c-82e6094aeee4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused"
Sep 07 12:18:50.086587 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:18:50.086552    2563 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-845d36-openshift-default-64d7978b84-kkzds container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused" start-of-body=
Sep 07 12:18:50.086772 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:18:50.086614    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-845d36-openshift-default-64d7978b84-kkzds" podUID="f2486ed1-45b8-49ec-ad3c-82e6094aeee4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused"
Sep 07 12:18:51.086246 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:18:51.086211    2563 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-845d36-openshift-default-64d7978b84-kkzds container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused" start-of-body=
Sep 07 12:18:51.086654 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:18:51.086270    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-845d36-openshift-default-64d7978b84-kkzds" podUID="f2486ed1-45b8-49ec-ad3c-82e6094aeee4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused"
Sep 07 12:18:51.707031 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:18:51.706984    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:18:52.086293 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:18:52.086207    2563 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-845d36-openshift-default-64d7978b84-kkzds container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused" start-of-body=
Sep 07 12:18:52.086293 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:18:52.086262    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-845d36-openshift-default-64d7978b84-kkzds" podUID="f2486ed1-45b8-49ec-ad3c-82e6094aeee4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused"
Sep 07 12:18:53.085699 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:18:53.085663    2563 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-845d36-openshift-default-64d7978b84-kkzds container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused" start-of-body=
Sep 07 12:18:53.085881 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:18:53.085731    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-845d36-openshift-default-64d7978b84-kkzds" podUID="f2486ed1-45b8-49ec-ad3c-82e6094aeee4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused"
Sep 07 12:18:54.164577 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:18:54.164541    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-cleanup-845d36-openshift-default-64d7978b84-kkzds"
Sep 07 12:18:54.165049 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:18:54.165028    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-cleanup-845d36-openshift-default-64d7978b84-kkzds"
Sep 07 12:18:54.165749 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:18:54.165729    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-cleanup-845d36-openshift-default-64d7978b84-kkzds"
Sep 07 12:18:58.358557 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:18:58.358521    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-845d36-69c46b5f8d-2kjgn"]
Sep 07 12:18:58.359155 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:18:58.358733    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-845d36-69c46b5f8d-2kjgn" podUID="5f741d59-c73c-48b8-a6d5-887ee0467484" containerName="maas-api" containerID="cri-o://24c0944c552c41ae4c4ef23848f104a828d1b1c30fa2a614573dd5e6970971dd" gracePeriod=30
Sep 07 12:18:58.563874 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:18:58.563836    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:18:58.599274 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:18:58.599255    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-845d36-69c46b5f8d-2kjgn"
Sep 07 12:18:58.622848 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:18:58.622775    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:18:58.739076 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:18:58.739046    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vcrml\" (UniqueName: \"kubernetes.io/projected/5f741d59-c73c-48b8-a6d5-887ee0467484-kube-api-access-vcrml\") pod \"5f741d59-c73c-48b8-a6d5-887ee0467484\" (UID: \"5f741d59-c73c-48b8-a6d5-887ee0467484\") "
Sep 07 12:18:58.739271 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:18:58.739087    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5f741d59-c73c-48b8-a6d5-887ee0467484-maas-api-tls\") pod \"5f741d59-c73c-48b8-a6d5-887ee0467484\" (UID: \"5f741d59-c73c-48b8-a6d5-887ee0467484\") "
Sep 07 12:18:58.739271 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:18:58.739160    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5f741d59-c73c-48b8-a6d5-887ee0467484-openshift-service-ca\") pod \"5f741d59-c73c-48b8-a6d5-887ee0467484\" (UID: \"5f741d59-c73c-48b8-a6d5-887ee0467484\") "
Sep 07 12:18:58.739404 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:18:58.739296    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5f741d59-c73c-48b8-a6d5-887ee0467484-metrics-certs\") pod \"5f741d59-c73c-48b8-a6d5-887ee0467484\" (UID: \"5f741d59-c73c-48b8-a6d5-887ee0467484\") "
Sep 07 12:18:58.739823 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:18:58.739602    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5f741d59-c73c-48b8-a6d5-887ee0467484-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "5f741d59-c73c-48b8-a6d5-887ee0467484" (UID: "5f741d59-c73c-48b8-a6d5-887ee0467484"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 07 12:18:58.742101 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:18:58.742038    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5f741d59-c73c-48b8-a6d5-887ee0467484-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "5f741d59-c73c-48b8-a6d5-887ee0467484" (UID: "5f741d59-c73c-48b8-a6d5-887ee0467484"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 07 12:18:58.742101 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:18:58.742050    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5f741d59-c73c-48b8-a6d5-887ee0467484-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "5f741d59-c73c-48b8-a6d5-887ee0467484" (UID: "5f741d59-c73c-48b8-a6d5-887ee0467484"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 07 12:18:58.742101 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:18:58.742063    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5f741d59-c73c-48b8-a6d5-887ee0467484-kube-api-access-vcrml" (OuterVolumeSpecName: "kube-api-access-vcrml") pod "5f741d59-c73c-48b8-a6d5-887ee0467484" (UID: "5f741d59-c73c-48b8-a6d5-887ee0467484"). InnerVolumeSpecName "kube-api-access-vcrml". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 12:18:58.840756 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:18:58.840715    2563 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5f741d59-c73c-48b8-a6d5-887ee0467484-metrics-certs\") on node \"ip-10-0-135-173.ec2.internal\" DevicePath \"\""
Sep 07 12:18:58.840756 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:18:58.840751    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vcrml\" (UniqueName: \"kubernetes.io/projected/5f741d59-c73c-48b8-a6d5-887ee0467484-kube-api-access-vcrml\") on node \"ip-10-0-135-173.ec2.internal\" DevicePath \"\""
Sep 07 12:18:58.840756 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:18:58.840763    2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5f741d59-c73c-48b8-a6d5-887ee0467484-maas-api-tls\") on node \"ip-10-0-135-173.ec2.internal\" DevicePath \"\""
Sep 07 12:18:58.840756 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:18:58.840773    2563 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5f741d59-c73c-48b8-a6d5-887ee0467484-openshift-service-ca\") on node \"ip-10-0-135-173.ec2.internal\" DevicePath \"\""
Sep 07 12:18:58.970443 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:18:58.970399    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:18:59.034337 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:18:59.034309    2563 generic.go:356] "Generic (PLEG): container finished" podID="5f741d59-c73c-48b8-a6d5-887ee0467484" containerID="24c0944c552c41ae4c4ef23848f104a828d1b1c30fa2a614573dd5e6970971dd" exitCode=0
Sep 07 12:18:59.037969 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:18:59.037949    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-845d36-69c46b5f8d-2kjgn"
Sep 07 12:18:59.041682 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:18:59.041661    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-845d36-69c46b5f8d-2kjgn" event={"ID":"5f741d59-c73c-48b8-a6d5-887ee0467484","Type":"ContainerDied","Data":"24c0944c552c41ae4c4ef23848f104a828d1b1c30fa2a614573dd5e6970971dd"}
Sep 07 12:18:59.041769 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:18:59.041699    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-845d36-69c46b5f8d-2kjgn" event={"ID":"5f741d59-c73c-48b8-a6d5-887ee0467484","Type":"ContainerDied","Data":"1a492c2fe98f21f19191e1015db231f6bc830d9f1d50a1b9fbaeeec8f4f6bff4"}
Sep 07 12:18:59.041769 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:18:59.041722    2563 scope.go:117] "RemoveContainer" containerID="24c0944c552c41ae4c4ef23848f104a828d1b1c30fa2a614573dd5e6970971dd"
Sep 07 12:18:59.052010 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:18:59.051991    2563 scope.go:117] "RemoveContainer" containerID="24c0944c552c41ae4c4ef23848f104a828d1b1c30fa2a614573dd5e6970971dd"
Sep 07 12:18:59.052268 ip-10-0-135-173 kubenswrapper[2563]: E0907 12:18:59.052248    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"24c0944c552c41ae4c4ef23848f104a828d1b1c30fa2a614573dd5e6970971dd\": container with ID starting with 24c0944c552c41ae4c4ef23848f104a828d1b1c30fa2a614573dd5e6970971dd not found: ID does not exist" containerID="24c0944c552c41ae4c4ef23848f104a828d1b1c30fa2a614573dd5e6970971dd"
Sep 07 12:18:59.052336 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:18:59.052280    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"24c0944c552c41ae4c4ef23848f104a828d1b1c30fa2a614573dd5e6970971dd"} err="failed to get container status \"24c0944c552c41ae4c4ef23848f104a828d1b1c30fa2a614573dd5e6970971dd\": rpc error: code = NotFound desc = could not find container \"24c0944c552c41ae4c4ef23848f104a828d1b1c30fa2a614573dd5e6970971dd\": container with ID starting with 24c0944c552c41ae4c4ef23848f104a828d1b1c30fa2a614573dd5e6970971dd not found: ID does not exist"
Sep 07 12:18:59.061797 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:18:59.061777    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-845d36-69c46b5f8d-2kjgn"]
Sep 07 12:18:59.067588 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:18:59.067567    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-845d36-69c46b5f8d-2kjgn"]
Sep 07 12:19:01.040378 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:19:01.040343    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5f741d59-c73c-48b8-a6d5-887ee0467484" path="/var/lib/kubelet/pods/5f741d59-c73c-48b8-a6d5-887ee0467484/volumes"
Sep 07 12:19:04.401113 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:19:04.401082    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-p4l4w"]
Sep 07 12:19:04.872100 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:19:04.872060    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:19:04.972644 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:19:04.972605    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:19:05.118689 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:19:05.118653    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-845d36-openshift-default-64d7978b84-kkzds"]
Sep 07 12:19:05.118870 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:19:05.118849    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-cleanup-845d36-openshift-default-64d7978b84-kkzds" podUID="f2486ed1-45b8-49ec-ad3c-82e6094aeee4" containerName="istio-proxy" containerID="cri-o://3b369678548808b7c7e4467fc7db6e4e7fa02d8124001d5f65570255227d463d" gracePeriod=30
Sep 07 12:19:05.143203 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:19:05.143130    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:19:08.839088 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:19:08.839050    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:19:09.167508 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:19:09.167406    2563 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-845d36-openshift-default-64d7978b84-kkzds container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=upstream connect error or disconnect/reset before headers. reset reason: remote connection failure, transport failure reason: delayed connect error: Connection refused
Sep 07 12:19:09.167508 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:19:09.167487    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ipp-cleanup-845d36-openshift-default-64d7978b84-kkzds" podUID="f2486ed1-45b8-49ec-ad3c-82e6094aeee4" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 07 12:19:10.364887 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:19:10.364865    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-845d36-openshift-default-64d7978b84-kkzds"
Sep 07 12:19:10.441290 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:19:10.441218    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f2486ed1-45b8-49ec-ad3c-82e6094aeee4-istiod-ca-cert\") pod \"f2486ed1-45b8-49ec-ad3c-82e6094aeee4\" (UID: \"f2486ed1-45b8-49ec-ad3c-82e6094aeee4\") "
Sep 07 12:19:10.441290 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:19:10.441288    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f2486ed1-45b8-49ec-ad3c-82e6094aeee4-workload-certs\") pod \"f2486ed1-45b8-49ec-ad3c-82e6094aeee4\" (UID: \"f2486ed1-45b8-49ec-ad3c-82e6094aeee4\") "
Sep 07 12:19:10.441488 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:19:10.441311    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f2486ed1-45b8-49ec-ad3c-82e6094aeee4-istio-token\") pod \"f2486ed1-45b8-49ec-ad3c-82e6094aeee4\" (UID: \"f2486ed1-45b8-49ec-ad3c-82e6094aeee4\") "
Sep 07 12:19:10.441488 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:19:10.441366    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f2486ed1-45b8-49ec-ad3c-82e6094aeee4-workload-socket\") pod \"f2486ed1-45b8-49ec-ad3c-82e6094aeee4\" (UID: \"f2486ed1-45b8-49ec-ad3c-82e6094aeee4\") "
Sep 07 12:19:10.441488 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:19:10.441386    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f2486ed1-45b8-49ec-ad3c-82e6094aeee4-istio-envoy\") pod \"f2486ed1-45b8-49ec-ad3c-82e6094aeee4\" (UID: \"f2486ed1-45b8-49ec-ad3c-82e6094aeee4\") "
Sep 07 12:19:10.441488 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:19:10.441432    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2fmsh\" (UniqueName: \"kubernetes.io/projected/f2486ed1-45b8-49ec-ad3c-82e6094aeee4-kube-api-access-2fmsh\") pod \"f2486ed1-45b8-49ec-ad3c-82e6094aeee4\" (UID: \"f2486ed1-45b8-49ec-ad3c-82e6094aeee4\") "
Sep 07 12:19:10.441488 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:19:10.441461    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f2486ed1-45b8-49ec-ad3c-82e6094aeee4-credential-socket\") pod \"f2486ed1-45b8-49ec-ad3c-82e6094aeee4\" (UID: \"f2486ed1-45b8-49ec-ad3c-82e6094aeee4\") "
Sep 07 12:19:10.441751 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:19:10.441499    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f2486ed1-45b8-49ec-ad3c-82e6094aeee4-istio-podinfo\") pod \"f2486ed1-45b8-49ec-ad3c-82e6094aeee4\" (UID: \"f2486ed1-45b8-49ec-ad3c-82e6094aeee4\") "
Sep 07 12:19:10.441751 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:19:10.441532    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f2486ed1-45b8-49ec-ad3c-82e6094aeee4-istio-data\") pod \"f2486ed1-45b8-49ec-ad3c-82e6094aeee4\" (UID: \"f2486ed1-45b8-49ec-ad3c-82e6094aeee4\") "
Sep 07 12:19:10.441751 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:19:10.441649    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f2486ed1-45b8-49ec-ad3c-82e6094aeee4-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "f2486ed1-45b8-49ec-ad3c-82e6094aeee4" (UID: "f2486ed1-45b8-49ec-ad3c-82e6094aeee4"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 12:19:10.441751 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:19:10.441662    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f2486ed1-45b8-49ec-ad3c-82e6094aeee4-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "f2486ed1-45b8-49ec-ad3c-82e6094aeee4" (UID: "f2486ed1-45b8-49ec-ad3c-82e6094aeee4"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 12:19:10.441751 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:19:10.441724    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f2486ed1-45b8-49ec-ad3c-82e6094aeee4-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "f2486ed1-45b8-49ec-ad3c-82e6094aeee4" (UID: "f2486ed1-45b8-49ec-ad3c-82e6094aeee4"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 07 12:19:10.442023 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:19:10.441842    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f2486ed1-45b8-49ec-ad3c-82e6094aeee4-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "f2486ed1-45b8-49ec-ad3c-82e6094aeee4" (UID: "f2486ed1-45b8-49ec-ad3c-82e6094aeee4"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 12:19:10.442023 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:19:10.441929    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f2486ed1-45b8-49ec-ad3c-82e6094aeee4-istio-data" (OuterVolumeSpecName: "istio-data") pod "f2486ed1-45b8-49ec-ad3c-82e6094aeee4" (UID: "f2486ed1-45b8-49ec-ad3c-82e6094aeee4"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 12:19:10.442132 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:19:10.442020    2563 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f2486ed1-45b8-49ec-ad3c-82e6094aeee4-workload-certs\") on node \"ip-10-0-135-173.ec2.internal\" DevicePath \"\""
Sep 07 12:19:10.442132 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:19:10.442039    2563 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f2486ed1-45b8-49ec-ad3c-82e6094aeee4-workload-socket\") on node \"ip-10-0-135-173.ec2.internal\" DevicePath \"\""
Sep 07 12:19:10.442132 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:19:10.442055    2563 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f2486ed1-45b8-49ec-ad3c-82e6094aeee4-credential-socket\") on node \"ip-10-0-135-173.ec2.internal\" DevicePath \"\""
Sep 07 12:19:10.442132 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:19:10.442066    2563 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f2486ed1-45b8-49ec-ad3c-82e6094aeee4-istio-data\") on node \"ip-10-0-135-173.ec2.internal\" DevicePath \"\""
Sep 07 12:19:10.442132 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:19:10.442075    2563 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f2486ed1-45b8-49ec-ad3c-82e6094aeee4-istiod-ca-cert\") on node \"ip-10-0-135-173.ec2.internal\" DevicePath \"\""
Sep 07 12:19:10.443607 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:19:10.443582    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f2486ed1-45b8-49ec-ad3c-82e6094aeee4-istio-token" (OuterVolumeSpecName: "istio-token") pod "f2486ed1-45b8-49ec-ad3c-82e6094aeee4" (UID: "f2486ed1-45b8-49ec-ad3c-82e6094aeee4"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 12:19:10.443722 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:19:10.443641    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f2486ed1-45b8-49ec-ad3c-82e6094aeee4-kube-api-access-2fmsh" (OuterVolumeSpecName: "kube-api-access-2fmsh") pod "f2486ed1-45b8-49ec-ad3c-82e6094aeee4" (UID: "f2486ed1-45b8-49ec-ad3c-82e6094aeee4"). InnerVolumeSpecName "kube-api-access-2fmsh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 12:19:10.444097 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:19:10.444076    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/f2486ed1-45b8-49ec-ad3c-82e6094aeee4-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "f2486ed1-45b8-49ec-ad3c-82e6094aeee4" (UID: "f2486ed1-45b8-49ec-ad3c-82e6094aeee4"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 07 12:19:10.444189 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:19:10.444100    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f2486ed1-45b8-49ec-ad3c-82e6094aeee4-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "f2486ed1-45b8-49ec-ad3c-82e6094aeee4" (UID: "f2486ed1-45b8-49ec-ad3c-82e6094aeee4"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 12:19:10.542842 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:19:10.542816    2563 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f2486ed1-45b8-49ec-ad3c-82e6094aeee4-istio-envoy\") on node \"ip-10-0-135-173.ec2.internal\" DevicePath \"\""
Sep 07 12:19:10.542842 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:19:10.542840    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2fmsh\" (UniqueName: \"kubernetes.io/projected/f2486ed1-45b8-49ec-ad3c-82e6094aeee4-kube-api-access-2fmsh\") on node \"ip-10-0-135-173.ec2.internal\" DevicePath \"\""
Sep 07 12:19:10.542988 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:19:10.542850    2563 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f2486ed1-45b8-49ec-ad3c-82e6094aeee4-istio-podinfo\") on node \"ip-10-0-135-173.ec2.internal\" DevicePath \"\""
Sep 07 12:19:10.542988 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:19:10.542859    2563 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f2486ed1-45b8-49ec-ad3c-82e6094aeee4-istio-token\") on node \"ip-10-0-135-173.ec2.internal\" DevicePath \"\""
Sep 07 12:19:11.096166 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:19:11.096131    2563 generic.go:356] "Generic (PLEG): container finished" podID="f2486ed1-45b8-49ec-ad3c-82e6094aeee4" containerID="3b369678548808b7c7e4467fc7db6e4e7fa02d8124001d5f65570255227d463d" exitCode=0
Sep 07 12:19:11.096304 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:19:11.096176    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-845d36-openshift-default-64d7978b84-kkzds" event={"ID":"f2486ed1-45b8-49ec-ad3c-82e6094aeee4","Type":"ContainerDied","Data":"3b369678548808b7c7e4467fc7db6e4e7fa02d8124001d5f65570255227d463d"}
Sep 07 12:19:11.096304 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:19:11.096227    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-845d36-openshift-default-64d7978b84-kkzds" event={"ID":"f2486ed1-45b8-49ec-ad3c-82e6094aeee4","Type":"ContainerDied","Data":"03802548ab09a1d6cd07b9a3d452f57e7131d1447b2fe112015d27f682c7ea09"}
Sep 07 12:19:11.096304 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:19:11.096249    2563 scope.go:117] "RemoveContainer" containerID="3b369678548808b7c7e4467fc7db6e4e7fa02d8124001d5f65570255227d463d"
Sep 07 12:19:11.096304 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:19:11.096184    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-845d36-openshift-default-64d7978b84-kkzds"
Sep 07 12:19:11.104921 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:19:11.104903    2563 scope.go:117] "RemoveContainer" containerID="a6ca9c841b8cdf2180a3474ec48e9fbb6e26f8ec6029d18b825844dda76d9359"
Sep 07 12:19:11.110954 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:19:11.110934    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-845d36-openshift-default-64d7978b84-kkzds"]
Sep 07 12:19:11.113314 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:19:11.113296    2563 scope.go:117] "RemoveContainer" containerID="3b369678548808b7c7e4467fc7db6e4e7fa02d8124001d5f65570255227d463d"
Sep 07 12:19:11.113727 ip-10-0-135-173 kubenswrapper[2563]: E0907 12:19:11.113708    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3b369678548808b7c7e4467fc7db6e4e7fa02d8124001d5f65570255227d463d\": container with ID starting with 3b369678548808b7c7e4467fc7db6e4e7fa02d8124001d5f65570255227d463d not found: ID does not exist" containerID="3b369678548808b7c7e4467fc7db6e4e7fa02d8124001d5f65570255227d463d"
Sep 07 12:19:11.113810 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:19:11.113733    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3b369678548808b7c7e4467fc7db6e4e7fa02d8124001d5f65570255227d463d"} err="failed to get container status \"3b369678548808b7c7e4467fc7db6e4e7fa02d8124001d5f65570255227d463d\": rpc error: code = NotFound desc = could not find container \"3b369678548808b7c7e4467fc7db6e4e7fa02d8124001d5f65570255227d463d\": container with ID starting with 3b369678548808b7c7e4467fc7db6e4e7fa02d8124001d5f65570255227d463d not found: ID does not exist"
Sep 07 12:19:11.113810 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:19:11.113748    2563 scope.go:117] "RemoveContainer" containerID="a6ca9c841b8cdf2180a3474ec48e9fbb6e26f8ec6029d18b825844dda76d9359"
Sep 07 12:19:11.114010 ip-10-0-135-173 kubenswrapper[2563]: E0907 12:19:11.113991    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a6ca9c841b8cdf2180a3474ec48e9fbb6e26f8ec6029d18b825844dda76d9359\": container with ID starting with a6ca9c841b8cdf2180a3474ec48e9fbb6e26f8ec6029d18b825844dda76d9359 not found: ID does not exist" containerID="a6ca9c841b8cdf2180a3474ec48e9fbb6e26f8ec6029d18b825844dda76d9359"
Sep 07 12:19:11.114058 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:19:11.114015    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a6ca9c841b8cdf2180a3474ec48e9fbb6e26f8ec6029d18b825844dda76d9359"} err="failed to get container status \"a6ca9c841b8cdf2180a3474ec48e9fbb6e26f8ec6029d18b825844dda76d9359\": rpc error: code = NotFound desc = could not find container \"a6ca9c841b8cdf2180a3474ec48e9fbb6e26f8ec6029d18b825844dda76d9359\": container with ID starting with a6ca9c841b8cdf2180a3474ec48e9fbb6e26f8ec6029d18b825844dda76d9359 not found: ID does not exist"
Sep 07 12:19:11.115066 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:19:11.115049    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-845d36-openshift-default-64d7978b84-kkzds"]
Sep 07 12:19:13.040827 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:19:13.040793    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f2486ed1-45b8-49ec-ad3c-82e6094aeee4" path="/var/lib/kubelet/pods/f2486ed1-45b8-49ec-ad3c-82e6094aeee4/volumes"
Sep 07 12:19:14.646319 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:19:14.646281    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:19:14.697535 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:19:14.697504    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:19:24.269434 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:19:24.269370    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:19:24.341241 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:19:24.341206    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:19:24.400335 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:19:24.400291    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:19:24.480603 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:19:24.480553    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:19:24.541655 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:19:24.541571    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:19:24.868023 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:19:24.867933    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:19:25.216746 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:19:25.216716    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-p4l4w"]
Sep 07 12:19:28.929333 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:19:28.929302    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:19:29.856774 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:19:29.856735    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-ipp-a-aab734/ipp-route-aab734-kserve-6fb79867c5-hcfd8"]
Sep 07 12:19:29.896240 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:19:29.896188    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:19:35.598546 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:19:35.598460    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-p4l4w"]
Sep 07 12:19:44.881278 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:19:44.881214    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-ipp-a-aab734/ipp-route-aab734-kserve-6fb79867c5-hcfd8" podUID="fd792abf-4178-43c0-b4f0-94df7367bff3" containerName="main" containerID="cri-o://ef9691c0ab2332d8c969e5786ebccbff335a299b0091044040fa219b20e33bc9" gracePeriod=45
Sep 07 12:19:45.134632 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:19:45.134570    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-aab734/ipp-route-aab734-kserve-6fb79867c5-hcfd8"
Sep 07 12:19:45.264713 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:19:45.264678    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fd792abf-4178-43c0-b4f0-94df7367bff3-tmp-dir\") pod \"fd792abf-4178-43c0-b4f0-94df7367bff3\" (UID: \"fd792abf-4178-43c0-b4f0-94df7367bff3\") "
Sep 07 12:19:45.264885 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:19:45.264728    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fd792abf-4178-43c0-b4f0-94df7367bff3-tls-certs\") pod \"fd792abf-4178-43c0-b4f0-94df7367bff3\" (UID: \"fd792abf-4178-43c0-b4f0-94df7367bff3\") "
Sep 07 12:19:45.264885 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:19:45.264771    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fd792abf-4178-43c0-b4f0-94df7367bff3-home\") pod \"fd792abf-4178-43c0-b4f0-94df7367bff3\" (UID: \"fd792abf-4178-43c0-b4f0-94df7367bff3\") "
Sep 07 12:19:45.264885 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:19:45.264796    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fd792abf-4178-43c0-b4f0-94df7367bff3-model-cache\") pod \"fd792abf-4178-43c0-b4f0-94df7367bff3\" (UID: \"fd792abf-4178-43c0-b4f0-94df7367bff3\") "
Sep 07 12:19:45.264885 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:19:45.264829    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pqpxt\" (UniqueName: \"kubernetes.io/projected/fd792abf-4178-43c0-b4f0-94df7367bff3-kube-api-access-pqpxt\") pod \"fd792abf-4178-43c0-b4f0-94df7367bff3\" (UID: \"fd792abf-4178-43c0-b4f0-94df7367bff3\") "
Sep 07 12:19:45.264885 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:19:45.264869    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fd792abf-4178-43c0-b4f0-94df7367bff3-dshm\") pod \"fd792abf-4178-43c0-b4f0-94df7367bff3\" (UID: \"fd792abf-4178-43c0-b4f0-94df7367bff3\") "
Sep 07 12:19:45.265163 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:19:45.264929    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fd792abf-4178-43c0-b4f0-94df7367bff3-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "fd792abf-4178-43c0-b4f0-94df7367bff3" (UID: "fd792abf-4178-43c0-b4f0-94df7367bff3"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 12:19:45.265163 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:19:45.265096    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fd792abf-4178-43c0-b4f0-94df7367bff3-home" (OuterVolumeSpecName: "home") pod "fd792abf-4178-43c0-b4f0-94df7367bff3" (UID: "fd792abf-4178-43c0-b4f0-94df7367bff3"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 12:19:45.265163 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:19:45.265116    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fd792abf-4178-43c0-b4f0-94df7367bff3-model-cache" (OuterVolumeSpecName: "model-cache") pod "fd792abf-4178-43c0-b4f0-94df7367bff3" (UID: "fd792abf-4178-43c0-b4f0-94df7367bff3"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 12:19:45.265321 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:19:45.265289    2563 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fd792abf-4178-43c0-b4f0-94df7367bff3-home\") on node \"ip-10-0-135-173.ec2.internal\" DevicePath \"\""
Sep 07 12:19:45.265321 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:19:45.265310    2563 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fd792abf-4178-43c0-b4f0-94df7367bff3-model-cache\") on node \"ip-10-0-135-173.ec2.internal\" DevicePath \"\""
Sep 07 12:19:45.265395 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:19:45.265326    2563 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fd792abf-4178-43c0-b4f0-94df7367bff3-tmp-dir\") on node \"ip-10-0-135-173.ec2.internal\" DevicePath \"\""
Sep 07 12:19:45.267218 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:19:45.267186    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fd792abf-4178-43c0-b4f0-94df7367bff3-kube-api-access-pqpxt" (OuterVolumeSpecName: "kube-api-access-pqpxt") pod "fd792abf-4178-43c0-b4f0-94df7367bff3" (UID: "fd792abf-4178-43c0-b4f0-94df7367bff3"). InnerVolumeSpecName "kube-api-access-pqpxt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 12:19:45.267339 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:19:45.267250    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fd792abf-4178-43c0-b4f0-94df7367bff3-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "fd792abf-4178-43c0-b4f0-94df7367bff3" (UID: "fd792abf-4178-43c0-b4f0-94df7367bff3"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 07 12:19:45.267339 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:19:45.267264    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fd792abf-4178-43c0-b4f0-94df7367bff3-dshm" (OuterVolumeSpecName: "dshm") pod "fd792abf-4178-43c0-b4f0-94df7367bff3" (UID: "fd792abf-4178-43c0-b4f0-94df7367bff3"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 12:19:45.270696 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:19:45.270667    2563 generic.go:356] "Generic (PLEG): container finished" podID="fd792abf-4178-43c0-b4f0-94df7367bff3" containerID="ef9691c0ab2332d8c969e5786ebccbff335a299b0091044040fa219b20e33bc9" exitCode=0
Sep 07 12:19:45.270783 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:19:45.270716    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-aab734/ipp-route-aab734-kserve-6fb79867c5-hcfd8"
Sep 07 12:19:45.270783 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:19:45.270723    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-aab734/ipp-route-aab734-kserve-6fb79867c5-hcfd8" event={"ID":"fd792abf-4178-43c0-b4f0-94df7367bff3","Type":"ContainerDied","Data":"ef9691c0ab2332d8c969e5786ebccbff335a299b0091044040fa219b20e33bc9"}
Sep 07 12:19:45.270783 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:19:45.270764    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-aab734/ipp-route-aab734-kserve-6fb79867c5-hcfd8" event={"ID":"fd792abf-4178-43c0-b4f0-94df7367bff3","Type":"ContainerDied","Data":"66869743fb5a8ae190c3e97b4da21d6d332f1b7ebe9ca7787d6f8552b41d5537"}
Sep 07 12:19:45.270897 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:19:45.270786    2563 scope.go:117] "RemoveContainer" containerID="ef9691c0ab2332d8c969e5786ebccbff335a299b0091044040fa219b20e33bc9"
Sep 07 12:19:45.287746 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:19:45.287730    2563 scope.go:117] "RemoveContainer" containerID="ef9691c0ab2332d8c969e5786ebccbff335a299b0091044040fa219b20e33bc9"
Sep 07 12:19:45.288017 ip-10-0-135-173 kubenswrapper[2563]: E0907 12:19:45.288001    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ef9691c0ab2332d8c969e5786ebccbff335a299b0091044040fa219b20e33bc9\": container with ID starting with ef9691c0ab2332d8c969e5786ebccbff335a299b0091044040fa219b20e33bc9 not found: ID does not exist" containerID="ef9691c0ab2332d8c969e5786ebccbff335a299b0091044040fa219b20e33bc9"
Sep 07 12:19:45.288067 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:19:45.288024    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ef9691c0ab2332d8c969e5786ebccbff335a299b0091044040fa219b20e33bc9"} err="failed to get container status \"ef9691c0ab2332d8c969e5786ebccbff335a299b0091044040fa219b20e33bc9\": rpc error: code = NotFound desc = could not find container \"ef9691c0ab2332d8c969e5786ebccbff335a299b0091044040fa219b20e33bc9\": container with ID starting with ef9691c0ab2332d8c969e5786ebccbff335a299b0091044040fa219b20e33bc9 not found: ID does not exist"
Sep 07 12:19:45.296084 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:19:45.296062    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-ipp-a-aab734/ipp-route-aab734-kserve-6fb79867c5-hcfd8"]
Sep 07 12:19:45.301859 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:19:45.301838    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-ipp-a-aab734/ipp-route-aab734-kserve-6fb79867c5-hcfd8"]
Sep 07 12:19:45.366726 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:19:45.366700    2563 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fd792abf-4178-43c0-b4f0-94df7367bff3-tls-certs\") on node \"ip-10-0-135-173.ec2.internal\" DevicePath \"\""
Sep 07 12:19:45.366726 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:19:45.366722    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pqpxt\" (UniqueName: \"kubernetes.io/projected/fd792abf-4178-43c0-b4f0-94df7367bff3-kube-api-access-pqpxt\") on node \"ip-10-0-135-173.ec2.internal\" DevicePath \"\""
Sep 07 12:19:45.366726 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:19:45.366730    2563 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fd792abf-4178-43c0-b4f0-94df7367bff3-dshm\") on node \"ip-10-0-135-173.ec2.internal\" DevicePath \"\""
Sep 07 12:19:47.040595 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:19:47.040561    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fd792abf-4178-43c0-b4f0-94df7367bff3" path="/var/lib/kubelet/pods/fd792abf-4178-43c0-b4f0-94df7367bff3/volumes"
Sep 07 12:19:54.458148 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:19:54.458115    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:19:57.315595 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:19:57.315557    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:20:07.394310 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:20:07.394227    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:20:07.442239 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:20:07.442209    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:20:12.811144 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:20:12.811105    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:20:20.960127 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:20:20.960090    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:20:35.608723 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:20:35.608635    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:20:35.709465 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:20:35.709406    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:20:35.874150 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:20:35.874071    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-a-aab734-openshift-default-8558845d6b-lq2x8"]
Sep 07 12:20:35.874372 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:20:35.874347    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-a-aab734-openshift-default-8558845d6b-lq2x8" podUID="a0c5913e-5a31-47c6-b62d-cca418880291" containerName="istio-proxy" containerID="cri-o://6bf6915193b230d9c907eba0dd7cc1d37b67092d0b5f91b68ed7ec0e4a798d20" gracePeriod=30
Sep 07 12:20:35.888784 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:20:35.888748    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:20:36.346808 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:20:36.346772    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:20:38.750477 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:20:38.750435    2563 patch_prober.go:28] interesting pod/e2e-ipp-a-aab734-openshift-default-8558845d6b-lq2x8 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 07 12:20:38.750931 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:20:38.750516    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ipp-a-aab734-openshift-default-8558845d6b-lq2x8" podUID="a0c5913e-5a31-47c6-b62d-cca418880291" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 07 12:20:41.126117 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:20:41.126091    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-aab734-openshift-default-8558845d6b-lq2x8"
Sep 07 12:20:41.261043 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:20:41.261009    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a0c5913e-5a31-47c6-b62d-cca418880291-workload-certs\") pod \"a0c5913e-5a31-47c6-b62d-cca418880291\" (UID: \"a0c5913e-5a31-47c6-b62d-cca418880291\") "
Sep 07 12:20:41.261207 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:20:41.261074    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a0c5913e-5a31-47c6-b62d-cca418880291-istio-data\") pod \"a0c5913e-5a31-47c6-b62d-cca418880291\" (UID: \"a0c5913e-5a31-47c6-b62d-cca418880291\") "
Sep 07 12:20:41.261207 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:20:41.261119    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a0c5913e-5a31-47c6-b62d-cca418880291-workload-socket\") pod \"a0c5913e-5a31-47c6-b62d-cca418880291\" (UID: \"a0c5913e-5a31-47c6-b62d-cca418880291\") "
Sep 07 12:20:41.261207 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:20:41.261146    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a0c5913e-5a31-47c6-b62d-cca418880291-credential-socket\") pod \"a0c5913e-5a31-47c6-b62d-cca418880291\" (UID: \"a0c5913e-5a31-47c6-b62d-cca418880291\") "
Sep 07 12:20:41.261207 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:20:41.261193    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a0c5913e-5a31-47c6-b62d-cca418880291-istio-token\") pod \"a0c5913e-5a31-47c6-b62d-cca418880291\" (UID: \"a0c5913e-5a31-47c6-b62d-cca418880291\") "
Sep 07 12:20:41.261387 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:20:41.261225    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vmdp5\" (UniqueName: \"kubernetes.io/projected/a0c5913e-5a31-47c6-b62d-cca418880291-kube-api-access-vmdp5\") pod \"a0c5913e-5a31-47c6-b62d-cca418880291\" (UID: \"a0c5913e-5a31-47c6-b62d-cca418880291\") "
Sep 07 12:20:41.261387 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:20:41.261248    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a0c5913e-5a31-47c6-b62d-cca418880291-istio-envoy\") pod \"a0c5913e-5a31-47c6-b62d-cca418880291\" (UID: \"a0c5913e-5a31-47c6-b62d-cca418880291\") "
Sep 07 12:20:41.261387 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:20:41.261271    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a0c5913e-5a31-47c6-b62d-cca418880291-istiod-ca-cert\") pod \"a0c5913e-5a31-47c6-b62d-cca418880291\" (UID: \"a0c5913e-5a31-47c6-b62d-cca418880291\") "
Sep 07 12:20:41.261387 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:20:41.261281    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a0c5913e-5a31-47c6-b62d-cca418880291-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "a0c5913e-5a31-47c6-b62d-cca418880291" (UID: "a0c5913e-5a31-47c6-b62d-cca418880291"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 12:20:41.261631 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:20:41.261440    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a0c5913e-5a31-47c6-b62d-cca418880291-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "a0c5913e-5a31-47c6-b62d-cca418880291" (UID: "a0c5913e-5a31-47c6-b62d-cca418880291"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 12:20:41.261631 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:20:41.261455    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a0c5913e-5a31-47c6-b62d-cca418880291-istio-data" (OuterVolumeSpecName: "istio-data") pod "a0c5913e-5a31-47c6-b62d-cca418880291" (UID: "a0c5913e-5a31-47c6-b62d-cca418880291"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 12:20:41.261631 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:20:41.261517    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a0c5913e-5a31-47c6-b62d-cca418880291-istio-podinfo\") pod \"a0c5913e-5a31-47c6-b62d-cca418880291\" (UID: \"a0c5913e-5a31-47c6-b62d-cca418880291\") "
Sep 07 12:20:41.261631 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:20:41.261556    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a0c5913e-5a31-47c6-b62d-cca418880291-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "a0c5913e-5a31-47c6-b62d-cca418880291" (UID: "a0c5913e-5a31-47c6-b62d-cca418880291"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 12:20:41.261853 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:20:41.261721    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a0c5913e-5a31-47c6-b62d-cca418880291-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "a0c5913e-5a31-47c6-b62d-cca418880291" (UID: "a0c5913e-5a31-47c6-b62d-cca418880291"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 07 12:20:41.261958 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:20:41.261907    2563 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a0c5913e-5a31-47c6-b62d-cca418880291-workload-socket\") on node \"ip-10-0-135-173.ec2.internal\" DevicePath \"\""
Sep 07 12:20:41.261958 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:20:41.261930    2563 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a0c5913e-5a31-47c6-b62d-cca418880291-credential-socket\") on node \"ip-10-0-135-173.ec2.internal\" DevicePath \"\""
Sep 07 12:20:41.261958 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:20:41.261945    2563 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a0c5913e-5a31-47c6-b62d-cca418880291-istiod-ca-cert\") on node \"ip-10-0-135-173.ec2.internal\" DevicePath \"\""
Sep 07 12:20:41.261958 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:20:41.261958    2563 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a0c5913e-5a31-47c6-b62d-cca418880291-workload-certs\") on node \"ip-10-0-135-173.ec2.internal\" DevicePath \"\""
Sep 07 12:20:41.262176 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:20:41.261969    2563 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a0c5913e-5a31-47c6-b62d-cca418880291-istio-data\") on node \"ip-10-0-135-173.ec2.internal\" DevicePath \"\""
Sep 07 12:20:41.263579 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:20:41.263557    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/a0c5913e-5a31-47c6-b62d-cca418880291-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "a0c5913e-5a31-47c6-b62d-cca418880291" (UID: "a0c5913e-5a31-47c6-b62d-cca418880291"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 07 12:20:41.263685 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:20:41.263572    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a0c5913e-5a31-47c6-b62d-cca418880291-istio-token" (OuterVolumeSpecName: "istio-token") pod "a0c5913e-5a31-47c6-b62d-cca418880291" (UID: "a0c5913e-5a31-47c6-b62d-cca418880291"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 12:20:41.264700 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:20:41.264676    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a0c5913e-5a31-47c6-b62d-cca418880291-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "a0c5913e-5a31-47c6-b62d-cca418880291" (UID: "a0c5913e-5a31-47c6-b62d-cca418880291"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 12:20:41.264818 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:20:41.264734    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a0c5913e-5a31-47c6-b62d-cca418880291-kube-api-access-vmdp5" (OuterVolumeSpecName: "kube-api-access-vmdp5") pod "a0c5913e-5a31-47c6-b62d-cca418880291" (UID: "a0c5913e-5a31-47c6-b62d-cca418880291"). InnerVolumeSpecName "kube-api-access-vmdp5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 12:20:41.362892 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:20:41.362865    2563 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a0c5913e-5a31-47c6-b62d-cca418880291-istio-token\") on node \"ip-10-0-135-173.ec2.internal\" DevicePath \"\""
Sep 07 12:20:41.362892 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:20:41.362889    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vmdp5\" (UniqueName: \"kubernetes.io/projected/a0c5913e-5a31-47c6-b62d-cca418880291-kube-api-access-vmdp5\") on node \"ip-10-0-135-173.ec2.internal\" DevicePath \"\""
Sep 07 12:20:41.363062 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:20:41.362898    2563 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a0c5913e-5a31-47c6-b62d-cca418880291-istio-envoy\") on node \"ip-10-0-135-173.ec2.internal\" DevicePath \"\""
Sep 07 12:20:41.363062 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:20:41.362907    2563 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a0c5913e-5a31-47c6-b62d-cca418880291-istio-podinfo\") on node \"ip-10-0-135-173.ec2.internal\" DevicePath \"\""
Sep 07 12:20:41.563938 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:20:41.563856    2563 generic.go:356] "Generic (PLEG): container finished" podID="a0c5913e-5a31-47c6-b62d-cca418880291" containerID="6bf6915193b230d9c907eba0dd7cc1d37b67092d0b5f91b68ed7ec0e4a798d20" exitCode=0
Sep 07 12:20:41.563938 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:20:41.563910    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-aab734-openshift-default-8558845d6b-lq2x8"
Sep 07 12:20:41.564153 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:20:41.563914    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-aab734-openshift-default-8558845d6b-lq2x8" event={"ID":"a0c5913e-5a31-47c6-b62d-cca418880291","Type":"ContainerDied","Data":"6bf6915193b230d9c907eba0dd7cc1d37b67092d0b5f91b68ed7ec0e4a798d20"}
Sep 07 12:20:41.564153 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:20:41.564028    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-aab734-openshift-default-8558845d6b-lq2x8" event={"ID":"a0c5913e-5a31-47c6-b62d-cca418880291","Type":"ContainerDied","Data":"577b559ed9c54877646fc638392c172d39d093064bc62514bfcb6d90eb0a34f7"}
Sep 07 12:20:41.564153 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:20:41.564058    2563 scope.go:117] "RemoveContainer" containerID="6bf6915193b230d9c907eba0dd7cc1d37b67092d0b5f91b68ed7ec0e4a798d20"
Sep 07 12:20:41.573366 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:20:41.573350    2563 scope.go:117] "RemoveContainer" containerID="5cd87e26d7bb0f226c83cd8b7c0356c4ffc56c721a91f951b499953d363218a3"
Sep 07 12:20:41.581032 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:20:41.581009    2563 scope.go:117] "RemoveContainer" containerID="6bf6915193b230d9c907eba0dd7cc1d37b67092d0b5f91b68ed7ec0e4a798d20"
Sep 07 12:20:41.581277 ip-10-0-135-173 kubenswrapper[2563]: E0907 12:20:41.581259    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6bf6915193b230d9c907eba0dd7cc1d37b67092d0b5f91b68ed7ec0e4a798d20\": container with ID starting with 6bf6915193b230d9c907eba0dd7cc1d37b67092d0b5f91b68ed7ec0e4a798d20 not found: ID does not exist" containerID="6bf6915193b230d9c907eba0dd7cc1d37b67092d0b5f91b68ed7ec0e4a798d20"
Sep 07 12:20:41.581335 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:20:41.581286    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6bf6915193b230d9c907eba0dd7cc1d37b67092d0b5f91b68ed7ec0e4a798d20"} err="failed to get container status \"6bf6915193b230d9c907eba0dd7cc1d37b67092d0b5f91b68ed7ec0e4a798d20\": rpc error: code = NotFound desc = could not find container \"6bf6915193b230d9c907eba0dd7cc1d37b67092d0b5f91b68ed7ec0e4a798d20\": container with ID starting with 6bf6915193b230d9c907eba0dd7cc1d37b67092d0b5f91b68ed7ec0e4a798d20 not found: ID does not exist"
Sep 07 12:20:41.581335 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:20:41.581304    2563 scope.go:117] "RemoveContainer" containerID="5cd87e26d7bb0f226c83cd8b7c0356c4ffc56c721a91f951b499953d363218a3"
Sep 07 12:20:41.581603 ip-10-0-135-173 kubenswrapper[2563]: E0907 12:20:41.581584    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5cd87e26d7bb0f226c83cd8b7c0356c4ffc56c721a91f951b499953d363218a3\": container with ID starting with 5cd87e26d7bb0f226c83cd8b7c0356c4ffc56c721a91f951b499953d363218a3 not found: ID does not exist" containerID="5cd87e26d7bb0f226c83cd8b7c0356c4ffc56c721a91f951b499953d363218a3"
Sep 07 12:20:41.581657 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:20:41.581609    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5cd87e26d7bb0f226c83cd8b7c0356c4ffc56c721a91f951b499953d363218a3"} err="failed to get container status \"5cd87e26d7bb0f226c83cd8b7c0356c4ffc56c721a91f951b499953d363218a3\": rpc error: code = NotFound desc = could not find container \"5cd87e26d7bb0f226c83cd8b7c0356c4ffc56c721a91f951b499953d363218a3\": container with ID starting with 5cd87e26d7bb0f226c83cd8b7c0356c4ffc56c721a91f951b499953d363218a3 not found: ID does not exist"
Sep 07 12:20:41.584286 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:20:41.584265    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-a-aab734-openshift-default-8558845d6b-lq2x8"]
Sep 07 12:20:41.587881 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:20:41.587859    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-a-aab734-openshift-default-8558845d6b-lq2x8"]
Sep 07 12:20:43.040633 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:20:43.040600    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a0c5913e-5a31-47c6-b62d-cca418880291" path="/var/lib/kubelet/pods/a0c5913e-5a31-47c6-b62d-cca418880291/volumes"
Sep 07 12:20:46.886258 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:20:46.886226    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:20:47.214358 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:20:47.214323    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:20:47.269677 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:20:47.269641    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:20:47.466209 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:20:47.466115    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:20:50.108956 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:20:50.108124    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-p4l4w"]
Sep 07 12:20:53.131850 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:20:53.131816    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:20:53.312164 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:20:53.312127    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:20:53.373494 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:20:53.373465    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-b-3e7ca1-openshift-default-7f74dcbfc6-f79fh"]
Sep 07 12:20:53.373736 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:20:53.373713    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-b-3e7ca1-openshift-default-7f74dcbfc6-f79fh" podUID="ada380e3-be47-48af-9fde-9d422ac172ed" containerName="istio-proxy" containerID="cri-o://5dc5864fd2f0165bedfd7908ef41892b3c31f5206dad42bfd32ab0c8e18b3474" gracePeriod=30
Sep 07 12:20:53.405294 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:20:53.405218    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:20:53.871950 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:20:53.871921    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:20:58.619660 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:20:58.619630    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-3e7ca1-openshift-default-7f74dcbfc6-f79fh"
Sep 07 12:20:58.651917 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:20:58.651850    2563 generic.go:356] "Generic (PLEG): container finished" podID="ada380e3-be47-48af-9fde-9d422ac172ed" containerID="5dc5864fd2f0165bedfd7908ef41892b3c31f5206dad42bfd32ab0c8e18b3474" exitCode=0
Sep 07 12:20:58.651917 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:20:58.651895    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-3e7ca1-openshift-default-7f74dcbfc6-f79fh"
Sep 07 12:20:58.652067 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:20:58.651933    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-3e7ca1-openshift-default-7f74dcbfc6-f79fh" event={"ID":"ada380e3-be47-48af-9fde-9d422ac172ed","Type":"ContainerDied","Data":"5dc5864fd2f0165bedfd7908ef41892b3c31f5206dad42bfd32ab0c8e18b3474"}
Sep 07 12:20:58.652067 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:20:58.651990    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-3e7ca1-openshift-default-7f74dcbfc6-f79fh" event={"ID":"ada380e3-be47-48af-9fde-9d422ac172ed","Type":"ContainerDied","Data":"1aa79100e8303b6cd2ec8a00c3e167a383feb99842dd8680cb7ceb84d5c6431e"}
Sep 07 12:20:58.652067 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:20:58.652014    2563 scope.go:117] "RemoveContainer" containerID="5dc5864fd2f0165bedfd7908ef41892b3c31f5206dad42bfd32ab0c8e18b3474"
Sep 07 12:20:58.660566 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:20:58.660545    2563 scope.go:117] "RemoveContainer" containerID="012c5190c29ca3d7ef18f97592291166385bef3c634621f92ef1aecc25d9689d"
Sep 07 12:20:58.668318 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:20:58.668296    2563 scope.go:117] "RemoveContainer" containerID="5dc5864fd2f0165bedfd7908ef41892b3c31f5206dad42bfd32ab0c8e18b3474"
Sep 07 12:20:58.668591 ip-10-0-135-173 kubenswrapper[2563]: E0907 12:20:58.668572    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5dc5864fd2f0165bedfd7908ef41892b3c31f5206dad42bfd32ab0c8e18b3474\": container with ID starting with 5dc5864fd2f0165bedfd7908ef41892b3c31f5206dad42bfd32ab0c8e18b3474 not found: ID does not exist" containerID="5dc5864fd2f0165bedfd7908ef41892b3c31f5206dad42bfd32ab0c8e18b3474"
Sep 07 12:20:58.668648 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:20:58.668600    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5dc5864fd2f0165bedfd7908ef41892b3c31f5206dad42bfd32ab0c8e18b3474"} err="failed to get container status \"5dc5864fd2f0165bedfd7908ef41892b3c31f5206dad42bfd32ab0c8e18b3474\": rpc error: code = NotFound desc = could not find container \"5dc5864fd2f0165bedfd7908ef41892b3c31f5206dad42bfd32ab0c8e18b3474\": container with ID starting with 5dc5864fd2f0165bedfd7908ef41892b3c31f5206dad42bfd32ab0c8e18b3474 not found: ID does not exist"
Sep 07 12:20:58.668648 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:20:58.668617    2563 scope.go:117] "RemoveContainer" containerID="012c5190c29ca3d7ef18f97592291166385bef3c634621f92ef1aecc25d9689d"
Sep 07 12:20:58.668883 ip-10-0-135-173 kubenswrapper[2563]: E0907 12:20:58.668869    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"012c5190c29ca3d7ef18f97592291166385bef3c634621f92ef1aecc25d9689d\": container with ID starting with 012c5190c29ca3d7ef18f97592291166385bef3c634621f92ef1aecc25d9689d not found: ID does not exist" containerID="012c5190c29ca3d7ef18f97592291166385bef3c634621f92ef1aecc25d9689d"
Sep 07 12:20:58.668924 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:20:58.668886    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"012c5190c29ca3d7ef18f97592291166385bef3c634621f92ef1aecc25d9689d"} err="failed to get container status \"012c5190c29ca3d7ef18f97592291166385bef3c634621f92ef1aecc25d9689d\": rpc error: code = NotFound desc = could not find container \"012c5190c29ca3d7ef18f97592291166385bef3c634621f92ef1aecc25d9689d\": container with ID starting with 012c5190c29ca3d7ef18f97592291166385bef3c634621f92ef1aecc25d9689d not found: ID does not exist"
Sep 07 12:20:58.716247 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:20:58.716223    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ada380e3-be47-48af-9fde-9d422ac172ed-credential-socket\") pod \"ada380e3-be47-48af-9fde-9d422ac172ed\" (UID: \"ada380e3-be47-48af-9fde-9d422ac172ed\") "
Sep 07 12:20:58.716335 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:20:58.716291    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ada380e3-be47-48af-9fde-9d422ac172ed-istio-podinfo\") pod \"ada380e3-be47-48af-9fde-9d422ac172ed\" (UID: \"ada380e3-be47-48af-9fde-9d422ac172ed\") "
Sep 07 12:20:58.716335 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:20:58.716315    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ada380e3-be47-48af-9fde-9d422ac172ed-istiod-ca-cert\") pod \"ada380e3-be47-48af-9fde-9d422ac172ed\" (UID: \"ada380e3-be47-48af-9fde-9d422ac172ed\") "
Sep 07 12:20:58.716435 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:20:58.716361    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gnmcz\" (UniqueName: \"kubernetes.io/projected/ada380e3-be47-48af-9fde-9d422ac172ed-kube-api-access-gnmcz\") pod \"ada380e3-be47-48af-9fde-9d422ac172ed\" (UID: \"ada380e3-be47-48af-9fde-9d422ac172ed\") "
Sep 07 12:20:58.716493 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:20:58.716435    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ada380e3-be47-48af-9fde-9d422ac172ed-istio-envoy\") pod \"ada380e3-be47-48af-9fde-9d422ac172ed\" (UID: \"ada380e3-be47-48af-9fde-9d422ac172ed\") "
Sep 07 12:20:58.716493 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:20:58.716462    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ada380e3-be47-48af-9fde-9d422ac172ed-workload-certs\") pod \"ada380e3-be47-48af-9fde-9d422ac172ed\" (UID: \"ada380e3-be47-48af-9fde-9d422ac172ed\") "
Sep 07 12:20:58.716493 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:20:58.716488    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ada380e3-be47-48af-9fde-9d422ac172ed-istio-data\") pod \"ada380e3-be47-48af-9fde-9d422ac172ed\" (UID: \"ada380e3-be47-48af-9fde-9d422ac172ed\") "
Sep 07 12:20:58.716609 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:20:58.716526    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ada380e3-be47-48af-9fde-9d422ac172ed-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "ada380e3-be47-48af-9fde-9d422ac172ed" (UID: "ada380e3-be47-48af-9fde-9d422ac172ed"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 12:20:58.716609 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:20:58.716535    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ada380e3-be47-48af-9fde-9d422ac172ed-istio-token\") pod \"ada380e3-be47-48af-9fde-9d422ac172ed\" (UID: \"ada380e3-be47-48af-9fde-9d422ac172ed\") "
Sep 07 12:20:58.716677 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:20:58.716605    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ada380e3-be47-48af-9fde-9d422ac172ed-workload-socket\") pod \"ada380e3-be47-48af-9fde-9d422ac172ed\" (UID: \"ada380e3-be47-48af-9fde-9d422ac172ed\") "
Sep 07 12:20:58.716754 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:20:58.716724    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ada380e3-be47-48af-9fde-9d422ac172ed-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "ada380e3-be47-48af-9fde-9d422ac172ed" (UID: "ada380e3-be47-48af-9fde-9d422ac172ed"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 07 12:20:58.716952 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:20:58.716923    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ada380e3-be47-48af-9fde-9d422ac172ed-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "ada380e3-be47-48af-9fde-9d422ac172ed" (UID: "ada380e3-be47-48af-9fde-9d422ac172ed"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 12:20:58.717017 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:20:58.716963    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ada380e3-be47-48af-9fde-9d422ac172ed-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "ada380e3-be47-48af-9fde-9d422ac172ed" (UID: "ada380e3-be47-48af-9fde-9d422ac172ed"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 12:20:58.717063 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:20:58.717022    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ada380e3-be47-48af-9fde-9d422ac172ed-istio-data" (OuterVolumeSpecName: "istio-data") pod "ada380e3-be47-48af-9fde-9d422ac172ed" (UID: "ada380e3-be47-48af-9fde-9d422ac172ed"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 12:20:58.717141 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:20:58.717124    2563 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ada380e3-be47-48af-9fde-9d422ac172ed-workload-certs\") on node \"ip-10-0-135-173.ec2.internal\" DevicePath \"\""
Sep 07 12:20:58.717188 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:20:58.717147    2563 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ada380e3-be47-48af-9fde-9d422ac172ed-istio-data\") on node \"ip-10-0-135-173.ec2.internal\" DevicePath \"\""
Sep 07 12:20:58.717188 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:20:58.717161    2563 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ada380e3-be47-48af-9fde-9d422ac172ed-workload-socket\") on node \"ip-10-0-135-173.ec2.internal\" DevicePath \"\""
Sep 07 12:20:58.717188 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:20:58.717175    2563 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ada380e3-be47-48af-9fde-9d422ac172ed-credential-socket\") on node \"ip-10-0-135-173.ec2.internal\" DevicePath \"\""
Sep 07 12:20:58.717300 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:20:58.717188    2563 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ada380e3-be47-48af-9fde-9d422ac172ed-istiod-ca-cert\") on node \"ip-10-0-135-173.ec2.internal\" DevicePath \"\""
Sep 07 12:20:58.718382 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:20:58.718356    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/ada380e3-be47-48af-9fde-9d422ac172ed-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "ada380e3-be47-48af-9fde-9d422ac172ed" (UID: "ada380e3-be47-48af-9fde-9d422ac172ed"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 07 12:20:58.718505 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:20:58.718481    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ada380e3-be47-48af-9fde-9d422ac172ed-kube-api-access-gnmcz" (OuterVolumeSpecName: "kube-api-access-gnmcz") pod "ada380e3-be47-48af-9fde-9d422ac172ed" (UID: "ada380e3-be47-48af-9fde-9d422ac172ed"). InnerVolumeSpecName "kube-api-access-gnmcz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 12:20:58.718852 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:20:58.718830    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ada380e3-be47-48af-9fde-9d422ac172ed-istio-token" (OuterVolumeSpecName: "istio-token") pod "ada380e3-be47-48af-9fde-9d422ac172ed" (UID: "ada380e3-be47-48af-9fde-9d422ac172ed"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 12:20:58.718933 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:20:58.718915    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ada380e3-be47-48af-9fde-9d422ac172ed-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "ada380e3-be47-48af-9fde-9d422ac172ed" (UID: "ada380e3-be47-48af-9fde-9d422ac172ed"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 12:20:58.817815 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:20:58.817791    2563 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ada380e3-be47-48af-9fde-9d422ac172ed-istio-token\") on node \"ip-10-0-135-173.ec2.internal\" DevicePath \"\""
Sep 07 12:20:58.817815 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:20:58.817814    2563 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ada380e3-be47-48af-9fde-9d422ac172ed-istio-podinfo\") on node \"ip-10-0-135-173.ec2.internal\" DevicePath \"\""
Sep 07 12:20:58.817968 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:20:58.817823    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gnmcz\" (UniqueName: \"kubernetes.io/projected/ada380e3-be47-48af-9fde-9d422ac172ed-kube-api-access-gnmcz\") on node \"ip-10-0-135-173.ec2.internal\" DevicePath \"\""
Sep 07 12:20:58.817968 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:20:58.817833    2563 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ada380e3-be47-48af-9fde-9d422ac172ed-istio-envoy\") on node \"ip-10-0-135-173.ec2.internal\" DevicePath \"\""
Sep 07 12:20:58.971766 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:20:58.971741    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-b-3e7ca1-openshift-default-7f74dcbfc6-f79fh"]
Sep 07 12:20:58.975648 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:20:58.975628    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-b-3e7ca1-openshift-default-7f74dcbfc6-f79fh"]
Sep 07 12:20:59.040597 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:20:59.040573    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ada380e3-be47-48af-9fde-9d422ac172ed" path="/var/lib/kubelet/pods/ada380e3-be47-48af-9fde-9d422ac172ed/volumes"
Sep 07 12:21:04.236687 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:21:04.236651    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-491296-6b6d75c56c-2w7xh"]
Sep 07 12:21:04.237203 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:21:04.236874    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-491296-6b6d75c56c-2w7xh" podUID="647bfbc0-36fb-4c24-b1f7-a2531da941cb" containerName="maas-api" containerID="cri-o://67d38edf6325e1248b8f8c0a91849edb791fea1b217c5ccf92793b18538967f2" gracePeriod=30
Sep 07 12:21:04.268817 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:21:04.268787    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:21:04.278494 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:21:04.278473    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-491296-29813055-hn6bf"]
Sep 07 12:21:04.280637 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:21:04.280613    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-491296-29813055-hn6bf"]
Sep 07 12:21:04.369299 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:21:04.369266    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:21:04.468644 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:21:04.468608    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:21:04.480358 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:21:04.480337    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-491296-6b6d75c56c-2w7xh"
Sep 07 12:21:04.565499 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:21:04.565404    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/647bfbc0-36fb-4c24-b1f7-a2531da941cb-maas-api-tls\") pod \"647bfbc0-36fb-4c24-b1f7-a2531da941cb\" (UID: \"647bfbc0-36fb-4c24-b1f7-a2531da941cb\") "
Sep 07 12:21:04.565637 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:21:04.565597    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/647bfbc0-36fb-4c24-b1f7-a2531da941cb-metrics-certs\") pod \"647bfbc0-36fb-4c24-b1f7-a2531da941cb\" (UID: \"647bfbc0-36fb-4c24-b1f7-a2531da941cb\") "
Sep 07 12:21:04.565704 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:21:04.565683    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/647bfbc0-36fb-4c24-b1f7-a2531da941cb-openshift-service-ca\") pod \"647bfbc0-36fb-4c24-b1f7-a2531da941cb\" (UID: \"647bfbc0-36fb-4c24-b1f7-a2531da941cb\") "
Sep 07 12:21:04.565770 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:21:04.565719    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9zj9h\" (UniqueName: \"kubernetes.io/projected/647bfbc0-36fb-4c24-b1f7-a2531da941cb-kube-api-access-9zj9h\") pod \"647bfbc0-36fb-4c24-b1f7-a2531da941cb\" (UID: \"647bfbc0-36fb-4c24-b1f7-a2531da941cb\") "
Sep 07 12:21:04.566052 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:21:04.566026    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/647bfbc0-36fb-4c24-b1f7-a2531da941cb-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "647bfbc0-36fb-4c24-b1f7-a2531da941cb" (UID: "647bfbc0-36fb-4c24-b1f7-a2531da941cb"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 07 12:21:04.567569 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:21:04.567535    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/647bfbc0-36fb-4c24-b1f7-a2531da941cb-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "647bfbc0-36fb-4c24-b1f7-a2531da941cb" (UID: "647bfbc0-36fb-4c24-b1f7-a2531da941cb"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 07 12:21:04.567811 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:21:04.567793    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/647bfbc0-36fb-4c24-b1f7-a2531da941cb-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "647bfbc0-36fb-4c24-b1f7-a2531da941cb" (UID: "647bfbc0-36fb-4c24-b1f7-a2531da941cb"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 07 12:21:04.567874 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:21:04.567855    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/647bfbc0-36fb-4c24-b1f7-a2531da941cb-kube-api-access-9zj9h" (OuterVolumeSpecName: "kube-api-access-9zj9h") pod "647bfbc0-36fb-4c24-b1f7-a2531da941cb" (UID: "647bfbc0-36fb-4c24-b1f7-a2531da941cb"). InnerVolumeSpecName "kube-api-access-9zj9h". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 12:21:04.569332 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:21:04.569301    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:21:04.666947 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:21:04.666919    2563 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/647bfbc0-36fb-4c24-b1f7-a2531da941cb-openshift-service-ca\") on node \"ip-10-0-135-173.ec2.internal\" DevicePath \"\""
Sep 07 12:21:04.666947 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:21:04.666944    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9zj9h\" (UniqueName: \"kubernetes.io/projected/647bfbc0-36fb-4c24-b1f7-a2531da941cb-kube-api-access-9zj9h\") on node \"ip-10-0-135-173.ec2.internal\" DevicePath \"\""
Sep 07 12:21:04.667113 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:21:04.666956    2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/647bfbc0-36fb-4c24-b1f7-a2531da941cb-maas-api-tls\") on node \"ip-10-0-135-173.ec2.internal\" DevicePath \"\""
Sep 07 12:21:04.667113 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:21:04.666968    2563 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/647bfbc0-36fb-4c24-b1f7-a2531da941cb-metrics-certs\") on node \"ip-10-0-135-173.ec2.internal\" DevicePath \"\""
Sep 07 12:21:04.685736 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:21:04.683321    2563 generic.go:356] "Generic (PLEG): container finished" podID="647bfbc0-36fb-4c24-b1f7-a2531da941cb" containerID="67d38edf6325e1248b8f8c0a91849edb791fea1b217c5ccf92793b18538967f2" exitCode=0
Sep 07 12:21:04.685736 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:21:04.683487    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-491296-6b6d75c56c-2w7xh" event={"ID":"647bfbc0-36fb-4c24-b1f7-a2531da941cb","Type":"ContainerDied","Data":"67d38edf6325e1248b8f8c0a91849edb791fea1b217c5ccf92793b18538967f2"}
Sep 07 12:21:04.685736 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:21:04.683539    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-491296-6b6d75c56c-2w7xh" event={"ID":"647bfbc0-36fb-4c24-b1f7-a2531da941cb","Type":"ContainerDied","Data":"cac8add44ab99862a440abad746affbd869bd6d2d6d8ff9b91c6ebd6998cf7e0"}
Sep 07 12:21:04.685736 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:21:04.683567    2563 scope.go:117] "RemoveContainer" containerID="67d38edf6325e1248b8f8c0a91849edb791fea1b217c5ccf92793b18538967f2"
Sep 07 12:21:04.685736 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:21:04.683698    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-491296-6b6d75c56c-2w7xh"
Sep 07 12:21:04.695020 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:21:04.695003    2563 scope.go:117] "RemoveContainer" containerID="67d38edf6325e1248b8f8c0a91849edb791fea1b217c5ccf92793b18538967f2"
Sep 07 12:21:04.695279 ip-10-0-135-173 kubenswrapper[2563]: E0907 12:21:04.695261    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"67d38edf6325e1248b8f8c0a91849edb791fea1b217c5ccf92793b18538967f2\": container with ID starting with 67d38edf6325e1248b8f8c0a91849edb791fea1b217c5ccf92793b18538967f2 not found: ID does not exist" containerID="67d38edf6325e1248b8f8c0a91849edb791fea1b217c5ccf92793b18538967f2"
Sep 07 12:21:04.695345 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:21:04.695286    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"67d38edf6325e1248b8f8c0a91849edb791fea1b217c5ccf92793b18538967f2"} err="failed to get container status \"67d38edf6325e1248b8f8c0a91849edb791fea1b217c5ccf92793b18538967f2\": rpc error: code = NotFound desc = could not find container \"67d38edf6325e1248b8f8c0a91849edb791fea1b217c5ccf92793b18538967f2\": container with ID starting with 67d38edf6325e1248b8f8c0a91849edb791fea1b217c5ccf92793b18538967f2 not found: ID does not exist"
Sep 07 12:21:04.703647 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:21:04.703621    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-491296-6b6d75c56c-2w7xh"]
Sep 07 12:21:04.707276 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:21:04.707253    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-491296-6b6d75c56c-2w7xh"]
Sep 07 12:21:04.973324 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:21:04.973295    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:21:05.040306 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:21:05.040277    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0c7cdf98-fe7f-470a-82d0-cd71f862e8b0" path="/var/lib/kubelet/pods/0c7cdf98-fe7f-470a-82d0-cd71f862e8b0/volumes"
Sep 07 12:21:05.040646 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:21:05.040633    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="647bfbc0-36fb-4c24-b1f7-a2531da941cb" path="/var/lib/kubelet/pods/647bfbc0-36fb-4c24-b1f7-a2531da941cb/volumes"
Sep 07 12:21:08.511707 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:21:08.511675    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-p4l4w"]
Sep 07 12:21:09.217142 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:21:09.217112    2563 scope.go:117] "RemoveContainer" containerID="b0e2bd4f864221dec8a65188d0c62da75f5a605711e6ef1dd163a92fc8be73a3"
Sep 07 12:21:10.825587 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:21:10.825541    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:21:10.923750 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:21:10.923711    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:21:11.157122 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:21:11.157043    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:21:21.448593 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:21:21.448558    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-613886-hc9qk"]
Sep 07 12:21:21.448970 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:21:21.448904    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ada380e3-be47-48af-9fde-9d422ac172ed" containerName="istio-proxy"
Sep 07 12:21:21.448970 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:21:21.448915    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="ada380e3-be47-48af-9fde-9d422ac172ed" containerName="istio-proxy"
Sep 07 12:21:21.448970 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:21:21.448926    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a0c5913e-5a31-47c6-b62d-cca418880291" containerName="istio-proxy"
Sep 07 12:21:21.448970 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:21:21.448931    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="a0c5913e-5a31-47c6-b62d-cca418880291" containerName="istio-proxy"
Sep 07 12:21:21.448970 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:21:21.448939    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="647bfbc0-36fb-4c24-b1f7-a2531da941cb" containerName="maas-api"
Sep 07 12:21:21.448970 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:21:21.448944    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="647bfbc0-36fb-4c24-b1f7-a2531da941cb" containerName="maas-api"
Sep 07 12:21:21.448970 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:21:21.448951    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5f741d59-c73c-48b8-a6d5-887ee0467484" containerName="maas-api"
Sep 07 12:21:21.448970 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:21:21.448958    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="5f741d59-c73c-48b8-a6d5-887ee0467484" containerName="maas-api"
Sep 07 12:21:21.448970 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:21:21.448971    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fd792abf-4178-43c0-b4f0-94df7367bff3" containerName="main"
Sep 07 12:21:21.448970 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:21:21.448976    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="fd792abf-4178-43c0-b4f0-94df7367bff3" containerName="main"
Sep 07 12:21:21.449283 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:21:21.448986    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a0c5913e-5a31-47c6-b62d-cca418880291" containerName="istio-proxy"
Sep 07 12:21:21.449283 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:21:21.448991    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="a0c5913e-5a31-47c6-b62d-cca418880291" containerName="istio-proxy"
Sep 07 12:21:21.449283 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:21:21.449000    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f2486ed1-45b8-49ec-ad3c-82e6094aeee4" containerName="istio-proxy"
Sep 07 12:21:21.449283 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:21:21.449006    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="f2486ed1-45b8-49ec-ad3c-82e6094aeee4" containerName="istio-proxy"
Sep 07 12:21:21.449283 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:21:21.449012    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f2486ed1-45b8-49ec-ad3c-82e6094aeee4" containerName="istio-proxy"
Sep 07 12:21:21.449283 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:21:21.449017    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="f2486ed1-45b8-49ec-ad3c-82e6094aeee4" containerName="istio-proxy"
Sep 07 12:21:21.449283 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:21:21.449070    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="ada380e3-be47-48af-9fde-9d422ac172ed" containerName="istio-proxy"
Sep 07 12:21:21.449283 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:21:21.449075    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="5f741d59-c73c-48b8-a6d5-887ee0467484" containerName="maas-api"
Sep 07 12:21:21.449283 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:21:21.449082    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="ada380e3-be47-48af-9fde-9d422ac172ed" containerName="istio-proxy"
Sep 07 12:21:21.449283 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:21:21.449088    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="fd792abf-4178-43c0-b4f0-94df7367bff3" containerName="main"
Sep 07 12:21:21.449283 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:21:21.449094    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="a0c5913e-5a31-47c6-b62d-cca418880291" containerName="istio-proxy"
Sep 07 12:21:21.449283 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:21:21.449100    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="647bfbc0-36fb-4c24-b1f7-a2531da941cb" containerName="maas-api"
Sep 07 12:21:21.449283 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:21:21.449106    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="f2486ed1-45b8-49ec-ad3c-82e6094aeee4" containerName="istio-proxy"
Sep 07 12:21:21.449283 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:21:21.449111    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="f2486ed1-45b8-49ec-ad3c-82e6094aeee4" containerName="istio-proxy"
Sep 07 12:21:21.451970 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:21:21.451954    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-613886-hc9qk"
Sep 07 12:21:21.453704 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:21:21.453681    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-54hhj\""
Sep 07 12:21:21.453851 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:21:21.453682    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 07 12:21:21.458898 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:21:21.458872    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-613886-hc9qk"]
Sep 07 12:21:21.509643 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:21:21.509616    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5d47cd30-3b7f-408a-ba18-d29e4affd354-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w3-613886-hc9qk\" (UID: \"5d47cd30-3b7f-408a-ba18-d29e4affd354\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-613886-hc9qk"
Sep 07 12:21:21.610925 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:21:21.610894    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5d47cd30-3b7f-408a-ba18-d29e4affd354-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w3-613886-hc9qk\" (UID: \"5d47cd30-3b7f-408a-ba18-d29e4affd354\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-613886-hc9qk"
Sep 07 12:21:21.611506 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:21:21.611488    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5d47cd30-3b7f-408a-ba18-d29e4affd354-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w3-613886-hc9qk\" (UID: \"5d47cd30-3b7f-408a-ba18-d29e4affd354\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-613886-hc9qk"
Sep 07 12:21:21.731845 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:21:21.731771    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:21:21.763111 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:21:21.763087    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-613886-hc9qk"
Sep 07 12:21:22.033607 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:21:22.033498    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-613886-hc9qk"]
Sep 07 12:21:22.036523 ip-10-0-135-173 kubenswrapper[2563]: W0907 12:21:22.036493    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5d47cd30_3b7f_408a_ba18_d29e4affd354.slice/crio-db5071493a2da24181925c73a113597fb983b31a9b94f14aecf0845fa58de132 WatchSource:0}: Error finding container db5071493a2da24181925c73a113597fb983b31a9b94f14aecf0845fa58de132: Status 404 returned error can't find the container with id db5071493a2da24181925c73a113597fb983b31a9b94f14aecf0845fa58de132
Sep 07 12:21:22.767965 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:21:22.767933    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-613886-hc9qk" event={"ID":"5d47cd30-3b7f-408a-ba18-d29e4affd354","Type":"ContainerStarted","Data":"776aef9a45d4d28be55ab6dd7c2f74e72f557302aa9dd6fb1355858daae31f0f"}
Sep 07 12:21:22.768388 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:21:22.767982    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-613886-hc9qk" event={"ID":"5d47cd30-3b7f-408a-ba18-d29e4affd354","Type":"ContainerStarted","Data":"db5071493a2da24181925c73a113597fb983b31a9b94f14aecf0845fa58de132"}
Sep 07 12:21:22.781664 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:21:22.781622    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-613886-hc9qk" podStartSLOduration=1.7816066510000002 podStartE2EDuration="1.781606651s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 12:21:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 12:21:22.779904777 +0000 UTC m=+2054.298636590" watchObservedRunningTime="2026-09-07 12:21:22.781606651 +0000 UTC m=+2054.300338464"
Sep 07 12:21:23.774709 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:21:23.774672    2563 generic.go:356] "Generic (PLEG): container finished" podID="5d47cd30-3b7f-408a-ba18-d29e4affd354" containerID="776aef9a45d4d28be55ab6dd7c2f74e72f557302aa9dd6fb1355858daae31f0f" exitCode=0
Sep 07 12:21:23.775133 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:21:23.774720    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-613886-hc9qk" event={"ID":"5d47cd30-3b7f-408a-ba18-d29e4affd354","Type":"ContainerDied","Data":"776aef9a45d4d28be55ab6dd7c2f74e72f557302aa9dd6fb1355858daae31f0f"}
Sep 07 12:21:24.828357 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:21:24.828320    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:21:24.903461 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:21:24.903435    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-613886-hc9qk"
Sep 07 12:21:25.043090 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:21:25.043017    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5d47cd30-3b7f-408a-ba18-d29e4affd354-maas-api-service-ca\") pod \"5d47cd30-3b7f-408a-ba18-d29e4affd354\" (UID: \"5d47cd30-3b7f-408a-ba18-d29e4affd354\") "
Sep 07 12:21:25.043346 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:21:25.043325    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5d47cd30-3b7f-408a-ba18-d29e4affd354-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "5d47cd30-3b7f-408a-ba18-d29e4affd354" (UID: "5d47cd30-3b7f-408a-ba18-d29e4affd354"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 07 12:21:25.144159 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:21:25.144126    2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5d47cd30-3b7f-408a-ba18-d29e4affd354-maas-api-service-ca\") on node \"ip-10-0-135-173.ec2.internal\" DevicePath \"\""
Sep 07 12:21:25.784967 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:21:25.784934    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-613886-hc9qk"
Sep 07 12:21:25.784967 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:21:25.784955    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-613886-hc9qk" event={"ID":"5d47cd30-3b7f-408a-ba18-d29e4affd354","Type":"ContainerDied","Data":"db5071493a2da24181925c73a113597fb983b31a9b94f14aecf0845fa58de132"}
Sep 07 12:21:25.785181 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:21:25.785022    2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="db5071493a2da24181925c73a113597fb983b31a9b94f14aecf0845fa58de132"
Sep 07 12:21:31.681224 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:21:31.681140    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:21:32.153895 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:21:32.153857    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:21:32.253596 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:21:32.253553    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:21:32.322652 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:21:32.322619    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-613886-hc9qk"]
Sep 07 12:21:32.326849 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:21:32.326815    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:21:32.328472 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:21:32.328447    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-613886-hc9qk"]
Sep 07 12:21:32.380212 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:21:32.380181    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:21:32.719767 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:21:32.719729    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:21:33.039948 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:21:33.039865    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5d47cd30-3b7f-408a-ba18-d29e4affd354" path="/var/lib/kubelet/pods/5d47cd30-3b7f-408a-ba18-d29e4affd354/volumes"
Sep 07 12:21:34.897406 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:21:34.897374    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-p4l4w"]
Sep 07 12:21:38.593233 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:21:38.593190    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:21:38.653811 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:21:38.653783    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:21:38.856877 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:21:38.856794    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:21:43.203205 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:21:43.203169    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-p4l4w"]
Sep 07 12:22:05.465633 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:22:05.465599    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:22:07.157599 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:22:07.157561    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:22:07.258699 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:22:07.258665    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:22:09.272128 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:22:09.272097    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-4nfbc_b34ca85d-ac51-4724-a508-1026106b3f3a/console-operator/2.log"
Sep 07 12:22:09.276505 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:22:09.276484    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-4nfbc_b34ca85d-ac51-4724-a508-1026106b3f3a/console-operator/2.log"
Sep 07 12:22:39.620105 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:22:39.620066    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:22:44.918872 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:22:44.918828    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:22:49.648658 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:22:49.648621    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:22:53.278844 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:22:53.278809    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:22:55.146145 ip-10-0-135-173 kubenswrapper[2563]: E0907 12:22:55.146099    2563 kubelet_node_status.go:536] "Error updating node status, will retry" err="failed to patch status \"{\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"MemoryPressure\\\"},{\\\"type\\\":\\\"DiskPressure\\\"},{\\\"type\\\":\\\"PIDPressure\\\"},{\\\"type\\\":\\\"Ready\\\"}],\\\"conditions\\\":[{\\\"lastHeartbeatTime\\\":\\\"2026-09-07T12:22:55Z\\\",\\\"type\\\":\\\"MemoryPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-09-07T12:22:55Z\\\",\\\"type\\\":\\\"DiskPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-09-07T12:22:55Z\\\",\\\"type\\\":\\\"PIDPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-09-07T12:22:55Z\\\",\\\"type\\\":\\\"Ready\\\"}],\\\"runtimeHandlers\\\":[{\\\"features\\\":{\\\"recursiveReadOnlyMounts\\\":true,\\\"userNamespaces\\\":true},\\\"name\\\":\\\"runc\\\"},{\\\"features\\\":{\\\"recursiveReadOnlyMounts\\\":true,\\\"userNamespaces\\\":true},\\\"name\\\":\\\"crun\\\"},{\\\"features\\\":{\\\"recursiveReadOnlyMounts\\\":true,\\\"userNamespaces\\\":true},\\\"name\\\":\\\"\\\"}]}}\" for node \"ip-10-0-135-173.ec2.internal\": Internal error occurred: failed calling webhook \"node.network-node-identity.openshift.io\": failed to call webhook: Post \"https://network-node-identity.clusters-70737425-554b-4289-aebf-e53ca32b3bc0.svc:9743/node?timeout=10s\": dial tcp 172.30.186.39:9743: connect: connection refused"
Sep 07 12:22:55.159462 ip-10-0-135-173 kubenswrapper[2563]: E0907 12:22:55.159411    2563 kubelet_node_status.go:536] "Error updating node status, will retry" err="failed to patch status \"{\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"MemoryPressure\\\"},{\\\"type\\\":\\\"DiskPressure\\\"},{\\\"type\\\":\\\"PIDPressure\\\"},{\\\"type\\\":\\\"Ready\\\"}],\\\"conditions\\\":[{\\\"lastHeartbeatTime\\\":\\\"2026-09-07T12:22:55Z\\\",\\\"type\\\":\\\"MemoryPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-09-07T12:22:55Z\\\",\\\"type\\\":\\\"DiskPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-09-07T12:22:55Z\\\",\\\"type\\\":\\\"PIDPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-09-07T12:22:55Z\\\",\\\"type\\\":\\\"Ready\\\"}],\\\"runtimeHandlers\\\":[{\\\"features\\\":{\\\"recursiveReadOnlyMounts\\\":true,\\\"userNamespaces\\\":true},\\\"name\\\":\\\"runc\\\"},{\\\"features\\\":{\\\"recursiveReadOnlyMounts\\\":true,\\\"userNamespaces\\\":true},\\\"name\\\":\\\"crun\\\"},{\\\"features\\\":{\\\"recursiveReadOnlyMounts\\\":true,\\\"userNamespaces\\\":true},\\\"name\\\":\\\"\\\"}]}}\" for node \"ip-10-0-135-173.ec2.internal\": Internal error occurred: failed calling webhook \"node.network-node-identity.openshift.io\": failed to call webhook: Post \"https://network-node-identity.clusters-70737425-554b-4289-aebf-e53ca32b3bc0.svc:9743/node?timeout=10s\": dial tcp 172.30.186.39:9743: connect: connection refused"
Sep 07 12:22:55.170054 ip-10-0-135-173 kubenswrapper[2563]: E0907 12:22:55.170026    2563 kubelet_node_status.go:536] "Error updating node status, will retry" err="failed to patch status \"{\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"MemoryPressure\\\"},{\\\"type\\\":\\\"DiskPressure\\\"},{\\\"type\\\":\\\"PIDPressure\\\"},{\\\"type\\\":\\\"Ready\\\"}],\\\"conditions\\\":[{\\\"lastHeartbeatTime\\\":\\\"2026-09-07T12:22:55Z\\\",\\\"type\\\":\\\"MemoryPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-09-07T12:22:55Z\\\",\\\"type\\\":\\\"DiskPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-09-07T12:22:55Z\\\",\\\"type\\\":\\\"PIDPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-09-07T12:22:55Z\\\",\\\"type\\\":\\\"Ready\\\"}],\\\"runtimeHandlers\\\":[{\\\"features\\\":{\\\"recursiveReadOnlyMounts\\\":true,\\\"userNamespaces\\\":true},\\\"name\\\":\\\"runc\\\"},{\\\"features\\\":{\\\"recursiveReadOnlyMounts\\\":true,\\\"userNamespaces\\\":true},\\\"name\\\":\\\"crun\\\"},{\\\"features\\\":{\\\"recursiveReadOnlyMounts\\\":true,\\\"userNamespaces\\\":true},\\\"name\\\":\\\"\\\"}]}}\" for node \"ip-10-0-135-173.ec2.internal\": Internal error occurred: failed calling webhook \"node.network-node-identity.openshift.io\": failed to call webhook: Post \"https://network-node-identity.clusters-70737425-554b-4289-aebf-e53ca32b3bc0.svc:9743/node?timeout=10s\": dial tcp 172.30.186.39:9743: connect: connection refused"
Sep 07 12:22:55.180756 ip-10-0-135-173 kubenswrapper[2563]: E0907 12:22:55.180730    2563 kubelet_node_status.go:536] "Error updating node status, will retry" err="failed to patch status \"{\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"MemoryPressure\\\"},{\\\"type\\\":\\\"DiskPressure\\\"},{\\\"type\\\":\\\"PIDPressure\\\"},{\\\"type\\\":\\\"Ready\\\"}],\\\"conditions\\\":[{\\\"lastHeartbeatTime\\\":\\\"2026-09-07T12:22:55Z\\\",\\\"type\\\":\\\"MemoryPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-09-07T12:22:55Z\\\",\\\"type\\\":\\\"DiskPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-09-07T12:22:55Z\\\",\\\"type\\\":\\\"PIDPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-09-07T12:22:55Z\\\",\\\"type\\\":\\\"Ready\\\"}],\\\"runtimeHandlers\\\":[{\\\"features\\\":{\\\"recursiveReadOnlyMounts\\\":true,\\\"userNamespaces\\\":true},\\\"name\\\":\\\"runc\\\"},{\\\"features\\\":{\\\"recursiveReadOnlyMounts\\\":true,\\\"userNamespaces\\\":true},\\\"name\\\":\\\"crun\\\"},{\\\"features\\\":{\\\"recursiveReadOnlyMounts\\\":true,\\\"userNamespaces\\\":true},\\\"name\\\":\\\"\\\"}]}}\" for node \"ip-10-0-135-173.ec2.internal\": Internal error occurred: failed calling webhook \"node.network-node-identity.openshift.io\": failed to call webhook: Post \"https://network-node-identity.clusters-70737425-554b-4289-aebf-e53ca32b3bc0.svc:9743/node?timeout=10s\": dial tcp 172.30.186.39:9743: connect: connection refused"
Sep 07 12:22:55.191155 ip-10-0-135-173 kubenswrapper[2563]: E0907 12:22:55.191128    2563 kubelet_node_status.go:536] "Error updating node status, will retry" err="failed to patch status \"{\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"MemoryPressure\\\"},{\\\"type\\\":\\\"DiskPressure\\\"},{\\\"type\\\":\\\"PIDPressure\\\"},{\\\"type\\\":\\\"Ready\\\"}],\\\"conditions\\\":[{\\\"lastHeartbeatTime\\\":\\\"2026-09-07T12:22:55Z\\\",\\\"type\\\":\\\"MemoryPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-09-07T12:22:55Z\\\",\\\"type\\\":\\\"DiskPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-09-07T12:22:55Z\\\",\\\"type\\\":\\\"PIDPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-09-07T12:22:55Z\\\",\\\"type\\\":\\\"Ready\\\"}],\\\"runtimeHandlers\\\":[{\\\"features\\\":{\\\"recursiveReadOnlyMounts\\\":true,\\\"userNamespaces\\\":true},\\\"name\\\":\\\"runc\\\"},{\\\"features\\\":{\\\"recursiveReadOnlyMounts\\\":true,\\\"userNamespaces\\\":true},\\\"name\\\":\\\"crun\\\"},{\\\"features\\\":{\\\"recursiveReadOnlyMounts\\\":true,\\\"userNamespaces\\\":true},\\\"name\\\":\\\"\\\"}]}}\" for node \"ip-10-0-135-173.ec2.internal\": Internal error occurred: failed calling webhook \"node.network-node-identity.openshift.io\": failed to call webhook: Post \"https://network-node-identity.clusters-70737425-554b-4289-aebf-e53ca32b3bc0.svc:9743/node?timeout=10s\": dial tcp 172.30.186.39:9743: connect: connection refused"
Sep 07 12:22:55.191155 ip-10-0-135-173 kubenswrapper[2563]: E0907 12:22:55.191155    2563 kubelet_node_status.go:523] "Unable to update node status" err="update node status exceeds retry count"
Sep 07 12:22:55.607477 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:22:55.607412    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:22:58.450024 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:22:58.449987    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:23:01.038719 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:23:01.038676    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:23:01.139454 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:23:01.139361    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:23:01.202768 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:23:01.202739    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:23:03.252095 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:23:03.252057    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:23:03.951670 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:23:03.951638    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-multus/cni-sysctl-allowlist-ds-nkkc9"]
Sep 07 12:23:03.952009 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:23:03.951996    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ada380e3-be47-48af-9fde-9d422ac172ed" containerName="istio-proxy"
Sep 07 12:23:03.952063 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:23:03.952011    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="ada380e3-be47-48af-9fde-9d422ac172ed" containerName="istio-proxy"
Sep 07 12:23:03.952063 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:23:03.952025    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5d47cd30-3b7f-408a-ba18-d29e4affd354" containerName="revoke-keys"
Sep 07 12:23:03.952063 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:23:03.952031    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="5d47cd30-3b7f-408a-ba18-d29e4affd354" containerName="revoke-keys"
Sep 07 12:23:03.952176 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:23:03.952093    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="a0c5913e-5a31-47c6-b62d-cca418880291" containerName="istio-proxy"
Sep 07 12:23:03.952176 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:23:03.952102    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="5d47cd30-3b7f-408a-ba18-d29e4affd354" containerName="revoke-keys"
Sep 07 12:23:03.955038 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:23:03.955019    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/cni-sysctl-allowlist-ds-nkkc9"
Sep 07 12:23:03.956799 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:23:03.956780    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-sysctl-allowlist\""
Sep 07 12:23:03.967404 ip-10-0-135-173 kubenswrapper[2563]: E0907 12:23:03.967361    2563 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"d85226d2-5ce5-4ffb-95b6-eeed0206a352\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"lastProbeTime\\\":null,\\\"lastTransitionTime\\\":\\\"2026-09-07T12:23:03Z\\\",\\\"observedGeneration\\\":1,\\\"status\\\":\\\"False\\\",\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"lastProbeTime\\\":null,\\\"lastTransitionTime\\\":\\\"2026-09-07T12:23:03Z\\\",\\\"observedGeneration\\\":1,\\\"status\\\":\\\"True\\\",\\\"type\\\":\\\"Initialized\\\"},{\\\"lastProbeTime\\\":null,\\\"lastTransitionTime\\\":\\\"2026-09-07T12:23:03Z\\\",\\\"message\\\":\\\"containers with unready status: [kube-multus-additional-cni-plugins]\\\",\\\"observedGeneration\\\":1,\\\"reason\\\":\\\"ContainersNotReady\\\",\\\"status\\\":\\\"False\\\",\\\"type\\\":\\\"Ready\\\"},{\\\"lastProbeTime\\\":null,\\\"lastTransitionTime\\\":\\\"2026-09-07T12:23:03Z\\\",\\\"message\\\":\\\"containers with unready status: [kube-multus-additional-cni-plugins]\\\",\\\"observedGeneration\\\":1,\\\"reason\\\":\\\"ContainersNotReady\\\",\\\"status\\\":\\\"False\\\",\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":1,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"containerStatuses\\\":[{\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:cfde312743ce0a1d3f82fc68e67e64a3fca57853ed9c621b1fe74db36e5943ea\\\",\\\"imageID\\\":\\\"\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"kube-multus-additional-cni-plugins\\\",\\\"ready\\\":false,\\\"restartCount\\\":0,\\\"started\\\":false,\\\"state\\\":{\\\"waiting\\\":{\\\"reason\\\":\\\"ContainerCreating\\\"}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/entrypoint\\\",\\\"name\\\":\\\"cni-sysctl-allowlist\\\"},{\\\"mountPath\\\":\\\"/host/etc/cni/tuning/\\\",\\\"name\\\":\\\"tuning-conf-dir\\\"},{\\\"mountPath\\\":\\\"/ready\\\",\\\"name\\\":\\\"ready\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-sgwhc\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]}],\\\"hostIP\\\":\\\"10.0.135.173\\\",\\\"hostIPs\\\":[{\\\"ip\\\":\\\"10.0.135.173\\\"}],\\\"observedGeneration\\\":1,\\\"podIP\\\":\\\"10.0.135.173\\\",\\\"podIPs\\\":[{\\\"ip\\\":\\\"10.0.135.173\\\"}],\\\"startTime\\\":\\\"2026-09-07T12:23:03Z\\\"}}\" for pod \"openshift-multus\"/\"cni-sysctl-allowlist-ds-nkkc9\": Internal error occurred: failed calling webhook \"pod.network-node-identity.openshift.io\": failed to call webhook: Post \"https://network-node-identity.clusters-70737425-554b-4289-aebf-e53ca32b3bc0.svc:9743/pod?timeout=10s\": dial tcp 172.30.186.39:9743: connect: connection refused" pod="openshift-multus/cni-sysctl-allowlist-ds-nkkc9"
Sep 07 12:23:03.985865 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:23:03.985840    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:23:04.085481 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:23:04.085450    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ready\" (UniqueName: \"kubernetes.io/empty-dir/d85226d2-5ce5-4ffb-95b6-eeed0206a352-ready\") pod \"cni-sysctl-allowlist-ds-nkkc9\" (UID: \"d85226d2-5ce5-4ffb-95b6-eeed0206a352\") " pod="openshift-multus/cni-sysctl-allowlist-ds-nkkc9"
Sep 07 12:23:04.085619 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:23:04.085493    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/d85226d2-5ce5-4ffb-95b6-eeed0206a352-cni-sysctl-allowlist\") pod \"cni-sysctl-allowlist-ds-nkkc9\" (UID: \"d85226d2-5ce5-4ffb-95b6-eeed0206a352\") " pod="openshift-multus/cni-sysctl-allowlist-ds-nkkc9"
Sep 07 12:23:04.085619 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:23:04.085611    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sgwhc\" (UniqueName: \"kubernetes.io/projected/d85226d2-5ce5-4ffb-95b6-eeed0206a352-kube-api-access-sgwhc\") pod \"cni-sysctl-allowlist-ds-nkkc9\" (UID: \"d85226d2-5ce5-4ffb-95b6-eeed0206a352\") " pod="openshift-multus/cni-sysctl-allowlist-ds-nkkc9"
Sep 07 12:23:04.085733 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:23:04.085715    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/d85226d2-5ce5-4ffb-95b6-eeed0206a352-tuning-conf-dir\") pod \"cni-sysctl-allowlist-ds-nkkc9\" (UID: \"d85226d2-5ce5-4ffb-95b6-eeed0206a352\") " pod="openshift-multus/cni-sysctl-allowlist-ds-nkkc9"
Sep 07 12:23:04.086356 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:23:04.086324    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:23:04.186265 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:23:04.186236    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/d85226d2-5ce5-4ffb-95b6-eeed0206a352-tuning-conf-dir\") pod \"cni-sysctl-allowlist-ds-nkkc9\" (UID: \"d85226d2-5ce5-4ffb-95b6-eeed0206a352\") " pod="openshift-multus/cni-sysctl-allowlist-ds-nkkc9"
Sep 07 12:23:04.186444 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:23:04.186278    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ready\" (UniqueName: \"kubernetes.io/empty-dir/d85226d2-5ce5-4ffb-95b6-eeed0206a352-ready\") pod \"cni-sysctl-allowlist-ds-nkkc9\" (UID: \"d85226d2-5ce5-4ffb-95b6-eeed0206a352\") " pod="openshift-multus/cni-sysctl-allowlist-ds-nkkc9"
Sep 07 12:23:04.186444 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:23:04.186318    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/d85226d2-5ce5-4ffb-95b6-eeed0206a352-cni-sysctl-allowlist\") pod \"cni-sysctl-allowlist-ds-nkkc9\" (UID: \"d85226d2-5ce5-4ffb-95b6-eeed0206a352\") " pod="openshift-multus/cni-sysctl-allowlist-ds-nkkc9"
Sep 07 12:23:04.186444 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:23:04.186360    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/d85226d2-5ce5-4ffb-95b6-eeed0206a352-tuning-conf-dir\") pod \"cni-sysctl-allowlist-ds-nkkc9\" (UID: \"d85226d2-5ce5-4ffb-95b6-eeed0206a352\") " pod="openshift-multus/cni-sysctl-allowlist-ds-nkkc9"
Sep 07 12:23:04.186444 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:23:04.186378    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sgwhc\" (UniqueName: \"kubernetes.io/projected/d85226d2-5ce5-4ffb-95b6-eeed0206a352-kube-api-access-sgwhc\") pod \"cni-sysctl-allowlist-ds-nkkc9\" (UID: \"d85226d2-5ce5-4ffb-95b6-eeed0206a352\") " pod="openshift-multus/cni-sysctl-allowlist-ds-nkkc9"
Sep 07 12:23:04.186844 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:23:04.186807    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ready\" (UniqueName: \"kubernetes.io/empty-dir/d85226d2-5ce5-4ffb-95b6-eeed0206a352-ready\") pod \"cni-sysctl-allowlist-ds-nkkc9\" (UID: \"d85226d2-5ce5-4ffb-95b6-eeed0206a352\") " pod="openshift-multus/cni-sysctl-allowlist-ds-nkkc9"
Sep 07 12:23:04.186971 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:23:04.186946    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:23:04.187027 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:23:04.186999    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/d85226d2-5ce5-4ffb-95b6-eeed0206a352-cni-sysctl-allowlist\") pod \"cni-sysctl-allowlist-ds-nkkc9\" (UID: \"d85226d2-5ce5-4ffb-95b6-eeed0206a352\") " pod="openshift-multus/cni-sysctl-allowlist-ds-nkkc9"
Sep 07 12:23:04.192886 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:23:04.192854    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sgwhc\" (UniqueName: \"kubernetes.io/projected/d85226d2-5ce5-4ffb-95b6-eeed0206a352-kube-api-access-sgwhc\") pod \"cni-sysctl-allowlist-ds-nkkc9\" (UID: \"d85226d2-5ce5-4ffb-95b6-eeed0206a352\") " pod="openshift-multus/cni-sysctl-allowlist-ds-nkkc9"
Sep 07 12:23:04.264668 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:23:04.264612    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/cni-sysctl-allowlist-ds-nkkc9"
Sep 07 12:23:04.277546 ip-10-0-135-173 kubenswrapper[2563]: W0907 12:23:04.277524    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd85226d2_5ce5_4ffb_95b6_eeed0206a352.slice/crio-19bb5417a4dbb31da1bb4090c494616da3d496003d9fcf2828838204ef919241 WatchSource:0}: Error finding container 19bb5417a4dbb31da1bb4090c494616da3d496003d9fcf2828838204ef919241: Status 404 returned error can't find the container with id 19bb5417a4dbb31da1bb4090c494616da3d496003d9fcf2828838204ef919241
Sep 07 12:23:04.287468 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:23:04.287438    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:23:05.247851 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:23:05.247821    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/cni-sysctl-allowlist-ds-nkkc9" event={"ID":"d85226d2-5ce5-4ffb-95b6-eeed0206a352","Type":"ContainerStarted","Data":"5942f6c25d1daae4944c44b3a6d7151700de35122ba904be663526b900106565"}
Sep 07 12:23:05.248006 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:23:05.247869    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/cni-sysctl-allowlist-ds-nkkc9" event={"ID":"d85226d2-5ce5-4ffb-95b6-eeed0206a352","Type":"ContainerStarted","Data":"19bb5417a4dbb31da1bb4090c494616da3d496003d9fcf2828838204ef919241"}
Sep 07 12:23:05.248090 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:23:05.248072    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-multus/cni-sysctl-allowlist-ds-nkkc9"
Sep 07 12:23:05.256947 ip-10-0-135-173 kubenswrapper[2563]: E0907 12:23:05.256872    2563 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"d85226d2-5ce5-4ffb-95b6-eeed0206a352\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"lastProbeTime\\\":null,\\\"lastTransitionTime\\\":\\\"2026-09-07T12:23:05Z\\\",\\\"observedGeneration\\\":1,\\\"status\\\":\\\"True\\\",\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"lastProbeTime\\\":null,\\\"lastTransitionTime\\\":\\\"2026-09-07T12:23:03Z\\\",\\\"observedGeneration\\\":1,\\\"status\\\":\\\"True\\\",\\\"type\\\":\\\"Initialized\\\"},{\\\"lastProbeTime\\\":null,\\\"lastTransitionTime\\\":\\\"2026-09-07T12:23:03Z\\\",\\\"message\\\":\\\"containers with unready status: [kube-multus-additional-cni-plugins]\\\",\\\"observedGeneration\\\":1,\\\"reason\\\":\\\"ContainersNotReady\\\",\\\"status\\\":\\\"False\\\",\\\"type\\\":\\\"Ready\\\"},{\\\"lastProbeTime\\\":null,\\\"lastTransitionTime\\\":\\\"2026-09-07T12:23:03Z\\\",\\\"message\\\":\\\"containers with unready status: [kube-multus-additional-cni-plugins]\\\",\\\"observedGeneration\\\":1,\\\"reason\\\":\\\"ContainersNotReady\\\",\\\"status\\\":\\\"False\\\",\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":1,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"containerStatuses\\\":[{\\\"allocatedResources\\\":{\\\"cpu\\\":\\\"10m\\\",\\\"memory\\\":\\\"10Mi\\\"},\\\"containerID\\\":\\\"cri-o://5942f6c25d1daae4944c44b3a6d7151700de35122ba904be663526b900106565\\\",\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:cfde312743ce0a1d3f82fc68e67e64a3fca57853ed9c621b1fe74db36e5943ea\\\",\\\"imageID\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:cfde312743ce0a1d3f82fc68e67e64a3fca57853ed9c621b1fe74db36e5943ea\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"kube-multus-additional-cni-plugins\\\",\\\"ready\\\":false,\\\"resources\\\":{\\\"requests\\\":{\\\"cpu\\\":\\\"10m\\\",\\\"memory\\\":\\\"10Mi\\\"}},\\\"restartCount\\\":0,\\\"started\\\":true,\\\"state\\\":{\\\"running\\\":{\\\"startedAt\\\":\\\"2026-09-07T12:23:04Z\\\"}},\\\"user\\\":{\\\"linux\\\":{\\\"gid\\\":0,\\\"supplementalGroups\\\":[0],\\\"uid\\\":0}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/entrypoint\\\",\\\"name\\\":\\\"cni-sysctl-allowlist\\\"},{\\\"mountPath\\\":\\\"/host/etc/cni/tuning/\\\",\\\"name\\\":\\\"tuning-conf-dir\\\"},{\\\"mountPath\\\":\\\"/ready\\\",\\\"name\\\":\\\"ready\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-sgwhc\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]}],\\\"hostIP\\\":\\\"10.0.135.173\\\",\\\"hostIPs\\\":[{\\\"ip\\\":\\\"10.0.135.173\\\"}],\\\"observedGeneration\\\":1,\\\"phase\\\":\\\"Running\\\",\\\"podIP\\\":\\\"10.0.135.173\\\",\\\"podIPs\\\":[{\\\"ip\\\":\\\"10.0.135.173\\\"}],\\\"startTime\\\":\\\"2026-09-07T12:23:03Z\\\"}}\" for pod \"openshift-multus\"/\"cni-sysctl-allowlist-ds-nkkc9\": Internal error occurred: failed calling webhook \"pod.network-node-identity.openshift.io\": failed to call webhook: Post \"https://network-node-identity.clusters-70737425-554b-4289-aebf-e53ca32b3bc0.svc:9743/pod?timeout=10s\": dial tcp 172.30.186.39:9743: connect: connection refused" pod="openshift-multus/cni-sysctl-allowlist-ds-nkkc9"
Sep 07 12:23:05.257597 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:23:05.257581    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-multus/cni-sysctl-allowlist-ds-nkkc9"
Sep 07 12:23:05.265244 ip-10-0-135-173 kubenswrapper[2563]: E0907 12:23:05.265198    2563 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"d85226d2-5ce5-4ffb-95b6-eeed0206a352\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"lastProbeTime\\\":null,\\\"lastTransitionTime\\\":\\\"2026-09-07T12:23:05Z\\\",\\\"observedGeneration\\\":1,\\\"status\\\":\\\"True\\\",\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"lastProbeTime\\\":null,\\\"lastTransitionTime\\\":\\\"2026-09-07T12:23:03Z\\\",\\\"observedGeneration\\\":1,\\\"status\\\":\\\"True\\\",\\\"type\\\":\\\"Initialized\\\"},{\\\"lastProbeTime\\\":null,\\\"lastTransitionTime\\\":\\\"2026-09-07T12:23:05Z\\\",\\\"observedGeneration\\\":1,\\\"status\\\":\\\"True\\\",\\\"type\\\":\\\"Ready\\\"},{\\\"lastProbeTime\\\":null,\\\"lastTransitionTime\\\":\\\"2026-09-07T12:23:05Z\\\",\\\"observedGeneration\\\":1,\\\"status\\\":\\\"True\\\",\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":1,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"containerStatuses\\\":[{\\\"allocatedResources\\\":{\\\"cpu\\\":\\\"10m\\\",\\\"memory\\\":\\\"10Mi\\\"},\\\"containerID\\\":\\\"cri-o://5942f6c25d1daae4944c44b3a6d7151700de35122ba904be663526b900106565\\\",\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:cfde312743ce0a1d3f82fc68e67e64a3fca57853ed9c621b1fe74db36e5943ea\\\",\\\"imageID\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:cfde312743ce0a1d3f82fc68e67e64a3fca57853ed9c621b1fe74db36e5943ea\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"kube-multus-additional-cni-plugins\\\",\\\"ready\\\":true,\\\"resources\\\":{\\\"requests\\\":{\\\"cpu\\\":\\\"10m\\\",\\\"memory\\\":\\\"10Mi\\\"}},\\\"restartCount\\\":0,\\\"started\\\":true,\\\"state\\\":{\\\"running\\\":{\\\"startedAt\\\":\\\"2026-09-07T12:23:04Z\\\"}},\\\"user\\\":{\\\"linux\\\":{\\\"gid\\\":0,\\\"supplementalGroups\\\":[0],\\\"uid\\\":0}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/entrypoint\\\",\\\"name\\\":\\\"cni-sysctl-allowlist\\\"},{\\\"mountPath\\\":\\\"/host/etc/cni/tuning/\\\",\\\"name\\\":\\\"tuning-conf-dir\\\"},{\\\"mountPath\\\":\\\"/ready\\\",\\\"name\\\":\\\"ready\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-sgwhc\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]}],\\\"hostIP\\\":\\\"10.0.135.173\\\",\\\"hostIPs\\\":[{\\\"ip\\\":\\\"10.0.135.173\\\"}],\\\"observedGeneration\\\":1,\\\"phase\\\":\\\"Running\\\",\\\"podIP\\\":\\\"10.0.135.173\\\",\\\"podIPs\\\":[{\\\"ip\\\":\\\"10.0.135.173\\\"}],\\\"startTime\\\":\\\"2026-09-07T12:23:03Z\\\"}}\" for pod \"openshift-multus\"/\"cni-sysctl-allowlist-ds-nkkc9\": Internal error occurred: failed calling webhook \"pod.network-node-identity.openshift.io\": failed to call webhook: Post \"https://network-node-identity.clusters-70737425-554b-4289-aebf-e53ca32b3bc0.svc:9743/pod?timeout=10s\": dial tcp 172.30.186.39:9743: connect: connection refused" pod="openshift-multus/cni-sysctl-allowlist-ds-nkkc9"
Sep 07 12:23:06.579283 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:23:06.579243    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:23:07.094035 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:23:07.093996    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:23:09.061794 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:23:09.061745    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/cni-sysctl-allowlist-ds-nkkc9" podStartSLOduration=6.061726047 podStartE2EDuration="6.061726047s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-07 12:23:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 12:23:09.061124885 +0000 UTC m=+2160.579856699" watchObservedRunningTime="2026-09-07 12:23:09.061726047 +0000 UTC m=+2160.580457862"
Sep 07 12:23:14.114802 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:23:14.114762    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:23:14.642406 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:23:14.642364    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:23:14.743295 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:23:14.743259    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:23:14.843992 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:23:14.843969    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:23:14.944616 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:23:14.944582    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:23:15.936525 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:23:15.936490    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-multus/cni-sysctl-allowlist-ds-nkkc9"]
Sep 07 12:23:15.936898 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:23:15.936679    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-multus/cni-sysctl-allowlist-ds-nkkc9" podUID="d85226d2-5ce5-4ffb-95b6-eeed0206a352" containerName="kube-multus-additional-cni-plugins" containerID="cri-o://5942f6c25d1daae4944c44b3a6d7151700de35122ba904be663526b900106565" gracePeriod=30
Sep 07 12:23:18.137695 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:23:18.137664    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:23:18.238227 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:23:18.238190    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:23:18.338590 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:23:18.338563    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:23:18.398779 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:23:18.398702    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:23:20.846579 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:23:20.846544    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:23:20.900757 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:23:20.900725    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:23:25.249601 ip-10-0-135-173 kubenswrapper[2563]: E0907 12:23:25.249557    2563 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="5942f6c25d1daae4944c44b3a6d7151700de35122ba904be663526b900106565" cmd=["/bin/bash","-c","test -f /ready/ready"]
Sep 07 12:23:25.250590 ip-10-0-135-173 kubenswrapper[2563]: E0907 12:23:25.250560    2563 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="5942f6c25d1daae4944c44b3a6d7151700de35122ba904be663526b900106565" cmd=["/bin/bash","-c","test -f /ready/ready"]
Sep 07 12:23:25.251446 ip-10-0-135-173 kubenswrapper[2563]: E0907 12:23:25.251406    2563 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="5942f6c25d1daae4944c44b3a6d7151700de35122ba904be663526b900106565" cmd=["/bin/bash","-c","test -f /ready/ready"]
Sep 07 12:23:25.251549 ip-10-0-135-173 kubenswrapper[2563]: E0907 12:23:25.251456    2563 prober.go:104] "Probe errored" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" probeType="Readiness" pod="openshift-multus/cni-sysctl-allowlist-ds-nkkc9" podUID="d85226d2-5ce5-4ffb-95b6-eeed0206a352" containerName="kube-multus-additional-cni-plugins" probeResult="unknown"
Sep 07 12:23:35.249548 ip-10-0-135-173 kubenswrapper[2563]: E0907 12:23:35.249509    2563 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="5942f6c25d1daae4944c44b3a6d7151700de35122ba904be663526b900106565" cmd=["/bin/bash","-c","test -f /ready/ready"]
Sep 07 12:23:35.250365 ip-10-0-135-173 kubenswrapper[2563]: E0907 12:23:35.250338    2563 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="5942f6c25d1daae4944c44b3a6d7151700de35122ba904be663526b900106565" cmd=["/bin/bash","-c","test -f /ready/ready"]
Sep 07 12:23:35.251166 ip-10-0-135-173 kubenswrapper[2563]: E0907 12:23:35.251137    2563 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="5942f6c25d1daae4944c44b3a6d7151700de35122ba904be663526b900106565" cmd=["/bin/bash","-c","test -f /ready/ready"]
Sep 07 12:23:35.251281 ip-10-0-135-173 kubenswrapper[2563]: E0907 12:23:35.251179    2563 prober.go:104] "Probe errored" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" probeType="Readiness" pod="openshift-multus/cni-sysctl-allowlist-ds-nkkc9" podUID="d85226d2-5ce5-4ffb-95b6-eeed0206a352" containerName="kube-multus-additional-cni-plugins" probeResult="unknown"
Sep 07 12:23:36.782394 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:23:36.782358    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:23:37.705546 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:23:37.705510    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-p4l4w"]
Sep 07 12:23:38.840923 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:23:38.840887    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:23:40.387110 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:23:40.387067    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:23:45.250209 ip-10-0-135-173 kubenswrapper[2563]: E0907 12:23:45.250169    2563 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="5942f6c25d1daae4944c44b3a6d7151700de35122ba904be663526b900106565" cmd=["/bin/bash","-c","test -f /ready/ready"]
Sep 07 12:23:45.251173 ip-10-0-135-173 kubenswrapper[2563]: E0907 12:23:45.251146    2563 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="5942f6c25d1daae4944c44b3a6d7151700de35122ba904be663526b900106565" cmd=["/bin/bash","-c","test -f /ready/ready"]
Sep 07 12:23:45.252037 ip-10-0-135-173 kubenswrapper[2563]: E0907 12:23:45.252006    2563 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="5942f6c25d1daae4944c44b3a6d7151700de35122ba904be663526b900106565" cmd=["/bin/bash","-c","test -f /ready/ready"]
Sep 07 12:23:45.252152 ip-10-0-135-173 kubenswrapper[2563]: E0907 12:23:45.252051    2563 prober.go:104] "Probe errored" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" probeType="Readiness" pod="openshift-multus/cni-sysctl-allowlist-ds-nkkc9" podUID="d85226d2-5ce5-4ffb-95b6-eeed0206a352" containerName="kube-multus-additional-cni-plugins" probeResult="unknown"
Sep 07 12:23:46.016314 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:23:46.016291    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_cni-sysctl-allowlist-ds-nkkc9_d85226d2-5ce5-4ffb-95b6-eeed0206a352/kube-multus-additional-cni-plugins/0.log"
Sep 07 12:23:46.018555 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:23:46.018537    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-multus/cni-sysctl-allowlist-ds-nkkc9"
Sep 07 12:23:46.153445 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:23:46.153387    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ready\" (UniqueName: \"kubernetes.io/empty-dir/d85226d2-5ce5-4ffb-95b6-eeed0206a352-ready\") pod \"d85226d2-5ce5-4ffb-95b6-eeed0206a352\" (UID: \"d85226d2-5ce5-4ffb-95b6-eeed0206a352\") "
Sep 07 12:23:46.153631 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:23:46.153460    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sgwhc\" (UniqueName: \"kubernetes.io/projected/d85226d2-5ce5-4ffb-95b6-eeed0206a352-kube-api-access-sgwhc\") pod \"d85226d2-5ce5-4ffb-95b6-eeed0206a352\" (UID: \"d85226d2-5ce5-4ffb-95b6-eeed0206a352\") "
Sep 07 12:23:46.153631 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:23:46.153538    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/d85226d2-5ce5-4ffb-95b6-eeed0206a352-tuning-conf-dir\") pod \"d85226d2-5ce5-4ffb-95b6-eeed0206a352\" (UID: \"d85226d2-5ce5-4ffb-95b6-eeed0206a352\") "
Sep 07 12:23:46.153631 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:23:46.153582    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/d85226d2-5ce5-4ffb-95b6-eeed0206a352-cni-sysctl-allowlist\") pod \"d85226d2-5ce5-4ffb-95b6-eeed0206a352\" (UID: \"d85226d2-5ce5-4ffb-95b6-eeed0206a352\") "
Sep 07 12:23:46.153785 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:23:46.153630    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/host-path/d85226d2-5ce5-4ffb-95b6-eeed0206a352-tuning-conf-dir" (OuterVolumeSpecName: "tuning-conf-dir") pod "d85226d2-5ce5-4ffb-95b6-eeed0206a352" (UID: "d85226d2-5ce5-4ffb-95b6-eeed0206a352"). InnerVolumeSpecName "tuning-conf-dir". PluginName "kubernetes.io/host-path", VolumeGIDValue ""
Sep 07 12:23:46.153785 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:23:46.153688    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d85226d2-5ce5-4ffb-95b6-eeed0206a352-ready" (OuterVolumeSpecName: "ready") pod "d85226d2-5ce5-4ffb-95b6-eeed0206a352" (UID: "d85226d2-5ce5-4ffb-95b6-eeed0206a352"). InnerVolumeSpecName "ready". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 12:23:46.153884 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:23:46.153868    2563 reconciler_common.go:299] "Volume detached for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/d85226d2-5ce5-4ffb-95b6-eeed0206a352-tuning-conf-dir\") on node \"ip-10-0-135-173.ec2.internal\" DevicePath \"\""
Sep 07 12:23:46.153921 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:23:46.153886    2563 reconciler_common.go:299] "Volume detached for volume \"ready\" (UniqueName: \"kubernetes.io/empty-dir/d85226d2-5ce5-4ffb-95b6-eeed0206a352-ready\") on node \"ip-10-0-135-173.ec2.internal\" DevicePath \"\""
Sep 07 12:23:46.153960 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:23:46.153926    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d85226d2-5ce5-4ffb-95b6-eeed0206a352-cni-sysctl-allowlist" (OuterVolumeSpecName: "cni-sysctl-allowlist") pod "d85226d2-5ce5-4ffb-95b6-eeed0206a352" (UID: "d85226d2-5ce5-4ffb-95b6-eeed0206a352"). InnerVolumeSpecName "cni-sysctl-allowlist". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 07 12:23:46.155624 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:23:46.155604    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d85226d2-5ce5-4ffb-95b6-eeed0206a352-kube-api-access-sgwhc" (OuterVolumeSpecName: "kube-api-access-sgwhc") pod "d85226d2-5ce5-4ffb-95b6-eeed0206a352" (UID: "d85226d2-5ce5-4ffb-95b6-eeed0206a352"). InnerVolumeSpecName "kube-api-access-sgwhc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 12:23:46.254873 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:23:46.254840    2563 reconciler_common.go:299] "Volume detached for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/d85226d2-5ce5-4ffb-95b6-eeed0206a352-cni-sysctl-allowlist\") on node \"ip-10-0-135-173.ec2.internal\" DevicePath \"\""
Sep 07 12:23:46.254873 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:23:46.254867    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sgwhc\" (UniqueName: \"kubernetes.io/projected/d85226d2-5ce5-4ffb-95b6-eeed0206a352-kube-api-access-sgwhc\") on node \"ip-10-0-135-173.ec2.internal\" DevicePath \"\""
Sep 07 12:23:46.444317 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:23:46.444194    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_cni-sysctl-allowlist-ds-nkkc9_d85226d2-5ce5-4ffb-95b6-eeed0206a352/kube-multus-additional-cni-plugins/0.log"
Sep 07 12:23:46.444317 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:23:46.444248    2563 generic.go:356] "Generic (PLEG): container finished" podID="d85226d2-5ce5-4ffb-95b6-eeed0206a352" containerID="5942f6c25d1daae4944c44b3a6d7151700de35122ba904be663526b900106565" exitCode=137
Sep 07 12:23:46.444317 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:23:46.444295    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-multus/cni-sysctl-allowlist-ds-nkkc9"
Sep 07 12:23:46.452694 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:23:46.444341    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/cni-sysctl-allowlist-ds-nkkc9" event={"ID":"d85226d2-5ce5-4ffb-95b6-eeed0206a352","Type":"ContainerDied","Data":"5942f6c25d1daae4944c44b3a6d7151700de35122ba904be663526b900106565"}
Sep 07 12:23:46.452694 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:23:46.444381    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/cni-sysctl-allowlist-ds-nkkc9" event={"ID":"d85226d2-5ce5-4ffb-95b6-eeed0206a352","Type":"ContainerDied","Data":"19bb5417a4dbb31da1bb4090c494616da3d496003d9fcf2828838204ef919241"}
Sep 07 12:23:46.452694 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:23:46.444400    2563 scope.go:117] "RemoveContainer" containerID="5942f6c25d1daae4944c44b3a6d7151700de35122ba904be663526b900106565"
Sep 07 12:23:46.454026 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:23:46.454010    2563 scope.go:117] "RemoveContainer" containerID="5942f6c25d1daae4944c44b3a6d7151700de35122ba904be663526b900106565"
Sep 07 12:23:46.454296 ip-10-0-135-173 kubenswrapper[2563]: E0907 12:23:46.454278    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5942f6c25d1daae4944c44b3a6d7151700de35122ba904be663526b900106565\": container with ID starting with 5942f6c25d1daae4944c44b3a6d7151700de35122ba904be663526b900106565 not found: ID does not exist" containerID="5942f6c25d1daae4944c44b3a6d7151700de35122ba904be663526b900106565"
Sep 07 12:23:46.454363 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:23:46.454309    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5942f6c25d1daae4944c44b3a6d7151700de35122ba904be663526b900106565"} err="failed to get container status \"5942f6c25d1daae4944c44b3a6d7151700de35122ba904be663526b900106565\": rpc error: code = NotFound desc = could not find container \"5942f6c25d1daae4944c44b3a6d7151700de35122ba904be663526b900106565\": container with ID starting with 5942f6c25d1daae4944c44b3a6d7151700de35122ba904be663526b900106565 not found: ID does not exist"
Sep 07 12:23:46.463907 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:23:46.463882    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-multus/cni-sysctl-allowlist-ds-nkkc9"]
Sep 07 12:23:46.467395 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:23:46.467376    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-multus/cni-sysctl-allowlist-ds-nkkc9"]
Sep 07 12:23:47.040742 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:23:47.040709    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d85226d2-5ce5-4ffb-95b6-eeed0206a352" path="/var/lib/kubelet/pods/d85226d2-5ce5-4ffb-95b6-eeed0206a352/volumes"
Sep 07 12:24:02.203469 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:24:02.203432    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-p4l4w"]
Sep 07 12:24:10.099590 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:24:10.099557    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-p4l4w"]
Sep 07 12:24:17.398627 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:24:17.398589    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-p4l4w"]
Sep 07 12:24:24.400125 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:24:24.400094    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-p4l4w"]
Sep 07 12:24:32.794570 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:24:32.794491    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-p4l4w"]
Sep 07 12:24:48.305828 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:24:48.305790    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-p4l4w"]
Sep 07 12:24:56.903379 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:24:56.903341    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-p4l4w"]
Sep 07 12:26:22.495279 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:26:22.495247    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-p4l4w"]
Sep 07 12:26:29.399784 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:26:29.399753    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-p4l4w"]
Sep 07 12:26:36.799230 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:26:36.799190    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-p4l4w"]
Sep 07 12:26:43.562703 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:26:43.562669    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-a-d86c0acb-kserve-f8fbddfc5-hhf4f"]
Sep 07 12:26:43.563075 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:26:43.563030    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d85226d2-5ce5-4ffb-95b6-eeed0206a352" containerName="kube-multus-additional-cni-plugins"
Sep 07 12:26:43.563075 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:26:43.563041    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="d85226d2-5ce5-4ffb-95b6-eeed0206a352" containerName="kube-multus-additional-cni-plugins"
Sep 07 12:26:43.563154 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:26:43.563107    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="d85226d2-5ce5-4ffb-95b6-eeed0206a352" containerName="kube-multus-additional-cni-plugins"
Sep 07 12:26:43.566149 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:26:43.566132    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-d86c0acb-kserve-f8fbddfc5-hhf4f"
Sep 07 12:26:43.568954 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:26:43.568932    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-a-d86c0acb-kserve-self-signed-certs\""
Sep 07 12:26:43.581728 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:26:43.581704    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-d86c0acb-kserve-f8fbddfc5-hhf4f"]
Sep 07 12:26:43.597992 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:26:43.597957    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2e85b1df-fbca-4e67-8d0e-3c0cf1be4dce-model-cache\") pod \"e2e-same-id-a-d86c0acb-kserve-f8fbddfc5-hhf4f\" (UID: \"2e85b1df-fbca-4e67-8d0e-3c0cf1be4dce\") " pod="llm/e2e-same-id-a-d86c0acb-kserve-f8fbddfc5-hhf4f"
Sep 07 12:26:43.598121 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:26:43.598004    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-czgg5\" (UniqueName: \"kubernetes.io/projected/2e85b1df-fbca-4e67-8d0e-3c0cf1be4dce-kube-api-access-czgg5\") pod \"e2e-same-id-a-d86c0acb-kserve-f8fbddfc5-hhf4f\" (UID: \"2e85b1df-fbca-4e67-8d0e-3c0cf1be4dce\") " pod="llm/e2e-same-id-a-d86c0acb-kserve-f8fbddfc5-hhf4f"
Sep 07 12:26:43.598186 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:26:43.598114    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2e85b1df-fbca-4e67-8d0e-3c0cf1be4dce-home\") pod \"e2e-same-id-a-d86c0acb-kserve-f8fbddfc5-hhf4f\" (UID: \"2e85b1df-fbca-4e67-8d0e-3c0cf1be4dce\") " pod="llm/e2e-same-id-a-d86c0acb-kserve-f8fbddfc5-hhf4f"
Sep 07 12:26:43.598186 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:26:43.598151    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2e85b1df-fbca-4e67-8d0e-3c0cf1be4dce-tmp-dir\") pod \"e2e-same-id-a-d86c0acb-kserve-f8fbddfc5-hhf4f\" (UID: \"2e85b1df-fbca-4e67-8d0e-3c0cf1be4dce\") " pod="llm/e2e-same-id-a-d86c0acb-kserve-f8fbddfc5-hhf4f"
Sep 07 12:26:43.598300 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:26:43.598218    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2e85b1df-fbca-4e67-8d0e-3c0cf1be4dce-tls-certs\") pod \"e2e-same-id-a-d86c0acb-kserve-f8fbddfc5-hhf4f\" (UID: \"2e85b1df-fbca-4e67-8d0e-3c0cf1be4dce\") " pod="llm/e2e-same-id-a-d86c0acb-kserve-f8fbddfc5-hhf4f"
Sep 07 12:26:43.598300 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:26:43.598263    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2e85b1df-fbca-4e67-8d0e-3c0cf1be4dce-dshm\") pod \"e2e-same-id-a-d86c0acb-kserve-f8fbddfc5-hhf4f\" (UID: \"2e85b1df-fbca-4e67-8d0e-3c0cf1be4dce\") " pod="llm/e2e-same-id-a-d86c0acb-kserve-f8fbddfc5-hhf4f"
Sep 07 12:26:43.655759 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:26:43.655692    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:26:43.699391 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:26:43.699362    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2e85b1df-fbca-4e67-8d0e-3c0cf1be4dce-dshm\") pod \"e2e-same-id-a-d86c0acb-kserve-f8fbddfc5-hhf4f\" (UID: \"2e85b1df-fbca-4e67-8d0e-3c0cf1be4dce\") " pod="llm/e2e-same-id-a-d86c0acb-kserve-f8fbddfc5-hhf4f"
Sep 07 12:26:43.699527 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:26:43.699404    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2e85b1df-fbca-4e67-8d0e-3c0cf1be4dce-model-cache\") pod \"e2e-same-id-a-d86c0acb-kserve-f8fbddfc5-hhf4f\" (UID: \"2e85b1df-fbca-4e67-8d0e-3c0cf1be4dce\") " pod="llm/e2e-same-id-a-d86c0acb-kserve-f8fbddfc5-hhf4f"
Sep 07 12:26:43.699527 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:26:43.699460    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-czgg5\" (UniqueName: \"kubernetes.io/projected/2e85b1df-fbca-4e67-8d0e-3c0cf1be4dce-kube-api-access-czgg5\") pod \"e2e-same-id-a-d86c0acb-kserve-f8fbddfc5-hhf4f\" (UID: \"2e85b1df-fbca-4e67-8d0e-3c0cf1be4dce\") " pod="llm/e2e-same-id-a-d86c0acb-kserve-f8fbddfc5-hhf4f"
Sep 07 12:26:43.699527 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:26:43.699506    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2e85b1df-fbca-4e67-8d0e-3c0cf1be4dce-home\") pod \"e2e-same-id-a-d86c0acb-kserve-f8fbddfc5-hhf4f\" (UID: \"2e85b1df-fbca-4e67-8d0e-3c0cf1be4dce\") " pod="llm/e2e-same-id-a-d86c0acb-kserve-f8fbddfc5-hhf4f"
Sep 07 12:26:43.699700 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:26:43.699529    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2e85b1df-fbca-4e67-8d0e-3c0cf1be4dce-tmp-dir\") pod \"e2e-same-id-a-d86c0acb-kserve-f8fbddfc5-hhf4f\" (UID: \"2e85b1df-fbca-4e67-8d0e-3c0cf1be4dce\") " pod="llm/e2e-same-id-a-d86c0acb-kserve-f8fbddfc5-hhf4f"
Sep 07 12:26:43.699700 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:26:43.699583    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2e85b1df-fbca-4e67-8d0e-3c0cf1be4dce-tls-certs\") pod \"e2e-same-id-a-d86c0acb-kserve-f8fbddfc5-hhf4f\" (UID: \"2e85b1df-fbca-4e67-8d0e-3c0cf1be4dce\") " pod="llm/e2e-same-id-a-d86c0acb-kserve-f8fbddfc5-hhf4f"
Sep 07 12:26:43.699901 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:26:43.699882    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2e85b1df-fbca-4e67-8d0e-3c0cf1be4dce-model-cache\") pod \"e2e-same-id-a-d86c0acb-kserve-f8fbddfc5-hhf4f\" (UID: \"2e85b1df-fbca-4e67-8d0e-3c0cf1be4dce\") " pod="llm/e2e-same-id-a-d86c0acb-kserve-f8fbddfc5-hhf4f"
Sep 07 12:26:43.699962 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:26:43.699925    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2e85b1df-fbca-4e67-8d0e-3c0cf1be4dce-home\") pod \"e2e-same-id-a-d86c0acb-kserve-f8fbddfc5-hhf4f\" (UID: \"2e85b1df-fbca-4e67-8d0e-3c0cf1be4dce\") " pod="llm/e2e-same-id-a-d86c0acb-kserve-f8fbddfc5-hhf4f"
Sep 07 12:26:43.699999 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:26:43.699971    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2e85b1df-fbca-4e67-8d0e-3c0cf1be4dce-tmp-dir\") pod \"e2e-same-id-a-d86c0acb-kserve-f8fbddfc5-hhf4f\" (UID: \"2e85b1df-fbca-4e67-8d0e-3c0cf1be4dce\") " pod="llm/e2e-same-id-a-d86c0acb-kserve-f8fbddfc5-hhf4f"
Sep 07 12:26:43.701755 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:26:43.701732    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2e85b1df-fbca-4e67-8d0e-3c0cf1be4dce-dshm\") pod \"e2e-same-id-a-d86c0acb-kserve-f8fbddfc5-hhf4f\" (UID: \"2e85b1df-fbca-4e67-8d0e-3c0cf1be4dce\") " pod="llm/e2e-same-id-a-d86c0acb-kserve-f8fbddfc5-hhf4f"
Sep 07 12:26:43.701960 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:26:43.701943    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2e85b1df-fbca-4e67-8d0e-3c0cf1be4dce-tls-certs\") pod \"e2e-same-id-a-d86c0acb-kserve-f8fbddfc5-hhf4f\" (UID: \"2e85b1df-fbca-4e67-8d0e-3c0cf1be4dce\") " pod="llm/e2e-same-id-a-d86c0acb-kserve-f8fbddfc5-hhf4f"
Sep 07 12:26:43.706076 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:26:43.706056    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-czgg5\" (UniqueName: \"kubernetes.io/projected/2e85b1df-fbca-4e67-8d0e-3c0cf1be4dce-kube-api-access-czgg5\") pod \"e2e-same-id-a-d86c0acb-kserve-f8fbddfc5-hhf4f\" (UID: \"2e85b1df-fbca-4e67-8d0e-3c0cf1be4dce\") " pod="llm/e2e-same-id-a-d86c0acb-kserve-f8fbddfc5-hhf4f"
Sep 07 12:26:43.875627 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:26:43.875549    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-d86c0acb-kserve-f8fbddfc5-hhf4f"
Sep 07 12:26:44.146224 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:26:44.146112    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-d86c0acb-kserve-f8fbddfc5-hhf4f"]
Sep 07 12:26:44.148704 ip-10-0-135-173 kubenswrapper[2563]: W0907 12:26:44.148675    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2e85b1df_fbca_4e67_8d0e_3c0cf1be4dce.slice/crio-51e4451ae671668816c31255b6563b76ddb29287dd3e2abba950e5396961f5ed WatchSource:0}: Error finding container 51e4451ae671668816c31255b6563b76ddb29287dd3e2abba950e5396961f5ed: Status 404 returned error can't find the container with id 51e4451ae671668816c31255b6563b76ddb29287dd3e2abba950e5396961f5ed
Sep 07 12:26:44.284139 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:26:44.284097    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-d86c0acb-kserve-f8fbddfc5-hhf4f" event={"ID":"2e85b1df-fbca-4e67-8d0e-3c0cf1be4dce","Type":"ContainerStarted","Data":"b3581a650c66bb70a6e75a6189daea8f3108f3bede548a9aaff10d173c15ba33"}
Sep 07 12:26:44.284139 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:26:44.284141    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-d86c0acb-kserve-f8fbddfc5-hhf4f" event={"ID":"2e85b1df-fbca-4e67-8d0e-3c0cf1be4dce","Type":"ContainerStarted","Data":"51e4451ae671668816c31255b6563b76ddb29287dd3e2abba950e5396961f5ed"}
Sep 07 12:26:44.299299 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:26:44.299257    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-a-d86c0acb-kserve-f8fbddfc5-hhf4f" podStartSLOduration=1.299241988 podStartE2EDuration="1.299241988s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 12:26:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 12:26:44.297852901 +0000 UTC m=+2375.816584726" watchObservedRunningTime="2026-09-07 12:26:44.299241988 +0000 UTC m=+2375.817973800"
Sep 07 12:26:46.306726 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:26:46.306681    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-p4l4w"]
Sep 07 12:26:46.697172 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:26:46.697132    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-b-d86c0acb-kserve-6d95449d95-7p5cv"]
Sep 07 12:26:46.700674 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:26:46.700653    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-d86c0acb-kserve-6d95449d95-7p5cv"
Sep 07 12:26:46.702166 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:26:46.702145    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-b-d86c0acb-kserve-self-signed-certs\""
Sep 07 12:26:46.708926 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:26:46.708901    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-d86c0acb-kserve-6d95449d95-7p5cv"]
Sep 07 12:26:46.727679 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:26:46.727655    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cec4771b-2ae0-4171-bcc9-d37222b84764-tls-certs\") pod \"e2e-same-id-b-d86c0acb-kserve-6d95449d95-7p5cv\" (UID: \"cec4771b-2ae0-4171-bcc9-d37222b84764\") " pod="llm/e2e-same-id-b-d86c0acb-kserve-6d95449d95-7p5cv"
Sep 07 12:26:46.727782 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:26:46.727682    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cec4771b-2ae0-4171-bcc9-d37222b84764-model-cache\") pod \"e2e-same-id-b-d86c0acb-kserve-6d95449d95-7p5cv\" (UID: \"cec4771b-2ae0-4171-bcc9-d37222b84764\") " pod="llm/e2e-same-id-b-d86c0acb-kserve-6d95449d95-7p5cv"
Sep 07 12:26:46.727782 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:26:46.727711    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cec4771b-2ae0-4171-bcc9-d37222b84764-dshm\") pod \"e2e-same-id-b-d86c0acb-kserve-6d95449d95-7p5cv\" (UID: \"cec4771b-2ae0-4171-bcc9-d37222b84764\") " pod="llm/e2e-same-id-b-d86c0acb-kserve-6d95449d95-7p5cv"
Sep 07 12:26:46.727914 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:26:46.727823    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cec4771b-2ae0-4171-bcc9-d37222b84764-tmp-dir\") pod \"e2e-same-id-b-d86c0acb-kserve-6d95449d95-7p5cv\" (UID: \"cec4771b-2ae0-4171-bcc9-d37222b84764\") " pod="llm/e2e-same-id-b-d86c0acb-kserve-6d95449d95-7p5cv"
Sep 07 12:26:46.727974 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:26:46.727910    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-trknf\" (UniqueName: \"kubernetes.io/projected/cec4771b-2ae0-4171-bcc9-d37222b84764-kube-api-access-trknf\") pod \"e2e-same-id-b-d86c0acb-kserve-6d95449d95-7p5cv\" (UID: \"cec4771b-2ae0-4171-bcc9-d37222b84764\") " pod="llm/e2e-same-id-b-d86c0acb-kserve-6d95449d95-7p5cv"
Sep 07 12:26:46.727974 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:26:46.727950    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cec4771b-2ae0-4171-bcc9-d37222b84764-home\") pod \"e2e-same-id-b-d86c0acb-kserve-6d95449d95-7p5cv\" (UID: \"cec4771b-2ae0-4171-bcc9-d37222b84764\") " pod="llm/e2e-same-id-b-d86c0acb-kserve-6d95449d95-7p5cv"
Sep 07 12:26:46.805602 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:26:46.805559    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:26:46.828870 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:26:46.828841    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-trknf\" (UniqueName: \"kubernetes.io/projected/cec4771b-2ae0-4171-bcc9-d37222b84764-kube-api-access-trknf\") pod \"e2e-same-id-b-d86c0acb-kserve-6d95449d95-7p5cv\" (UID: \"cec4771b-2ae0-4171-bcc9-d37222b84764\") " pod="llm/e2e-same-id-b-d86c0acb-kserve-6d95449d95-7p5cv"
Sep 07 12:26:46.829030 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:26:46.828881    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cec4771b-2ae0-4171-bcc9-d37222b84764-home\") pod \"e2e-same-id-b-d86c0acb-kserve-6d95449d95-7p5cv\" (UID: \"cec4771b-2ae0-4171-bcc9-d37222b84764\") " pod="llm/e2e-same-id-b-d86c0acb-kserve-6d95449d95-7p5cv"
Sep 07 12:26:46.829030 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:26:46.828931    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cec4771b-2ae0-4171-bcc9-d37222b84764-tls-certs\") pod \"e2e-same-id-b-d86c0acb-kserve-6d95449d95-7p5cv\" (UID: \"cec4771b-2ae0-4171-bcc9-d37222b84764\") " pod="llm/e2e-same-id-b-d86c0acb-kserve-6d95449d95-7p5cv"
Sep 07 12:26:46.829030 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:26:46.828947    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cec4771b-2ae0-4171-bcc9-d37222b84764-model-cache\") pod \"e2e-same-id-b-d86c0acb-kserve-6d95449d95-7p5cv\" (UID: \"cec4771b-2ae0-4171-bcc9-d37222b84764\") " pod="llm/e2e-same-id-b-d86c0acb-kserve-6d95449d95-7p5cv"
Sep 07 12:26:46.829030 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:26:46.828976    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cec4771b-2ae0-4171-bcc9-d37222b84764-dshm\") pod \"e2e-same-id-b-d86c0acb-kserve-6d95449d95-7p5cv\" (UID: \"cec4771b-2ae0-4171-bcc9-d37222b84764\") " pod="llm/e2e-same-id-b-d86c0acb-kserve-6d95449d95-7p5cv"
Sep 07 12:26:46.829030 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:26:46.829016    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cec4771b-2ae0-4171-bcc9-d37222b84764-tmp-dir\") pod \"e2e-same-id-b-d86c0acb-kserve-6d95449d95-7p5cv\" (UID: \"cec4771b-2ae0-4171-bcc9-d37222b84764\") " pod="llm/e2e-same-id-b-d86c0acb-kserve-6d95449d95-7p5cv"
Sep 07 12:26:46.829362 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:26:46.829339    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cec4771b-2ae0-4171-bcc9-d37222b84764-home\") pod \"e2e-same-id-b-d86c0acb-kserve-6d95449d95-7p5cv\" (UID: \"cec4771b-2ae0-4171-bcc9-d37222b84764\") " pod="llm/e2e-same-id-b-d86c0acb-kserve-6d95449d95-7p5cv"
Sep 07 12:26:46.829406 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:26:46.829380    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cec4771b-2ae0-4171-bcc9-d37222b84764-tmp-dir\") pod \"e2e-same-id-b-d86c0acb-kserve-6d95449d95-7p5cv\" (UID: \"cec4771b-2ae0-4171-bcc9-d37222b84764\") " pod="llm/e2e-same-id-b-d86c0acb-kserve-6d95449d95-7p5cv"
Sep 07 12:26:46.829525 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:26:46.829503    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cec4771b-2ae0-4171-bcc9-d37222b84764-model-cache\") pod \"e2e-same-id-b-d86c0acb-kserve-6d95449d95-7p5cv\" (UID: \"cec4771b-2ae0-4171-bcc9-d37222b84764\") " pod="llm/e2e-same-id-b-d86c0acb-kserve-6d95449d95-7p5cv"
Sep 07 12:26:46.831302 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:26:46.831279    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cec4771b-2ae0-4171-bcc9-d37222b84764-dshm\") pod \"e2e-same-id-b-d86c0acb-kserve-6d95449d95-7p5cv\" (UID: \"cec4771b-2ae0-4171-bcc9-d37222b84764\") " pod="llm/e2e-same-id-b-d86c0acb-kserve-6d95449d95-7p5cv"
Sep 07 12:26:46.831624 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:26:46.831605    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cec4771b-2ae0-4171-bcc9-d37222b84764-tls-certs\") pod \"e2e-same-id-b-d86c0acb-kserve-6d95449d95-7p5cv\" (UID: \"cec4771b-2ae0-4171-bcc9-d37222b84764\") " pod="llm/e2e-same-id-b-d86c0acb-kserve-6d95449d95-7p5cv"
Sep 07 12:26:46.835920 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:26:46.835899    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-trknf\" (UniqueName: \"kubernetes.io/projected/cec4771b-2ae0-4171-bcc9-d37222b84764-kube-api-access-trknf\") pod \"e2e-same-id-b-d86c0acb-kserve-6d95449d95-7p5cv\" (UID: \"cec4771b-2ae0-4171-bcc9-d37222b84764\") " pod="llm/e2e-same-id-b-d86c0acb-kserve-6d95449d95-7p5cv"
Sep 07 12:26:47.011403 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:26:47.011319    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-d86c0acb-kserve-6d95449d95-7p5cv"
Sep 07 12:26:47.285944 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:26:47.285841    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-d86c0acb-kserve-6d95449d95-7p5cv"]
Sep 07 12:26:47.288582 ip-10-0-135-173 kubenswrapper[2563]: W0907 12:26:47.288536    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podcec4771b_2ae0_4171_bcc9_d37222b84764.slice/crio-bc7ca93ccb24e9571a16c76e41329d96c5a57882b440c304c9fe2088f30f8f99 WatchSource:0}: Error finding container bc7ca93ccb24e9571a16c76e41329d96c5a57882b440c304c9fe2088f30f8f99: Status 404 returned error can't find the container with id bc7ca93ccb24e9571a16c76e41329d96c5a57882b440c304c9fe2088f30f8f99
Sep 07 12:26:47.303484 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:26:47.303455    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-d86c0acb-kserve-6d95449d95-7p5cv" event={"ID":"cec4771b-2ae0-4171-bcc9-d37222b84764","Type":"ContainerStarted","Data":"bc7ca93ccb24e9571a16c76e41329d96c5a57882b440c304c9fe2088f30f8f99"}
Sep 07 12:26:48.309759 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:26:48.309725    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-d86c0acb-kserve-6d95449d95-7p5cv" event={"ID":"cec4771b-2ae0-4171-bcc9-d37222b84764","Type":"ContainerStarted","Data":"9a0508cb3735d5c5ad1b77228749c74f274bef33bc3189c2298432ea593ed822"}
Sep 07 12:26:48.325427 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:26:48.325370    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-b-d86c0acb-kserve-6d95449d95-7p5cv" podStartSLOduration=2.325356272 podStartE2EDuration="2.325356272s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 12:26:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 12:26:48.323173721 +0000 UTC m=+2379.841905533" watchObservedRunningTime="2026-09-07 12:26:48.325356272 +0000 UTC m=+2379.844088084"
Sep 07 12:26:53.876519 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:26:53.876482    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-a-d86c0acb-kserve-f8fbddfc5-hhf4f"
Sep 07 12:26:53.876519 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:26:53.876526    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-a-d86c0acb-kserve-f8fbddfc5-hhf4f"
Sep 07 12:26:53.889262 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:26:53.889233    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-a-d86c0acb-kserve-f8fbddfc5-hhf4f"
Sep 07 12:26:54.350643 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:26:54.350613    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-a-d86c0acb-kserve-f8fbddfc5-hhf4f"
Sep 07 12:26:55.397724 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:26:55.397692    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-p4l4w"]
Sep 07 12:26:57.012284 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:26:57.012244    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-b-d86c0acb-kserve-6d95449d95-7p5cv"
Sep 07 12:26:57.012798 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:26:57.012302    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-b-d86c0acb-kserve-6d95449d95-7p5cv"
Sep 07 12:26:57.024951 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:26:57.024926    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-b-d86c0acb-kserve-6d95449d95-7p5cv"
Sep 07 12:26:57.365372 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:26:57.365290    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-b-d86c0acb-kserve-6d95449d95-7p5cv"
Sep 07 12:27:09.306920 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:27:09.306888    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-4nfbc_b34ca85d-ac51-4724-a508-1026106b3f3a/console-operator/2.log"
Sep 07 12:27:09.311775 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:27:09.311751    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-4nfbc_b34ca85d-ac51-4724-a508-1026106b3f3a/console-operator/2.log"
Sep 07 12:27:18.721185 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:27:18.721150    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-d86c0acb-kserve-f8fbddfc5-hhf4f"]
Sep 07 12:27:18.743563 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:27:18.743530    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:27:20.695308 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:27:20.695270    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-d86c0acb-kserve-6d95449d95-7p5cv"]
Sep 07 12:27:20.714443 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:27:20.714391    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:27:20.904267 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:27:20.904232    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-p4l4w"]
Sep 07 12:27:29.705234 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:27:29.705199    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-p4l4w"]
Sep 07 12:27:33.741139 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:27:33.741027    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-a-d86c0acb-kserve-f8fbddfc5-hhf4f" podUID="2e85b1df-fbca-4e67-8d0e-3c0cf1be4dce" containerName="main" containerID="cri-o://b3581a650c66bb70a6e75a6189daea8f3108f3bede548a9aaff10d173c15ba33" gracePeriod=45
Sep 07 12:27:33.974538 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:27:33.974511    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-d86c0acb-kserve-f8fbddfc5-hhf4f"
Sep 07 12:27:34.055434 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:27:34.055335    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2e85b1df-fbca-4e67-8d0e-3c0cf1be4dce-tls-certs\") pod \"2e85b1df-fbca-4e67-8d0e-3c0cf1be4dce\" (UID: \"2e85b1df-fbca-4e67-8d0e-3c0cf1be4dce\") "
Sep 07 12:27:34.055434 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:27:34.055384    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2e85b1df-fbca-4e67-8d0e-3c0cf1be4dce-model-cache\") pod \"2e85b1df-fbca-4e67-8d0e-3c0cf1be4dce\" (UID: \"2e85b1df-fbca-4e67-8d0e-3c0cf1be4dce\") "
Sep 07 12:27:34.055434 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:27:34.055427    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2e85b1df-fbca-4e67-8d0e-3c0cf1be4dce-home\") pod \"2e85b1df-fbca-4e67-8d0e-3c0cf1be4dce\" (UID: \"2e85b1df-fbca-4e67-8d0e-3c0cf1be4dce\") "
Sep 07 12:27:34.055434 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:27:34.055446    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2e85b1df-fbca-4e67-8d0e-3c0cf1be4dce-dshm\") pod \"2e85b1df-fbca-4e67-8d0e-3c0cf1be4dce\" (UID: \"2e85b1df-fbca-4e67-8d0e-3c0cf1be4dce\") "
Sep 07 12:27:34.055769 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:27:34.055532    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2e85b1df-fbca-4e67-8d0e-3c0cf1be4dce-tmp-dir\") pod \"2e85b1df-fbca-4e67-8d0e-3c0cf1be4dce\" (UID: \"2e85b1df-fbca-4e67-8d0e-3c0cf1be4dce\") "
Sep 07 12:27:34.055769 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:27:34.055570    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-czgg5\" (UniqueName: \"kubernetes.io/projected/2e85b1df-fbca-4e67-8d0e-3c0cf1be4dce-kube-api-access-czgg5\") pod \"2e85b1df-fbca-4e67-8d0e-3c0cf1be4dce\" (UID: \"2e85b1df-fbca-4e67-8d0e-3c0cf1be4dce\") "
Sep 07 12:27:34.055769 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:27:34.055670    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2e85b1df-fbca-4e67-8d0e-3c0cf1be4dce-home" (OuterVolumeSpecName: "home") pod "2e85b1df-fbca-4e67-8d0e-3c0cf1be4dce" (UID: "2e85b1df-fbca-4e67-8d0e-3c0cf1be4dce"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 12:27:34.055769 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:27:34.055682    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2e85b1df-fbca-4e67-8d0e-3c0cf1be4dce-model-cache" (OuterVolumeSpecName: "model-cache") pod "2e85b1df-fbca-4e67-8d0e-3c0cf1be4dce" (UID: "2e85b1df-fbca-4e67-8d0e-3c0cf1be4dce"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 12:27:34.055978 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:27:34.055794    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2e85b1df-fbca-4e67-8d0e-3c0cf1be4dce-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "2e85b1df-fbca-4e67-8d0e-3c0cf1be4dce" (UID: "2e85b1df-fbca-4e67-8d0e-3c0cf1be4dce"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 12:27:34.055978 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:27:34.055890    2563 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2e85b1df-fbca-4e67-8d0e-3c0cf1be4dce-model-cache\") on node \"ip-10-0-135-173.ec2.internal\" DevicePath \"\""
Sep 07 12:27:34.055978 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:27:34.055909    2563 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2e85b1df-fbca-4e67-8d0e-3c0cf1be4dce-home\") on node \"ip-10-0-135-173.ec2.internal\" DevicePath \"\""
Sep 07 12:27:34.055978 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:27:34.055923    2563 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2e85b1df-fbca-4e67-8d0e-3c0cf1be4dce-tmp-dir\") on node \"ip-10-0-135-173.ec2.internal\" DevicePath \"\""
Sep 07 12:27:34.057574 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:27:34.057555    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2e85b1df-fbca-4e67-8d0e-3c0cf1be4dce-dshm" (OuterVolumeSpecName: "dshm") pod "2e85b1df-fbca-4e67-8d0e-3c0cf1be4dce" (UID: "2e85b1df-fbca-4e67-8d0e-3c0cf1be4dce"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 12:27:34.057660 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:27:34.057642    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2e85b1df-fbca-4e67-8d0e-3c0cf1be4dce-kube-api-access-czgg5" (OuterVolumeSpecName: "kube-api-access-czgg5") pod "2e85b1df-fbca-4e67-8d0e-3c0cf1be4dce" (UID: "2e85b1df-fbca-4e67-8d0e-3c0cf1be4dce"). InnerVolumeSpecName "kube-api-access-czgg5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 12:27:34.057707 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:27:34.057674    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2e85b1df-fbca-4e67-8d0e-3c0cf1be4dce-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "2e85b1df-fbca-4e67-8d0e-3c0cf1be4dce" (UID: "2e85b1df-fbca-4e67-8d0e-3c0cf1be4dce"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 07 12:27:34.157274 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:27:34.157242    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-czgg5\" (UniqueName: \"kubernetes.io/projected/2e85b1df-fbca-4e67-8d0e-3c0cf1be4dce-kube-api-access-czgg5\") on node \"ip-10-0-135-173.ec2.internal\" DevicePath \"\""
Sep 07 12:27:34.157274 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:27:34.157268    2563 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2e85b1df-fbca-4e67-8d0e-3c0cf1be4dce-tls-certs\") on node \"ip-10-0-135-173.ec2.internal\" DevicePath \"\""
Sep 07 12:27:34.157274 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:27:34.157277    2563 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2e85b1df-fbca-4e67-8d0e-3c0cf1be4dce-dshm\") on node \"ip-10-0-135-173.ec2.internal\" DevicePath \"\""
Sep 07 12:27:34.536756 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:27:34.536725    2563 generic.go:356] "Generic (PLEG): container finished" podID="2e85b1df-fbca-4e67-8d0e-3c0cf1be4dce" containerID="b3581a650c66bb70a6e75a6189daea8f3108f3bede548a9aaff10d173c15ba33" exitCode=0
Sep 07 12:27:34.536932 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:27:34.536766    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-d86c0acb-kserve-f8fbddfc5-hhf4f"
Sep 07 12:27:34.536932 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:27:34.536811    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-d86c0acb-kserve-f8fbddfc5-hhf4f" event={"ID":"2e85b1df-fbca-4e67-8d0e-3c0cf1be4dce","Type":"ContainerDied","Data":"b3581a650c66bb70a6e75a6189daea8f3108f3bede548a9aaff10d173c15ba33"}
Sep 07 12:27:34.536932 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:27:34.536851    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-d86c0acb-kserve-f8fbddfc5-hhf4f" event={"ID":"2e85b1df-fbca-4e67-8d0e-3c0cf1be4dce","Type":"ContainerDied","Data":"51e4451ae671668816c31255b6563b76ddb29287dd3e2abba950e5396961f5ed"}
Sep 07 12:27:34.536932 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:27:34.536869    2563 scope.go:117] "RemoveContainer" containerID="b3581a650c66bb70a6e75a6189daea8f3108f3bede548a9aaff10d173c15ba33"
Sep 07 12:27:34.546140 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:27:34.546123    2563 scope.go:117] "RemoveContainer" containerID="b3581a650c66bb70a6e75a6189daea8f3108f3bede548a9aaff10d173c15ba33"
Sep 07 12:27:34.546473 ip-10-0-135-173 kubenswrapper[2563]: E0907 12:27:34.546400    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b3581a650c66bb70a6e75a6189daea8f3108f3bede548a9aaff10d173c15ba33\": container with ID starting with b3581a650c66bb70a6e75a6189daea8f3108f3bede548a9aaff10d173c15ba33 not found: ID does not exist" containerID="b3581a650c66bb70a6e75a6189daea8f3108f3bede548a9aaff10d173c15ba33"
Sep 07 12:27:34.546473 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:27:34.546452    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b3581a650c66bb70a6e75a6189daea8f3108f3bede548a9aaff10d173c15ba33"} err="failed to get container status \"b3581a650c66bb70a6e75a6189daea8f3108f3bede548a9aaff10d173c15ba33\": rpc error: code = NotFound desc = could not find container \"b3581a650c66bb70a6e75a6189daea8f3108f3bede548a9aaff10d173c15ba33\": container with ID starting with b3581a650c66bb70a6e75a6189daea8f3108f3bede548a9aaff10d173c15ba33 not found: ID does not exist"
Sep 07 12:27:34.556680 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:27:34.556659    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-d86c0acb-kserve-f8fbddfc5-hhf4f"]
Sep 07 12:27:34.560596 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:27:34.560576    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-a-d86c0acb-kserve-f8fbddfc5-hhf4f"]
Sep 07 12:27:35.041403 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:27:35.041373    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2e85b1df-fbca-4e67-8d0e-3c0cf1be4dce" path="/var/lib/kubelet/pods/2e85b1df-fbca-4e67-8d0e-3c0cf1be4dce/volumes"
Sep 07 12:27:35.715087 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:27:35.715049    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-b-d86c0acb-kserve-6d95449d95-7p5cv" podUID="cec4771b-2ae0-4171-bcc9-d37222b84764" containerName="main" containerID="cri-o://9a0508cb3735d5c5ad1b77228749c74f274bef33bc3189c2298432ea593ed822" gracePeriod=45
Sep 07 12:27:35.967670 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:27:35.967606    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-d86c0acb-kserve-6d95449d95-7p5cv"
Sep 07 12:27:36.073740 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:27:36.073706    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-trknf\" (UniqueName: \"kubernetes.io/projected/cec4771b-2ae0-4171-bcc9-d37222b84764-kube-api-access-trknf\") pod \"cec4771b-2ae0-4171-bcc9-d37222b84764\" (UID: \"cec4771b-2ae0-4171-bcc9-d37222b84764\") "
Sep 07 12:27:36.074230 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:27:36.073758    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cec4771b-2ae0-4171-bcc9-d37222b84764-model-cache\") pod \"cec4771b-2ae0-4171-bcc9-d37222b84764\" (UID: \"cec4771b-2ae0-4171-bcc9-d37222b84764\") "
Sep 07 12:27:36.074230 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:27:36.073786    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cec4771b-2ae0-4171-bcc9-d37222b84764-tmp-dir\") pod \"cec4771b-2ae0-4171-bcc9-d37222b84764\" (UID: \"cec4771b-2ae0-4171-bcc9-d37222b84764\") "
Sep 07 12:27:36.074230 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:27:36.073815    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cec4771b-2ae0-4171-bcc9-d37222b84764-dshm\") pod \"cec4771b-2ae0-4171-bcc9-d37222b84764\" (UID: \"cec4771b-2ae0-4171-bcc9-d37222b84764\") "
Sep 07 12:27:36.074230 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:27:36.073884    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cec4771b-2ae0-4171-bcc9-d37222b84764-home\") pod \"cec4771b-2ae0-4171-bcc9-d37222b84764\" (UID: \"cec4771b-2ae0-4171-bcc9-d37222b84764\") "
Sep 07 12:27:36.074230 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:27:36.073923    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cec4771b-2ae0-4171-bcc9-d37222b84764-tls-certs\") pod \"cec4771b-2ae0-4171-bcc9-d37222b84764\" (UID: \"cec4771b-2ae0-4171-bcc9-d37222b84764\") "
Sep 07 12:27:36.074230 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:27:36.074024    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cec4771b-2ae0-4171-bcc9-d37222b84764-model-cache" (OuterVolumeSpecName: "model-cache") pod "cec4771b-2ae0-4171-bcc9-d37222b84764" (UID: "cec4771b-2ae0-4171-bcc9-d37222b84764"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 12:27:36.074230 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:27:36.074060    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cec4771b-2ae0-4171-bcc9-d37222b84764-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "cec4771b-2ae0-4171-bcc9-d37222b84764" (UID: "cec4771b-2ae0-4171-bcc9-d37222b84764"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 12:27:36.074230 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:27:36.074163    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cec4771b-2ae0-4171-bcc9-d37222b84764-home" (OuterVolumeSpecName: "home") pod "cec4771b-2ae0-4171-bcc9-d37222b84764" (UID: "cec4771b-2ae0-4171-bcc9-d37222b84764"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 12:27:36.074568 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:27:36.074278    2563 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cec4771b-2ae0-4171-bcc9-d37222b84764-home\") on node \"ip-10-0-135-173.ec2.internal\" DevicePath \"\""
Sep 07 12:27:36.074568 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:27:36.074295    2563 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cec4771b-2ae0-4171-bcc9-d37222b84764-model-cache\") on node \"ip-10-0-135-173.ec2.internal\" DevicePath \"\""
Sep 07 12:27:36.074568 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:27:36.074309    2563 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cec4771b-2ae0-4171-bcc9-d37222b84764-tmp-dir\") on node \"ip-10-0-135-173.ec2.internal\" DevicePath \"\""
Sep 07 12:27:36.075950 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:27:36.075929    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cec4771b-2ae0-4171-bcc9-d37222b84764-dshm" (OuterVolumeSpecName: "dshm") pod "cec4771b-2ae0-4171-bcc9-d37222b84764" (UID: "cec4771b-2ae0-4171-bcc9-d37222b84764"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 12:27:36.076022 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:27:36.076002    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cec4771b-2ae0-4171-bcc9-d37222b84764-kube-api-access-trknf" (OuterVolumeSpecName: "kube-api-access-trknf") pod "cec4771b-2ae0-4171-bcc9-d37222b84764" (UID: "cec4771b-2ae0-4171-bcc9-d37222b84764"). InnerVolumeSpecName "kube-api-access-trknf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 12:27:36.076075 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:27:36.076044    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cec4771b-2ae0-4171-bcc9-d37222b84764-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "cec4771b-2ae0-4171-bcc9-d37222b84764" (UID: "cec4771b-2ae0-4171-bcc9-d37222b84764"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 07 12:27:36.175663 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:27:36.175634    2563 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cec4771b-2ae0-4171-bcc9-d37222b84764-dshm\") on node \"ip-10-0-135-173.ec2.internal\" DevicePath \"\""
Sep 07 12:27:36.175663 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:27:36.175657    2563 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cec4771b-2ae0-4171-bcc9-d37222b84764-tls-certs\") on node \"ip-10-0-135-173.ec2.internal\" DevicePath \"\""
Sep 07 12:27:36.175663 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:27:36.175667    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-trknf\" (UniqueName: \"kubernetes.io/projected/cec4771b-2ae0-4171-bcc9-d37222b84764-kube-api-access-trknf\") on node \"ip-10-0-135-173.ec2.internal\" DevicePath \"\""
Sep 07 12:27:36.550385 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:27:36.550349    2563 generic.go:356] "Generic (PLEG): container finished" podID="cec4771b-2ae0-4171-bcc9-d37222b84764" containerID="9a0508cb3735d5c5ad1b77228749c74f274bef33bc3189c2298432ea593ed822" exitCode=0
Sep 07 12:27:36.550585 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:27:36.550401    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-d86c0acb-kserve-6d95449d95-7p5cv" event={"ID":"cec4771b-2ae0-4171-bcc9-d37222b84764","Type":"ContainerDied","Data":"9a0508cb3735d5c5ad1b77228749c74f274bef33bc3189c2298432ea593ed822"}
Sep 07 12:27:36.550585 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:27:36.550406    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-d86c0acb-kserve-6d95449d95-7p5cv"
Sep 07 12:27:36.550585 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:27:36.550452    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-d86c0acb-kserve-6d95449d95-7p5cv" event={"ID":"cec4771b-2ae0-4171-bcc9-d37222b84764","Type":"ContainerDied","Data":"bc7ca93ccb24e9571a16c76e41329d96c5a57882b440c304c9fe2088f30f8f99"}
Sep 07 12:27:36.550585 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:27:36.550469    2563 scope.go:117] "RemoveContainer" containerID="9a0508cb3735d5c5ad1b77228749c74f274bef33bc3189c2298432ea593ed822"
Sep 07 12:27:36.560463 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:27:36.560446    2563 scope.go:117] "RemoveContainer" containerID="9a0508cb3735d5c5ad1b77228749c74f274bef33bc3189c2298432ea593ed822"
Sep 07 12:27:36.560749 ip-10-0-135-173 kubenswrapper[2563]: E0907 12:27:36.560729    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9a0508cb3735d5c5ad1b77228749c74f274bef33bc3189c2298432ea593ed822\": container with ID starting with 9a0508cb3735d5c5ad1b77228749c74f274bef33bc3189c2298432ea593ed822 not found: ID does not exist" containerID="9a0508cb3735d5c5ad1b77228749c74f274bef33bc3189c2298432ea593ed822"
Sep 07 12:27:36.560801 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:27:36.560755    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9a0508cb3735d5c5ad1b77228749c74f274bef33bc3189c2298432ea593ed822"} err="failed to get container status \"9a0508cb3735d5c5ad1b77228749c74f274bef33bc3189c2298432ea593ed822\": rpc error: code = NotFound desc = could not find container \"9a0508cb3735d5c5ad1b77228749c74f274bef33bc3189c2298432ea593ed822\": container with ID starting with 9a0508cb3735d5c5ad1b77228749c74f274bef33bc3189c2298432ea593ed822 not found: ID does not exist"
Sep 07 12:27:36.570325 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:27:36.570305    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-d86c0acb-kserve-6d95449d95-7p5cv"]
Sep 07 12:27:36.575277 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:27:36.575257    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-b-d86c0acb-kserve-6d95449d95-7p5cv"]
Sep 07 12:27:37.040303 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:27:37.040272    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cec4771b-2ae0-4171-bcc9-d37222b84764" path="/var/lib/kubelet/pods/cec4771b-2ae0-4171-bcc9-d37222b84764/volumes"
Sep 07 12:27:39.698273 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:27:39.698244    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-8cd4c9f7c-8xjrp"]
Sep 07 12:27:39.698645 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:27:39.698637    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cec4771b-2ae0-4171-bcc9-d37222b84764" containerName="main"
Sep 07 12:27:39.698693 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:27:39.698650    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="cec4771b-2ae0-4171-bcc9-d37222b84764" containerName="main"
Sep 07 12:27:39.698693 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:27:39.698670    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2e85b1df-fbca-4e67-8d0e-3c0cf1be4dce" containerName="main"
Sep 07 12:27:39.698693 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:27:39.698675    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="2e85b1df-fbca-4e67-8d0e-3c0cf1be4dce" containerName="main"
Sep 07 12:27:39.698794 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:27:39.698728    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="2e85b1df-fbca-4e67-8d0e-3c0cf1be4dce" containerName="main"
Sep 07 12:27:39.698794 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:27:39.698735    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="cec4771b-2ae0-4171-bcc9-d37222b84764" containerName="main"
Sep 07 12:27:39.703153 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:27:39.703131    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-8cd4c9f7c-8xjrp"
Sep 07 12:27:39.705403 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:27:39.705371    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Sep 07 12:27:39.705557 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:27:39.705401    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Sep 07 12:27:39.705557 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:27:39.705373    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-p4vz9\""
Sep 07 12:27:39.705557 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:27:39.705456    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\""
Sep 07 12:27:39.705557 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:27:39.705491    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 07 12:27:39.712338 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:27:39.712315    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-8cd4c9f7c-8xjrp"]
Sep 07 12:27:39.809870 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:27:39.809843    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a0010a41-2f6e-4e92-a473-2d1564063b7e-metrics-certs\") pod \"maas-api-8cd4c9f7c-8xjrp\" (UID: \"a0010a41-2f6e-4e92-a473-2d1564063b7e\") " pod="odh-ai-gateway-infra/maas-api-8cd4c9f7c-8xjrp"
Sep 07 12:27:39.810009 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:27:39.809881    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sd224\" (UniqueName: \"kubernetes.io/projected/a0010a41-2f6e-4e92-a473-2d1564063b7e-kube-api-access-sd224\") pod \"maas-api-8cd4c9f7c-8xjrp\" (UID: \"a0010a41-2f6e-4e92-a473-2d1564063b7e\") " pod="odh-ai-gateway-infra/maas-api-8cd4c9f7c-8xjrp"
Sep 07 12:27:39.810009 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:27:39.809909    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a0010a41-2f6e-4e92-a473-2d1564063b7e-openshift-service-ca\") pod \"maas-api-8cd4c9f7c-8xjrp\" (UID: \"a0010a41-2f6e-4e92-a473-2d1564063b7e\") " pod="odh-ai-gateway-infra/maas-api-8cd4c9f7c-8xjrp"
Sep 07 12:27:39.810009 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:27:39.809966    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a0010a41-2f6e-4e92-a473-2d1564063b7e-maas-api-tls\") pod \"maas-api-8cd4c9f7c-8xjrp\" (UID: \"a0010a41-2f6e-4e92-a473-2d1564063b7e\") " pod="odh-ai-gateway-infra/maas-api-8cd4c9f7c-8xjrp"
Sep 07 12:27:39.911404 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:27:39.911373    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a0010a41-2f6e-4e92-a473-2d1564063b7e-metrics-certs\") pod \"maas-api-8cd4c9f7c-8xjrp\" (UID: \"a0010a41-2f6e-4e92-a473-2d1564063b7e\") " pod="odh-ai-gateway-infra/maas-api-8cd4c9f7c-8xjrp"
Sep 07 12:27:39.911580 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:27:39.911413    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sd224\" (UniqueName: \"kubernetes.io/projected/a0010a41-2f6e-4e92-a473-2d1564063b7e-kube-api-access-sd224\") pod \"maas-api-8cd4c9f7c-8xjrp\" (UID: \"a0010a41-2f6e-4e92-a473-2d1564063b7e\") " pod="odh-ai-gateway-infra/maas-api-8cd4c9f7c-8xjrp"
Sep 07 12:27:39.911580 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:27:39.911472    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a0010a41-2f6e-4e92-a473-2d1564063b7e-openshift-service-ca\") pod \"maas-api-8cd4c9f7c-8xjrp\" (UID: \"a0010a41-2f6e-4e92-a473-2d1564063b7e\") " pod="odh-ai-gateway-infra/maas-api-8cd4c9f7c-8xjrp"
Sep 07 12:27:39.911580 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:27:39.911501    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a0010a41-2f6e-4e92-a473-2d1564063b7e-maas-api-tls\") pod \"maas-api-8cd4c9f7c-8xjrp\" (UID: \"a0010a41-2f6e-4e92-a473-2d1564063b7e\") " pod="odh-ai-gateway-infra/maas-api-8cd4c9f7c-8xjrp"
Sep 07 12:27:39.912220 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:27:39.912201    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a0010a41-2f6e-4e92-a473-2d1564063b7e-openshift-service-ca\") pod \"maas-api-8cd4c9f7c-8xjrp\" (UID: \"a0010a41-2f6e-4e92-a473-2d1564063b7e\") " pod="odh-ai-gateway-infra/maas-api-8cd4c9f7c-8xjrp"
Sep 07 12:27:39.913859 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:27:39.913838    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a0010a41-2f6e-4e92-a473-2d1564063b7e-maas-api-tls\") pod \"maas-api-8cd4c9f7c-8xjrp\" (UID: \"a0010a41-2f6e-4e92-a473-2d1564063b7e\") " pod="odh-ai-gateway-infra/maas-api-8cd4c9f7c-8xjrp"
Sep 07 12:27:39.913947 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:27:39.913889    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a0010a41-2f6e-4e92-a473-2d1564063b7e-metrics-certs\") pod \"maas-api-8cd4c9f7c-8xjrp\" (UID: \"a0010a41-2f6e-4e92-a473-2d1564063b7e\") " pod="odh-ai-gateway-infra/maas-api-8cd4c9f7c-8xjrp"
Sep 07 12:27:39.917321 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:27:39.917298    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sd224\" (UniqueName: \"kubernetes.io/projected/a0010a41-2f6e-4e92-a473-2d1564063b7e-kube-api-access-sd224\") pod \"maas-api-8cd4c9f7c-8xjrp\" (UID: \"a0010a41-2f6e-4e92-a473-2d1564063b7e\") " pod="odh-ai-gateway-infra/maas-api-8cd4c9f7c-8xjrp"
Sep 07 12:27:40.015573 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:27:40.015499    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-8cd4c9f7c-8xjrp"
Sep 07 12:27:40.195454 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:27:40.195402    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-p4l4w"]
Sep 07 12:27:40.284288 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:27:40.284267    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-8cd4c9f7c-8xjrp"]
Sep 07 12:27:40.286305 ip-10-0-135-173 kubenswrapper[2563]: W0907 12:27:40.286279    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda0010a41_2f6e_4e92_a473_2d1564063b7e.slice/crio-fd093f99733d49890ef59d1333e4b418aa26123a5a4089afff747209d04b0d48 WatchSource:0}: Error finding container fd093f99733d49890ef59d1333e4b418aa26123a5a4089afff747209d04b0d48: Status 404 returned error can't find the container with id fd093f99733d49890ef59d1333e4b418aa26123a5a4089afff747209d04b0d48
Sep 07 12:27:40.287600 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:27:40.287581    2563 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 07 12:27:40.572580 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:27:40.572503    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-8cd4c9f7c-8xjrp" event={"ID":"a0010a41-2f6e-4e92-a473-2d1564063b7e","Type":"ContainerStarted","Data":"fd093f99733d49890ef59d1333e4b418aa26123a5a4089afff747209d04b0d48"}
Sep 07 12:27:41.579132 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:27:41.579093    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-8cd4c9f7c-8xjrp" event={"ID":"a0010a41-2f6e-4e92-a473-2d1564063b7e","Type":"ContainerStarted","Data":"aede888c81512af14476abf87f0245e62f0148aad085d2d437a621d0c716314c"}
Sep 07 12:27:41.579533 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:27:41.579211    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-8cd4c9f7c-8xjrp"
Sep 07 12:27:41.595852 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:27:41.595807    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-8cd4c9f7c-8xjrp" podStartSLOduration=2.168279603 podStartE2EDuration="2.595793201s" totalImagesPullingTime="427.513598ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 12:27:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 12:27:41.593867752 +0000 UTC m=+2433.112599567" watchObservedRunningTime="2026-09-07 12:27:41.595793201 +0000 UTC m=+2433.114525013"
Sep 07 12:27:47.125470 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:27:47.125435    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-8cd4c9f7c-8xjrp"]
Sep 07 12:27:47.125877 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:27:47.125619    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-8cd4c9f7c-8xjrp" podUID="a0010a41-2f6e-4e92-a473-2d1564063b7e" containerName="maas-api" containerID="cri-o://aede888c81512af14476abf87f0245e62f0148aad085d2d437a621d0c716314c" gracePeriod=30
Sep 07 12:27:47.131153 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:27:47.131129    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-8cd4c9f7c-8xjrp"
Sep 07 12:27:47.359726 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:27:47.359703    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-8cd4c9f7c-8xjrp"
Sep 07 12:27:47.406520 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:27:47.406458    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-p4l4w"]
Sep 07 12:27:47.483566 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:27:47.483542    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sd224\" (UniqueName: \"kubernetes.io/projected/a0010a41-2f6e-4e92-a473-2d1564063b7e-kube-api-access-sd224\") pod \"a0010a41-2f6e-4e92-a473-2d1564063b7e\" (UID: \"a0010a41-2f6e-4e92-a473-2d1564063b7e\") "
Sep 07 12:27:47.483691 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:27:47.483609    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a0010a41-2f6e-4e92-a473-2d1564063b7e-metrics-certs\") pod \"a0010a41-2f6e-4e92-a473-2d1564063b7e\" (UID: \"a0010a41-2f6e-4e92-a473-2d1564063b7e\") "
Sep 07 12:27:47.483733 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:27:47.483714    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a0010a41-2f6e-4e92-a473-2d1564063b7e-maas-api-tls\") pod \"a0010a41-2f6e-4e92-a473-2d1564063b7e\" (UID: \"a0010a41-2f6e-4e92-a473-2d1564063b7e\") "
Sep 07 12:27:47.483853 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:27:47.483835    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a0010a41-2f6e-4e92-a473-2d1564063b7e-openshift-service-ca\") pod \"a0010a41-2f6e-4e92-a473-2d1564063b7e\" (UID: \"a0010a41-2f6e-4e92-a473-2d1564063b7e\") "
Sep 07 12:27:47.484228 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:27:47.484199    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a0010a41-2f6e-4e92-a473-2d1564063b7e-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "a0010a41-2f6e-4e92-a473-2d1564063b7e" (UID: "a0010a41-2f6e-4e92-a473-2d1564063b7e"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 07 12:27:47.486140 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:27:47.486107    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a0010a41-2f6e-4e92-a473-2d1564063b7e-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "a0010a41-2f6e-4e92-a473-2d1564063b7e" (UID: "a0010a41-2f6e-4e92-a473-2d1564063b7e"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 07 12:27:47.486233 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:27:47.486157    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a0010a41-2f6e-4e92-a473-2d1564063b7e-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "a0010a41-2f6e-4e92-a473-2d1564063b7e" (UID: "a0010a41-2f6e-4e92-a473-2d1564063b7e"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 07 12:27:47.486233 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:27:47.486194    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a0010a41-2f6e-4e92-a473-2d1564063b7e-kube-api-access-sd224" (OuterVolumeSpecName: "kube-api-access-sd224") pod "a0010a41-2f6e-4e92-a473-2d1564063b7e" (UID: "a0010a41-2f6e-4e92-a473-2d1564063b7e"). InnerVolumeSpecName "kube-api-access-sd224". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 12:27:47.584853 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:27:47.584827    2563 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a0010a41-2f6e-4e92-a473-2d1564063b7e-openshift-service-ca\") on node \"ip-10-0-135-173.ec2.internal\" DevicePath \"\""
Sep 07 12:27:47.584853 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:27:47.584849    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sd224\" (UniqueName: \"kubernetes.io/projected/a0010a41-2f6e-4e92-a473-2d1564063b7e-kube-api-access-sd224\") on node \"ip-10-0-135-173.ec2.internal\" DevicePath \"\""
Sep 07 12:27:47.584981 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:27:47.584858    2563 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a0010a41-2f6e-4e92-a473-2d1564063b7e-metrics-certs\") on node \"ip-10-0-135-173.ec2.internal\" DevicePath \"\""
Sep 07 12:27:47.584981 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:27:47.584867    2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a0010a41-2f6e-4e92-a473-2d1564063b7e-maas-api-tls\") on node \"ip-10-0-135-173.ec2.internal\" DevicePath \"\""
Sep 07 12:27:47.617301 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:27:47.617272    2563 generic.go:356] "Generic (PLEG): container finished" podID="a0010a41-2f6e-4e92-a473-2d1564063b7e" containerID="aede888c81512af14476abf87f0245e62f0148aad085d2d437a621d0c716314c" exitCode=0
Sep 07 12:27:47.617400 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:27:47.617317    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-8cd4c9f7c-8xjrp"
Sep 07 12:27:47.617400 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:27:47.617353    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-8cd4c9f7c-8xjrp" event={"ID":"a0010a41-2f6e-4e92-a473-2d1564063b7e","Type":"ContainerDied","Data":"aede888c81512af14476abf87f0245e62f0148aad085d2d437a621d0c716314c"}
Sep 07 12:27:47.617503 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:27:47.617399    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-8cd4c9f7c-8xjrp" event={"ID":"a0010a41-2f6e-4e92-a473-2d1564063b7e","Type":"ContainerDied","Data":"fd093f99733d49890ef59d1333e4b418aa26123a5a4089afff747209d04b0d48"}
Sep 07 12:27:47.617503 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:27:47.617433    2563 scope.go:117] "RemoveContainer" containerID="aede888c81512af14476abf87f0245e62f0148aad085d2d437a621d0c716314c"
Sep 07 12:27:47.627214 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:27:47.627195    2563 scope.go:117] "RemoveContainer" containerID="aede888c81512af14476abf87f0245e62f0148aad085d2d437a621d0c716314c"
Sep 07 12:27:47.627489 ip-10-0-135-173 kubenswrapper[2563]: E0907 12:27:47.627470    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"aede888c81512af14476abf87f0245e62f0148aad085d2d437a621d0c716314c\": container with ID starting with aede888c81512af14476abf87f0245e62f0148aad085d2d437a621d0c716314c not found: ID does not exist" containerID="aede888c81512af14476abf87f0245e62f0148aad085d2d437a621d0c716314c"
Sep 07 12:27:47.627550 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:27:47.627496    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"aede888c81512af14476abf87f0245e62f0148aad085d2d437a621d0c716314c"} err="failed to get container status \"aede888c81512af14476abf87f0245e62f0148aad085d2d437a621d0c716314c\": rpc error: code = NotFound desc = could not find container \"aede888c81512af14476abf87f0245e62f0148aad085d2d437a621d0c716314c\": container with ID starting with aede888c81512af14476abf87f0245e62f0148aad085d2d437a621d0c716314c not found: ID does not exist"
Sep 07 12:27:47.637357 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:27:47.637335    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-8cd4c9f7c-8xjrp"]
Sep 07 12:27:47.640346 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:27:47.640326    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-8cd4c9f7c-8xjrp"]
Sep 07 12:27:49.040085 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:27:49.040050    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a0010a41-2f6e-4e92-a473-2d1564063b7e" path="/var/lib/kubelet/pods/a0010a41-2f6e-4e92-a473-2d1564063b7e/volumes"
Sep 07 12:27:54.101234 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:27:54.101199    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-p4l4w"]
Sep 07 12:28:00.701189 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:28:00.701159    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-p4l4w"]
Sep 07 12:28:08.099605 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:28:08.099567    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-p4l4w"]
Sep 07 12:28:09.295576 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:28:09.295546    2563 scope.go:117] "RemoveContainer" containerID="776aef9a45d4d28be55ab6dd7c2f74e72f557302aa9dd6fb1355858daae31f0f"
Sep 07 12:28:11.491489 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:28:11.491454    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-55cc594f49-5tr5s_8c6c9a5e-f913-4bef-adbc-df934f10b618/authorino/0.log"
Sep 07 12:28:17.324076 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:28:17.324046    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-648db6df7b-c82m6_9fe85f81-1199-4c0b-b58e-036142982b1a/copy-manifests/0.log"
Sep 07 12:28:17.330193 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:28:17.330169    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-648db6df7b-c82m6_9fe85f81-1199-4c0b-b58e-036142982b1a/manager/0.log"
Sep 07 12:28:17.971006 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:28:17.970973    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29813055-qcgd9_8de764fa-7c1b-4e38-a8d3-33b96f0825d3/cleanup/0.log"
Sep 07 12:28:18.827142 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:28:18.827109    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-55cc594f49-5tr5s_8c6c9a5e-f913-4bef-adbc-df934f10b618/authorino/0.log"
Sep 07 12:28:19.388324 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:28:19.388293    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-85d9db4d5d-p4l4w_697c2418-985e-43e8-b5bc-a2fc199cd677/limitador/0.log"
Sep 07 12:28:20.075049 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:28:20.075010    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5b6777d477-c7pps_4a58840b-efff-43a4-ac2f-906a5f1b4bdc/kube-auth-proxy/0.log"
Sep 07 12:28:21.197547 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:28:21.197518    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-embedding-simulated-kserve-bcc9ff8d8-5nncz_e0abbddf-d934-490d-b520-b4a2c0dab142/main/0.log"
Sep 07 12:28:21.510275 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:28:21.510192    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-74b68447df-nj42c_47d91c30-7252-4e9d-944f-494b41376f94/main/0.log"
Sep 07 12:28:21.619532 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:28:21.619501    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-57fc66f697-7n5g7_dfdfb25e-f951-4d2c-87b0-26ee1ec10310/main/0.log"
Sep 07 12:28:26.705161 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:28:26.705119    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:28:27.937718 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:28:27.937679    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:28:30.007179 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:28:30.007144    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-nzzj9_e92172b6-ddda-4b65-81b3-987b307e272e/global-pull-secret-syncer/0.log"
Sep 07 12:28:30.054253 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:28:30.054223    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log"
Sep 07 12:28:30.165812 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:28:30.165788    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-135-173.ec2.internal_7d94e6a6c726d03776a8387d983ae9ef/haproxy/0.log"
Sep 07 12:28:32.786355 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:28:32.786316    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:28:34.973067 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:28:34.973036    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-55cc594f49-5tr5s_8c6c9a5e-f913-4bef-adbc-df934f10b618/authorino/0.log"
Sep 07 12:28:35.155826 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:28:35.155796    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-85d9db4d5d-p4l4w_697c2418-985e-43e8-b5bc-a2fc199cd677/limitador/0.log"
Sep 07 12:28:36.727475 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:28:36.727438    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_5aade16a-29eb-43c8-8bc5-9889f0356fbf/alertmanager/0.log"
Sep 07 12:28:36.747991 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:28:36.747967    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_5aade16a-29eb-43c8-8bc5-9889f0356fbf/config-reloader/0.log"
Sep 07 12:28:36.769617 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:28:36.769592    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_5aade16a-29eb-43c8-8bc5-9889f0356fbf/kube-rbac-proxy-web/0.log"
Sep 07 12:28:36.790358 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:28:36.790335    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_5aade16a-29eb-43c8-8bc5-9889f0356fbf/kube-rbac-proxy/0.log"
Sep 07 12:28:36.811138 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:28:36.811115    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_5aade16a-29eb-43c8-8bc5-9889f0356fbf/kube-rbac-proxy-metric/0.log"
Sep 07 12:28:36.830575 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:28:36.830556    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_5aade16a-29eb-43c8-8bc5-9889f0356fbf/prom-label-proxy/0.log"
Sep 07 12:28:36.850761 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:28:36.850741    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_5aade16a-29eb-43c8-8bc5-9889f0356fbf/init-config-reloader/0.log"
Sep 07 12:28:36.916091 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:28:36.916067    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-74cb57df7-wzrfr_ea883d51-c080-4b94-90e3-c5e26c612f53/kube-state-metrics/0.log"
Sep 07 12:28:36.932688 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:28:36.932669    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-74cb57df7-wzrfr_ea883d51-c080-4b94-90e3-c5e26c612f53/kube-rbac-proxy-main/0.log"
Sep 07 12:28:36.967314 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:28:36.967296    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-74cb57df7-wzrfr_ea883d51-c080-4b94-90e3-c5e26c612f53/kube-rbac-proxy-self/0.log"
Sep 07 12:28:37.034864 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:28:37.034773    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-85bbbbbf4-wt826_2d925afb-5916-41ec-86db-9c11538cfa85/monitoring-plugin/0.log"
Sep 07 12:28:37.081008 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:28:37.080986    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-66czn_67bac748-66ed-4dcb-8a9c-1814e8074836/node-exporter/0.log"
Sep 07 12:28:37.099479 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:28:37.099456    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-66czn_67bac748-66ed-4dcb-8a9c-1814e8074836/kube-rbac-proxy/0.log"
Sep 07 12:28:37.118457 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:28:37.118429    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-66czn_67bac748-66ed-4dcb-8a9c-1814e8074836/init-textfile/0.log"
Sep 07 12:28:37.655645 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:28:37.655608    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-786fcc8696-69dmr_0ea1f45f-67b1-4a1e-86bb-d59f207a7265/telemeter-client/0.log"
Sep 07 12:28:37.676904 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:28:37.676879    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-786fcc8696-69dmr_0ea1f45f-67b1-4a1e-86bb-d59f207a7265/reload/0.log"
Sep 07 12:28:37.701495 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:28:37.701473    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-786fcc8696-69dmr_0ea1f45f-67b1-4a1e-86bb-d59f207a7265/kube-rbac-proxy/0.log"
Sep 07 12:28:37.730850 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:28:37.730830    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-65bdc89fcb-8tfm6_00d4ac48-8eff-4658-963c-b338ff9e2714/thanos-query/0.log"
Sep 07 12:28:37.756047 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:28:37.756023    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-65bdc89fcb-8tfm6_00d4ac48-8eff-4658-963c-b338ff9e2714/kube-rbac-proxy-web/0.log"
Sep 07 12:28:37.774427 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:28:37.774396    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-65bdc89fcb-8tfm6_00d4ac48-8eff-4658-963c-b338ff9e2714/kube-rbac-proxy/0.log"
Sep 07 12:28:37.792711 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:28:37.792681    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-65bdc89fcb-8tfm6_00d4ac48-8eff-4658-963c-b338ff9e2714/prom-label-proxy/0.log"
Sep 07 12:28:37.811592 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:28:37.811572    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-65bdc89fcb-8tfm6_00d4ac48-8eff-4658-963c-b338ff9e2714/kube-rbac-proxy-rules/0.log"
Sep 07 12:28:37.854942 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:28:37.854921    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-65bdc89fcb-8tfm6_00d4ac48-8eff-4658-963c-b338ff9e2714/kube-rbac-proxy-metrics/0.log"
Sep 07 12:28:39.922937 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:28:39.922910    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-4nfbc_b34ca85d-ac51-4724-a508-1026106b3f3a/console-operator/2.log"
Sep 07 12:28:39.932246 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:28:39.932221    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-4nfbc_b34ca85d-ac51-4724-a508-1026106b3f3a/console-operator/3.log"
Sep 07 12:28:40.435558 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:28:40.435527    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-bzv24/perf-node-gather-daemonset-cfqml"]
Sep 07 12:28:40.435899 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:28:40.435887    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a0010a41-2f6e-4e92-a473-2d1564063b7e" containerName="maas-api"
Sep 07 12:28:40.435952 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:28:40.435901    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="a0010a41-2f6e-4e92-a473-2d1564063b7e" containerName="maas-api"
Sep 07 12:28:40.435989 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:28:40.435970    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="a0010a41-2f6e-4e92-a473-2d1564063b7e" containerName="maas-api"
Sep 07 12:28:40.439038 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:28:40.439017    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-bzv24/perf-node-gather-daemonset-cfqml"
Sep 07 12:28:40.440768 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:28:40.440746    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-bzv24\"/\"openshift-service-ca.crt\""
Sep 07 12:28:40.440866 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:28:40.440748    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-bzv24\"/\"default-dockercfg-fprf9\""
Sep 07 12:28:40.440965 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:28:40.440947    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-bzv24\"/\"kube-root-ca.crt\""
Sep 07 12:28:40.447334 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:28:40.447310    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-bzv24/perf-node-gather-daemonset-cfqml"]
Sep 07 12:28:40.536896 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:28:40.536860    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/851047fa-74a7-4554-a75d-88ce0fb5587d-proc\") pod \"perf-node-gather-daemonset-cfqml\" (UID: \"851047fa-74a7-4554-a75d-88ce0fb5587d\") " pod="openshift-must-gather-bzv24/perf-node-gather-daemonset-cfqml"
Sep 07 12:28:40.536896 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:28:40.536892    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/851047fa-74a7-4554-a75d-88ce0fb5587d-podres\") pod \"perf-node-gather-daemonset-cfqml\" (UID: \"851047fa-74a7-4554-a75d-88ce0fb5587d\") " pod="openshift-must-gather-bzv24/perf-node-gather-daemonset-cfqml"
Sep 07 12:28:40.537099 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:28:40.536953    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/851047fa-74a7-4554-a75d-88ce0fb5587d-sys\") pod \"perf-node-gather-daemonset-cfqml\" (UID: \"851047fa-74a7-4554-a75d-88ce0fb5587d\") " pod="openshift-must-gather-bzv24/perf-node-gather-daemonset-cfqml"
Sep 07 12:28:40.537099 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:28:40.536970    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/851047fa-74a7-4554-a75d-88ce0fb5587d-lib-modules\") pod \"perf-node-gather-daemonset-cfqml\" (UID: \"851047fa-74a7-4554-a75d-88ce0fb5587d\") " pod="openshift-must-gather-bzv24/perf-node-gather-daemonset-cfqml"
Sep 07 12:28:40.537099 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:28:40.537078    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-25nqt\" (UniqueName: \"kubernetes.io/projected/851047fa-74a7-4554-a75d-88ce0fb5587d-kube-api-access-25nqt\") pod \"perf-node-gather-daemonset-cfqml\" (UID: \"851047fa-74a7-4554-a75d-88ce0fb5587d\") " pod="openshift-must-gather-bzv24/perf-node-gather-daemonset-cfqml"
Sep 07 12:28:40.637667 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:28:40.637628    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/851047fa-74a7-4554-a75d-88ce0fb5587d-podres\") pod \"perf-node-gather-daemonset-cfqml\" (UID: \"851047fa-74a7-4554-a75d-88ce0fb5587d\") " pod="openshift-must-gather-bzv24/perf-node-gather-daemonset-cfqml"
Sep 07 12:28:40.637803 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:28:40.637682    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/851047fa-74a7-4554-a75d-88ce0fb5587d-sys\") pod \"perf-node-gather-daemonset-cfqml\" (UID: \"851047fa-74a7-4554-a75d-88ce0fb5587d\") " pod="openshift-must-gather-bzv24/perf-node-gather-daemonset-cfqml"
Sep 07 12:28:40.637803 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:28:40.637699    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/851047fa-74a7-4554-a75d-88ce0fb5587d-lib-modules\") pod \"perf-node-gather-daemonset-cfqml\" (UID: \"851047fa-74a7-4554-a75d-88ce0fb5587d\") " pod="openshift-must-gather-bzv24/perf-node-gather-daemonset-cfqml"
Sep 07 12:28:40.637803 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:28:40.637744    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-25nqt\" (UniqueName: \"kubernetes.io/projected/851047fa-74a7-4554-a75d-88ce0fb5587d-kube-api-access-25nqt\") pod \"perf-node-gather-daemonset-cfqml\" (UID: \"851047fa-74a7-4554-a75d-88ce0fb5587d\") " pod="openshift-must-gather-bzv24/perf-node-gather-daemonset-cfqml"
Sep 07 12:28:40.637803 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:28:40.637758    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/851047fa-74a7-4554-a75d-88ce0fb5587d-sys\") pod \"perf-node-gather-daemonset-cfqml\" (UID: \"851047fa-74a7-4554-a75d-88ce0fb5587d\") " pod="openshift-must-gather-bzv24/perf-node-gather-daemonset-cfqml"
Sep 07 12:28:40.638001 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:28:40.637806    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/851047fa-74a7-4554-a75d-88ce0fb5587d-proc\") pod \"perf-node-gather-daemonset-cfqml\" (UID: \"851047fa-74a7-4554-a75d-88ce0fb5587d\") " pod="openshift-must-gather-bzv24/perf-node-gather-daemonset-cfqml"
Sep 07 12:28:40.638001 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:28:40.637817    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/851047fa-74a7-4554-a75d-88ce0fb5587d-podres\") pod \"perf-node-gather-daemonset-cfqml\" (UID: \"851047fa-74a7-4554-a75d-88ce0fb5587d\") " pod="openshift-must-gather-bzv24/perf-node-gather-daemonset-cfqml"
Sep 07 12:28:40.638001 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:28:40.637899    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/851047fa-74a7-4554-a75d-88ce0fb5587d-proc\") pod \"perf-node-gather-daemonset-cfqml\" (UID: \"851047fa-74a7-4554-a75d-88ce0fb5587d\") " pod="openshift-must-gather-bzv24/perf-node-gather-daemonset-cfqml"
Sep 07 12:28:40.638001 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:28:40.637917    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/851047fa-74a7-4554-a75d-88ce0fb5587d-lib-modules\") pod \"perf-node-gather-daemonset-cfqml\" (UID: \"851047fa-74a7-4554-a75d-88ce0fb5587d\") " pod="openshift-must-gather-bzv24/perf-node-gather-daemonset-cfqml"
Sep 07 12:28:40.644729 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:28:40.644709    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-25nqt\" (UniqueName: \"kubernetes.io/projected/851047fa-74a7-4554-a75d-88ce0fb5587d-kube-api-access-25nqt\") pod \"perf-node-gather-daemonset-cfqml\" (UID: \"851047fa-74a7-4554-a75d-88ce0fb5587d\") " pod="openshift-must-gather-bzv24/perf-node-gather-daemonset-cfqml"
Sep 07 12:28:40.748685 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:28:40.748657    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-bzv24/perf-node-gather-daemonset-cfqml"
Sep 07 12:28:41.021437 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:28:41.021394    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-bzv24/perf-node-gather-daemonset-cfqml"]
Sep 07 12:28:41.024190 ip-10-0-135-173 kubenswrapper[2563]: W0907 12:28:41.024158    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod851047fa_74a7_4554_a75d_88ce0fb5587d.slice/crio-6bdfa8dcc71e8cfa007d0b7dea841ddf0d3f5c75aa83ad3aad1c8529e0685eeb WatchSource:0}: Error finding container 6bdfa8dcc71e8cfa007d0b7dea841ddf0d3f5c75aa83ad3aad1c8529e0685eeb: Status 404 returned error can't find the container with id 6bdfa8dcc71e8cfa007d0b7dea841ddf0d3f5c75aa83ad3aad1c8529e0685eeb
Sep 07 12:28:41.245457 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:28:41.245409    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-695c895bd6-vhbb6_3b027785-afe9-41e7-b5b9-e9eb56f4d276/volume-data-source-validator/0.log"
Sep 07 12:28:41.880800 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:28:41.880761    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-bzv24/perf-node-gather-daemonset-cfqml" event={"ID":"851047fa-74a7-4554-a75d-88ce0fb5587d","Type":"ContainerStarted","Data":"d42c759cc5ca1a4008db5f1e8884480249a9ab2801a1a98c0746fb2c6dc9741a"}
Sep 07 12:28:41.880968 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:28:41.880814    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-bzv24/perf-node-gather-daemonset-cfqml" event={"ID":"851047fa-74a7-4554-a75d-88ce0fb5587d","Type":"ContainerStarted","Data":"6bdfa8dcc71e8cfa007d0b7dea841ddf0d3f5c75aa83ad3aad1c8529e0685eeb"}
Sep 07 12:28:41.880968 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:28:41.880841    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-bzv24/perf-node-gather-daemonset-cfqml"
Sep 07 12:28:41.895519 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:28:41.895473    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-bzv24/perf-node-gather-daemonset-cfqml" podStartSLOduration=1.895457516 podStartE2EDuration="1.895457516s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-07 12:28:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 12:28:41.894553132 +0000 UTC m=+2493.413284945" watchObservedRunningTime="2026-09-07 12:28:41.895457516 +0000 UTC m=+2493.414189329"
Sep 07 12:28:42.515590 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:28:42.515559    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-q9tkc_46cab3bd-0f24-46e2-910c-b6e9a5f6ae5e/dns/0.log"
Sep 07 12:28:42.534321 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:28:42.534295    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-q9tkc_46cab3bd-0f24-46e2-910c-b6e9a5f6ae5e/kube-rbac-proxy/0.log"
Sep 07 12:28:42.554004 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:28:42.553981    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-hxstl_5f723277-6450-4fd8-8f5d-01d9f9f6814b/dns-node-resolver/0.log"
Sep 07 12:28:43.166620 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:28:43.166590    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-74ccddf898-jp5pm_a5bfeb4e-09b1-4bdb-8692-e1e6d8b5bd73/registry/0.log"
Sep 07 12:28:43.235861 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:28:43.235833    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-wrd4z_b9aae943-4d29-4c1e-909f-0a1c034299f8/node-ca/0.log"
Sep 07 12:28:44.772856 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:28:44.772827    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5b6777d477-c7pps_4a58840b-efff-43a4-ac2f-906a5f1b4bdc/kube-auth-proxy/0.log"
Sep 07 12:28:45.860556 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:28:45.860526    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-nv8dv_d1606d20-6381-462c-9cbe-b65edc598e2f/serve-healthcheck-canary/0.log"
Sep 07 12:28:46.417248 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:28:46.417209    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-76bcd6ccf5-zls5m_b6023824-522a-4bed-b277-7394abe09bbc/insights-operator/0.log"
Sep 07 12:28:46.417891 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:28:46.417864    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-76bcd6ccf5-zls5m_b6023824-522a-4bed-b277-7394abe09bbc/insights-operator/1.log"
Sep 07 12:28:46.443750 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:28:46.443725    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-mchxp_f302b236-f400-4839-8d56-edaf5de4bb75/kube-rbac-proxy/0.log"
Sep 07 12:28:46.464820 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:28:46.464799    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-mchxp_f302b236-f400-4839-8d56-edaf5de4bb75/exporter/0.log"
Sep 07 12:28:46.485447 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:28:46.485426    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-mchxp_f302b236-f400-4839-8d56-edaf5de4bb75/extractor/0.log"
Sep 07 12:28:47.896101 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:28:47.896070    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-bzv24/perf-node-gather-daemonset-cfqml"
Sep 07 12:28:49.321217 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:28:49.321190    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-648db6df7b-c82m6_9fe85f81-1199-4c0b-b58e-036142982b1a/manager/0.log"
Sep 07 12:28:49.339472 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:28:49.339453    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-648db6df7b-c82m6_9fe85f81-1199-4c0b-b58e-036142982b1a/copy-manifests/0.log"
Sep 07 12:28:50.871834 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:28:50.871801    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-59499dc5ff-46274_ee12ae27-62c1-43cb-98ec-319b3243ff28/manager/0.log"
Sep 07 12:28:50.915503 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:28:50.915476    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-tf2ht_117fafc0-57de-4666-845f-f12646393515/openshift-lws-operator/0.log"
Sep 07 12:28:51.458956 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:28:51.458927    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5b6777d477-c7pps_4a58840b-efff-43a4-ac2f-906a5f1b4bdc/kube-auth-proxy/0.log"
Sep 07 12:28:52.459033 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:28:52.458999    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dl69z_3e71d2e5-bc97-415d-8e01-cdf554957349/konnectivity-agent/0.log" limit=1024
Sep 07 12:28:53.692039 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:28:53.692005    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5b6777d477-c7pps_4a58840b-efff-43a4-ac2f-906a5f1b4bdc/kube-auth-proxy/0.log"
Sep 07 12:28:57.608346 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:28:57.608314    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-mv7nw_55db3d09-7464-4f97-af5a-985162756ebb/kube-multus-additional-cni-plugins/0.log"
Sep 07 12:28:57.626243 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:28:57.626218    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-mv7nw_55db3d09-7464-4f97-af5a-985162756ebb/egress-router-binary-copy/0.log"
Sep 07 12:28:57.646391 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:28:57.646363    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-mv7nw_55db3d09-7464-4f97-af5a-985162756ebb/cni-plugins/0.log"
Sep 07 12:28:57.663873 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:28:57.663848    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-mv7nw_55db3d09-7464-4f97-af5a-985162756ebb/bond-cni-plugin/0.log"
Sep 07 12:28:57.681263 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:28:57.681236    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-mv7nw_55db3d09-7464-4f97-af5a-985162756ebb/routeoverride-cni/0.log"
Sep 07 12:28:57.699006 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:28:57.698987    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-mv7nw_55db3d09-7464-4f97-af5a-985162756ebb/whereabouts-cni-bincopy/0.log"
Sep 07 12:28:57.717532 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:28:57.717514    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-mv7nw_55db3d09-7464-4f97-af5a-985162756ebb/whereabouts-cni/0.log"
Sep 07 12:28:57.915167 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:28:57.915096    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-g2v7c_64362f4d-7d1d-4faa-a02e-4a7dec54cffc/kube-multus/0.log"
Sep 07 12:28:58.131894 ip-10-0-135-173 kubenswrapper[2563]: I0907 12:28:58.131864    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-xf2gj_3acaae8d-5930-49c6-9b9e-d286ac6a75d5/network-metrics-daemon/0.log"