Sep 18 19:59:03.104902 ip-10-0-130-81 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Sep 18 19:59:03.104914 ip-10-0-130-81 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Sep 18 19:59:03.104921 ip-10-0-130-81 systemd[1]: kubelet.service: Failed with result 'resources'.
Sep 18 19:59:03.105138 ip-10-0-130-81 systemd[1]: Failed to start Kubernetes Kubelet.
Sep 18 19:59:13.167922 ip-10-0-130-81 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Sep 18 19:59:13.167941 ip-10-0-130-81 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot 970e819bee1a488bb658030231851c81 --
Sep 18 20:01:06.806776 ip-10-0-130-81 systemd[1]: Starting Kubernetes Kubelet...
Sep 18 20:01:10.143717 ip-10-0-130-81 kubenswrapper[2532]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 18 20:01:10.143717 ip-10-0-130-81 kubenswrapper[2532]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Sep 18 20:01:10.143717 ip-10-0-130-81 kubenswrapper[2532]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 18 20:01:10.143717 ip-10-0-130-81 kubenswrapper[2532]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Sep 18 20:01:10.143717 ip-10-0-130-81 kubenswrapper[2532]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 18 20:01:10.144563 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:10.143779    2532 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Sep 18 20:01:10.147525 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.147510    2532 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 18 20:01:10.147525 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.147524    2532 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 18 20:01:10.147587 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.147528    2532 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 18 20:01:10.147587 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.147532    2532 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 18 20:01:10.147587 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.147535    2532 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 18 20:01:10.147587 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.147538    2532 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 18 20:01:10.147587 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.147541    2532 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 18 20:01:10.147587 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.147543    2532 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 18 20:01:10.147587 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.147546    2532 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 18 20:01:10.147587 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.147559    2532 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 18 20:01:10.147587 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.147562    2532 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 18 20:01:10.147587 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.147564    2532 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 18 20:01:10.147587 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.147568    2532 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 18 20:01:10.147587 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.147572    2532 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 18 20:01:10.147587 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.147574    2532 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 18 20:01:10.147587 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.147577    2532 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 18 20:01:10.147587 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.147579    2532 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 18 20:01:10.147587 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.147582    2532 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 18 20:01:10.147587 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.147585    2532 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 18 20:01:10.147587 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.147590    2532 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 18 20:01:10.147587 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.147595    2532 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 18 20:01:10.148048 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.147599    2532 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 18 20:01:10.148048 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.147603    2532 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 18 20:01:10.148048 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.147606    2532 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 18 20:01:10.148048 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.147609    2532 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 18 20:01:10.148048 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.147612    2532 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 18 20:01:10.148048 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.147615    2532 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 18 20:01:10.148048 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.147618    2532 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 18 20:01:10.148048 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.147620    2532 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 18 20:01:10.148048 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.147623    2532 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 18 20:01:10.148048 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.147625    2532 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 18 20:01:10.148048 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.147628    2532 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 18 20:01:10.148048 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.147630    2532 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 18 20:01:10.148048 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.147633    2532 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 18 20:01:10.148048 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.147635    2532 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 18 20:01:10.148048 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.147638    2532 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 18 20:01:10.148048 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.147640    2532 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 18 20:01:10.148048 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.147643    2532 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 18 20:01:10.148048 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.147646    2532 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 18 20:01:10.148048 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.147648    2532 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 18 20:01:10.148048 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.147651    2532 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 18 20:01:10.148596 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.147654    2532 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 18 20:01:10.148596 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.147656    2532 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 18 20:01:10.148596 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.147659    2532 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 18 20:01:10.148596 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.147662    2532 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 18 20:01:10.148596 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.147664    2532 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 18 20:01:10.148596 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.147666    2532 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 18 20:01:10.148596 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.147669    2532 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 18 20:01:10.148596 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.147671    2532 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 18 20:01:10.148596 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.147674    2532 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 18 20:01:10.148596 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.147676    2532 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 18 20:01:10.148596 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.147679    2532 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 18 20:01:10.148596 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.147681    2532 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 18 20:01:10.148596 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.147685    2532 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 18 20:01:10.148596 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.147689    2532 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 18 20:01:10.148596 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.147692    2532 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 18 20:01:10.148596 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.147694    2532 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 18 20:01:10.148596 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.147697    2532 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 18 20:01:10.148596 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.147699    2532 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 18 20:01:10.148596 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.147702    2532 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 18 20:01:10.148596 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.147705    2532 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 18 20:01:10.149090 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.147707    2532 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 18 20:01:10.149090 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.147710    2532 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 18 20:01:10.149090 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.147712    2532 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 18 20:01:10.149090 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.147715    2532 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 18 20:01:10.149090 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.147717    2532 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 18 20:01:10.149090 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.147720    2532 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 18 20:01:10.149090 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.147723    2532 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 18 20:01:10.149090 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.147725    2532 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 18 20:01:10.149090 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.147728    2532 feature_gate.go:328] unrecognized feature gate: Example
Sep 18 20:01:10.149090 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.147730    2532 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 18 20:01:10.149090 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.147733    2532 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 18 20:01:10.149090 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.147735    2532 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 18 20:01:10.149090 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.147738    2532 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 18 20:01:10.149090 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.147741    2532 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 18 20:01:10.149090 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.147743    2532 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 18 20:01:10.149090 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.147746    2532 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 18 20:01:10.149090 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.147749    2532 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 18 20:01:10.149090 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.147752    2532 feature_gate.go:328] unrecognized feature gate: Example2
Sep 18 20:01:10.149090 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.147754    2532 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 18 20:01:10.149090 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.147757    2532 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 18 20:01:10.149605 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.147759    2532 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 18 20:01:10.149605 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.147762    2532 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 18 20:01:10.149605 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.147764    2532 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 18 20:01:10.149605 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.147767    2532 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 18 20:01:10.149605 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.147769    2532 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 18 20:01:10.149605 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.147773    2532 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 18 20:01:10.149605 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.147775    2532 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 18 20:01:10.149605 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.147779    2532 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 18 20:01:10.149605 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.147781    2532 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 18 20:01:10.149605 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.147784    2532 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 18 20:01:10.149605 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.147787    2532 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 18 20:01:10.149605 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.147790    2532 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 18 20:01:10.149605 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.147792    2532 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 18 20:01:10.149605 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.147795    2532 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 18 20:01:10.149605 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.147797    2532 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 18 20:01:10.149605 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.147800    2532 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 18 20:01:10.149605 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.147802    2532 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 18 20:01:10.149605 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.147806    2532 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 18 20:01:10.149605 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.148472    2532 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 18 20:01:10.150056 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.148478    2532 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 18 20:01:10.150056 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.148481    2532 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 18 20:01:10.150056 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.148500    2532 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 18 20:01:10.150056 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.148504    2532 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 18 20:01:10.150056 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.148507    2532 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 18 20:01:10.150056 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.148510    2532 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 18 20:01:10.150056 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.148513    2532 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 18 20:01:10.150056 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.148516    2532 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 18 20:01:10.150056 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.148519    2532 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 18 20:01:10.150056 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.148522    2532 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 18 20:01:10.150056 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.148525    2532 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 18 20:01:10.150056 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.148528    2532 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 18 20:01:10.150056 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.148530    2532 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 18 20:01:10.150056 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.148533    2532 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 18 20:01:10.150056 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.148536    2532 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 18 20:01:10.150056 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.148538    2532 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 18 20:01:10.150056 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.148541    2532 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 18 20:01:10.150056 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.148544    2532 feature_gate.go:328] unrecognized feature gate: Example2
Sep 18 20:01:10.150056 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.148547    2532 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 18 20:01:10.150056 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.148550    2532 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 18 20:01:10.150618 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.148552    2532 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 18 20:01:10.150618 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.148555    2532 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 18 20:01:10.150618 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.148559    2532 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 18 20:01:10.150618 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.148563    2532 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 18 20:01:10.150618 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.148566    2532 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 18 20:01:10.150618 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.148569    2532 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 18 20:01:10.150618 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.148572    2532 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 18 20:01:10.150618 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.148575    2532 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 18 20:01:10.150618 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.148578    2532 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 18 20:01:10.150618 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.148581    2532 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 18 20:01:10.150618 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.148584    2532 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 18 20:01:10.150618 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.148587    2532 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 18 20:01:10.150618 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.148589    2532 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 18 20:01:10.150618 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.148592    2532 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 18 20:01:10.150618 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.148595    2532 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 18 20:01:10.150618 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.148599    2532 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 18 20:01:10.150618 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.148602    2532 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 18 20:01:10.150618 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.148605    2532 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 18 20:01:10.150618 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.148607    2532 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 18 20:01:10.150618 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.148610    2532 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 18 20:01:10.151146 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.148614    2532 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 18 20:01:10.151146 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.148616    2532 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 18 20:01:10.151146 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.148619    2532 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 18 20:01:10.151146 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.148622    2532 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 18 20:01:10.151146 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.148624    2532 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 18 20:01:10.151146 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.148627    2532 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 18 20:01:10.151146 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.148631    2532 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 18 20:01:10.151146 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.148634    2532 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 18 20:01:10.151146 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.148636    2532 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 18 20:01:10.151146 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.148640    2532 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 18 20:01:10.151146 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.148643    2532 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 18 20:01:10.151146 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.148646    2532 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 18 20:01:10.151146 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.148648    2532 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 18 20:01:10.151146 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.148651    2532 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 18 20:01:10.151146 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.148653    2532 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 18 20:01:10.151146 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.148656    2532 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 18 20:01:10.151146 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.148658    2532 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 18 20:01:10.151146 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.148661    2532 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 18 20:01:10.151146 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.148663    2532 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 18 20:01:10.151630 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.148666    2532 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 18 20:01:10.151630 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.148669    2532 feature_gate.go:328] unrecognized feature gate: Example
Sep 18 20:01:10.151630 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.148672    2532 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 18 20:01:10.151630 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.148675    2532 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 18 20:01:10.151630 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.148678    2532 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 18 20:01:10.151630 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.148681    2532 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 18 20:01:10.151630 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.148684    2532 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 18 20:01:10.151630 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.148687    2532 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 18 20:01:10.151630 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.148690    2532 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 18 20:01:10.151630 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.148692    2532 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 18 20:01:10.151630 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.148695    2532 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 18 20:01:10.151630 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.148698    2532 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 18 20:01:10.151630 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.148700    2532 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 18 20:01:10.151630 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.148704    2532 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 18 20:01:10.151630 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.148707    2532 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 18 20:01:10.151630 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.148710    2532 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 18 20:01:10.151630 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.148713    2532 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 18 20:01:10.151630 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.148715    2532 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 18 20:01:10.151630 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.148718    2532 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 18 20:01:10.152090 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.148720    2532 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 18 20:01:10.152090 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.148723    2532 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 18 20:01:10.152090 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.148725    2532 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 18 20:01:10.152090 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.148728    2532 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 18 20:01:10.152090 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.148730    2532 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 18 20:01:10.152090 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.148733    2532 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 18 20:01:10.152090 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.148735    2532 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 18 20:01:10.152090 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.148738    2532 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 18 20:01:10.152090 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.148740    2532 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 18 20:01:10.152090 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.148743    2532 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 18 20:01:10.152090 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.148745    2532 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 18 20:01:10.152090 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.148749    2532 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 18 20:01:10.152090 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.148752    2532 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 18 20:01:10.152090 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.148755    2532 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 18 20:01:10.152090 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.148758    2532 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 18 20:01:10.152090 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.148760    2532 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 18 20:01:10.152090 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.148763    2532 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 18 20:01:10.152090 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.148765    2532 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 18 20:01:10.152090 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.148768    2532 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 18 20:01:10.152090 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.148771    2532 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 18 20:01:10.152619 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.149756    2532 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 18 20:01:10.152619 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.149763    2532 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 18 20:01:10.152619 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.149766    2532 feature_gate.go:328] unrecognized feature gate: Example2
Sep 18 20:01:10.152619 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.149769    2532 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 18 20:01:10.152619 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.149772    2532 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 18 20:01:10.152619 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.149775    2532 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 18 20:01:10.152619 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.149779    2532 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 18 20:01:10.152619 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.149782    2532 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 18 20:01:10.152619 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.149786    2532 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 18 20:01:10.152619 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.149788    2532 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 18 20:01:10.152619 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.149791    2532 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 18 20:01:10.152619 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.149794    2532 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 18 20:01:10.152619 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.149797    2532 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 18 20:01:10.152619 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.149800    2532 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 18 20:01:10.152619 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.149803    2532 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 18 20:01:10.152619 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.149805    2532 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 18 20:01:10.152619 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.149808    2532 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 18 20:01:10.152619 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.149811    2532 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 18 20:01:10.152619 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.149814    2532 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 18 20:01:10.152619 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.149816    2532 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 18 20:01:10.153123 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.149819    2532 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 18 20:01:10.153123 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.149822    2532 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 18 20:01:10.153123 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.149825    2532 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 18 20:01:10.153123 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.149827    2532 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 18 20:01:10.153123 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.149831    2532 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 18 20:01:10.153123 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.149833    2532 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 18 20:01:10.153123 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.149836    2532 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 18 20:01:10.153123 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.149840    2532 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 18 20:01:10.153123 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.149844    2532 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 18 20:01:10.153123 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.149846    2532 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 18 20:01:10.153123 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.149849    2532 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 18 20:01:10.153123 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.149851    2532 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 18 20:01:10.153123 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.149854    2532 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 18 20:01:10.153123 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.149856    2532 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 18 20:01:10.153123 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.149859    2532 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 18 20:01:10.153123 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.149862    2532 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 18 20:01:10.153123 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.149865    2532 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 18 20:01:10.153123 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.149868    2532 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 18 20:01:10.153123 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.149870    2532 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 18 20:01:10.153613 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.149873    2532 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 18 20:01:10.153613 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.149876    2532 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 18 20:01:10.153613 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.149878    2532 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 18 20:01:10.153613 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.149881    2532 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 18 20:01:10.153613 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.149883    2532 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 18 20:01:10.153613 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.149885    2532 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 18 20:01:10.153613 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.149888    2532 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 18 20:01:10.153613 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.149890    2532 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 18 20:01:10.153613 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.149893    2532 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 18 20:01:10.153613 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.149895    2532 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 18 20:01:10.153613 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.149898    2532 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 18 20:01:10.153613 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.149900    2532 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 18 20:01:10.153613 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.149903    2532 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 18 20:01:10.153613 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.149906    2532 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 18 20:01:10.153613 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.149908    2532 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 18 20:01:10.153613 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.149911    2532 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 18 20:01:10.153613 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.149913    2532 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 18 20:01:10.153613 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.149916    2532 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 18 20:01:10.153613 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.149918    2532 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 18 20:01:10.153613 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.149921    2532 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 18 20:01:10.154106 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.149923    2532 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 18 20:01:10.154106 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.149926    2532 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 18 20:01:10.154106 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.149928    2532 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 18 20:01:10.154106 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.149931    2532 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 18 20:01:10.154106 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.149933    2532 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 18 20:01:10.154106 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.149936    2532 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 18 20:01:10.154106 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.149938    2532 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 18 20:01:10.154106 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.149941    2532 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 18 20:01:10.154106 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.149944    2532 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 18 20:01:10.154106 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.149947    2532 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 18 20:01:10.154106 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.149949    2532 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 18 20:01:10.154106 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.149952    2532 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 18 20:01:10.154106 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.149954    2532 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 18 20:01:10.154106 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.149957    2532 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 18 20:01:10.154106 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.149962    2532 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 18 20:01:10.154106 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.149965    2532 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 18 20:01:10.154106 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.149967    2532 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 18 20:01:10.154106 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.149969    2532 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 18 20:01:10.154106 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.149972    2532 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 18 20:01:10.154106 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.149974    2532 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 18 20:01:10.154603 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.149979    2532 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 18 20:01:10.154603 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.149982    2532 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 18 20:01:10.154603 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.149985    2532 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 18 20:01:10.154603 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.149988    2532 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 18 20:01:10.154603 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.149992    2532 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 18 20:01:10.154603 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.149994    2532 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 18 20:01:10.154603 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.149997    2532 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 18 20:01:10.154603 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.150000    2532 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 18 20:01:10.154603 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.150003    2532 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 18 20:01:10.154603 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.150007    2532 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 18 20:01:10.154603 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.150010    2532 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 18 20:01:10.154603 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.150013    2532 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 18 20:01:10.154603 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.150016    2532 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 18 20:01:10.154603 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.150019    2532 feature_gate.go:328] unrecognized feature gate: Example
Sep 18 20:01:10.154603 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.150023    2532 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 18 20:01:10.154603 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.150026    2532 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 18 20:01:10.154603 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.150029    2532 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 18 20:01:10.154603 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.150032    2532 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 18 20:01:10.154603 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.150035    2532 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 18 20:01:10.155067 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.150038    2532 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 18 20:01:10.155067 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:10.150105    2532 flags.go:64] FLAG: --address="0.0.0.0"
Sep 18 20:01:10.155067 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:10.150138    2532 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Sep 18 20:01:10.155067 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:10.150144    2532 flags.go:64] FLAG: --anonymous-auth="true"
Sep 18 20:01:10.155067 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:10.150149    2532 flags.go:64] FLAG: --application-metrics-count-limit="100"
Sep 18 20:01:10.155067 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:10.150153    2532 flags.go:64] FLAG: --authentication-token-webhook="false"
Sep 18 20:01:10.155067 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:10.150157    2532 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Sep 18 20:01:10.155067 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:10.150161    2532 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Sep 18 20:01:10.155067 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:10.150166    2532 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Sep 18 20:01:10.155067 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:10.150169    2532 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Sep 18 20:01:10.155067 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:10.150172    2532 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Sep 18 20:01:10.155067 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:10.150176    2532 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Sep 18 20:01:10.155067 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:10.150179    2532 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Sep 18 20:01:10.155067 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:10.150182    2532 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Sep 18 20:01:10.155067 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:10.150185    2532 flags.go:64] FLAG: --cgroup-root=""
Sep 18 20:01:10.155067 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:10.150188    2532 flags.go:64] FLAG: --cgroups-per-qos="true"
Sep 18 20:01:10.155067 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:10.150192    2532 flags.go:64] FLAG: --client-ca-file=""
Sep 18 20:01:10.155067 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:10.150197    2532 flags.go:64] FLAG: --cloud-provider="external"
Sep 18 20:01:10.155067 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:10.150200    2532 flags.go:64] FLAG: --cluster-dns="[]"
Sep 18 20:01:10.155067 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:10.150205    2532 flags.go:64] FLAG: --cluster-domain=""
Sep 18 20:01:10.155067 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:10.150208    2532 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Sep 18 20:01:10.155067 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:10.150211    2532 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Sep 18 20:01:10.155067 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:10.150214    2532 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Sep 18 20:01:10.155067 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:10.150217    2532 flags.go:64] FLAG: --container-log-max-files="5"
Sep 18 20:01:10.155661 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:10.150221    2532 flags.go:64] FLAG: --container-log-max-size="10Mi"
Sep 18 20:01:10.155661 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:10.150224    2532 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Sep 18 20:01:10.155661 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:10.150227    2532 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Sep 18 20:01:10.155661 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:10.150230    2532 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Sep 18 20:01:10.155661 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:10.150233    2532 flags.go:64] FLAG: --contention-profiling="false"
Sep 18 20:01:10.155661 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:10.150236    2532 flags.go:64] FLAG: --cpu-cfs-quota="true"
Sep 18 20:01:10.155661 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:10.150239    2532 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Sep 18 20:01:10.155661 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:10.150244    2532 flags.go:64] FLAG: --cpu-manager-policy="none"
Sep 18 20:01:10.155661 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:10.150247    2532 flags.go:64] FLAG: --cpu-manager-policy-options=""
Sep 18 20:01:10.155661 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:10.150251    2532 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Sep 18 20:01:10.155661 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:10.150254    2532 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Sep 18 20:01:10.155661 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:10.150257    2532 flags.go:64] FLAG: --enable-debugging-handlers="true"
Sep 18 20:01:10.155661 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:10.150260    2532 flags.go:64] FLAG: --enable-load-reader="false"
Sep 18 20:01:10.155661 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:10.150263    2532 flags.go:64] FLAG: --enable-server="true"
Sep 18 20:01:10.155661 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:10.150266    2532 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Sep 18 20:01:10.155661 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:10.150270    2532 flags.go:64] FLAG: --event-burst="100"
Sep 18 20:01:10.155661 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:10.150274    2532 flags.go:64] FLAG: --event-qps="50"
Sep 18 20:01:10.155661 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:10.150277    2532 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Sep 18 20:01:10.155661 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:10.150280    2532 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Sep 18 20:01:10.155661 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:10.150283    2532 flags.go:64] FLAG: --eviction-hard=""
Sep 18 20:01:10.155661 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:10.150287    2532 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Sep 18 20:01:10.155661 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:10.150290    2532 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Sep 18 20:01:10.155661 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:10.150293    2532 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Sep 18 20:01:10.155661 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:10.150296    2532 flags.go:64] FLAG: --eviction-soft=""
Sep 18 20:01:10.155661 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:10.150299    2532 flags.go:64] FLAG: --eviction-soft-grace-period=""
Sep 18 20:01:10.156276 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:10.150303    2532 flags.go:64] FLAG: --exit-on-lock-contention="false"
Sep 18 20:01:10.156276 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:10.150306    2532 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Sep 18 20:01:10.156276 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:10.150309    2532 flags.go:64] FLAG: --experimental-mounter-path=""
Sep 18 20:01:10.156276 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:10.150312    2532 flags.go:64] FLAG: --fail-cgroupv1="false"
Sep 18 20:01:10.156276 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:10.150315    2532 flags.go:64] FLAG: --fail-swap-on="true"
Sep 18 20:01:10.156276 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:10.150318    2532 flags.go:64] FLAG: --feature-gates=""
Sep 18 20:01:10.156276 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:10.150322    2532 flags.go:64] FLAG: --file-check-frequency="20s"
Sep 18 20:01:10.156276 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:10.150325    2532 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Sep 18 20:01:10.156276 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:10.150328    2532 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Sep 18 20:01:10.156276 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:10.150331    2532 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Sep 18 20:01:10.156276 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:10.150334    2532 flags.go:64] FLAG: --healthz-port="10248"
Sep 18 20:01:10.156276 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:10.150337    2532 flags.go:64] FLAG: --help="false"
Sep 18 20:01:10.156276 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:10.150341    2532 flags.go:64] FLAG: --hostname-override="ip-10-0-130-81.ec2.internal"
Sep 18 20:01:10.156276 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:10.150344    2532 flags.go:64] FLAG: --housekeeping-interval="10s"
Sep 18 20:01:10.156276 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:10.150348    2532 flags.go:64] FLAG: --http-check-frequency="20s"
Sep 18 20:01:10.156276 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:10.150351    2532 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Sep 18 20:01:10.156276 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:10.150354    2532 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Sep 18 20:01:10.156276 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:10.150357    2532 flags.go:64] FLAG: --image-gc-high-threshold="85"
Sep 18 20:01:10.156276 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:10.150360    2532 flags.go:64] FLAG: --image-gc-low-threshold="80"
Sep 18 20:01:10.156276 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:10.150363    2532 flags.go:64] FLAG: --image-service-endpoint=""
Sep 18 20:01:10.156276 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:10.150366    2532 flags.go:64] FLAG: --kernel-memcg-notification="false"
Sep 18 20:01:10.156276 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:10.150369    2532 flags.go:64] FLAG: --kube-api-burst="100"
Sep 18 20:01:10.156276 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:10.150372    2532 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Sep 18 20:01:10.156942 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:10.150376    2532 flags.go:64] FLAG: --kube-api-qps="50"
Sep 18 20:01:10.156942 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:10.150379    2532 flags.go:64] FLAG: --kube-reserved=""
Sep 18 20:01:10.156942 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:10.150382    2532 flags.go:64] FLAG: --kube-reserved-cgroup=""
Sep 18 20:01:10.156942 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:10.150385    2532 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Sep 18 20:01:10.156942 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:10.150389    2532 flags.go:64] FLAG: --kubelet-cgroups=""
Sep 18 20:01:10.156942 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:10.150391    2532 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Sep 18 20:01:10.156942 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:10.150394    2532 flags.go:64] FLAG: --lock-file=""
Sep 18 20:01:10.156942 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:10.150397    2532 flags.go:64] FLAG: --log-cadvisor-usage="false"
Sep 18 20:01:10.156942 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:10.150400    2532 flags.go:64] FLAG: --log-flush-frequency="5s"
Sep 18 20:01:10.156942 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:10.150405    2532 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Sep 18 20:01:10.156942 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:10.150410    2532 flags.go:64] FLAG: --log-json-split-stream="false"
Sep 18 20:01:10.156942 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:10.150413    2532 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Sep 18 20:01:10.156942 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:10.150416    2532 flags.go:64] FLAG: --log-text-split-stream="false"
Sep 18 20:01:10.156942 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:10.150418    2532 flags.go:64] FLAG: --logging-format="text"
Sep 18 20:01:10.156942 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:10.150421    2532 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Sep 18 20:01:10.156942 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:10.150425    2532 flags.go:64] FLAG: --make-iptables-util-chains="true"
Sep 18 20:01:10.156942 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:10.150427    2532 flags.go:64] FLAG: --manifest-url=""
Sep 18 20:01:10.156942 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:10.150430    2532 flags.go:64] FLAG: --manifest-url-header=""
Sep 18 20:01:10.156942 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:10.150434    2532 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Sep 18 20:01:10.156942 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:10.150437    2532 flags.go:64] FLAG: --max-open-files="1000000"
Sep 18 20:01:10.156942 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:10.150444    2532 flags.go:64] FLAG: --max-pods="110"
Sep 18 20:01:10.156942 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:10.150447    2532 flags.go:64] FLAG: --maximum-dead-containers="-1"
Sep 18 20:01:10.156942 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:10.150450    2532 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Sep 18 20:01:10.156942 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:10.150455    2532 flags.go:64] FLAG: --memory-manager-policy="None"
Sep 18 20:01:10.156942 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:10.150458    2532 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Sep 18 20:01:10.157586 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:10.150461    2532 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Sep 18 20:01:10.157586 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:10.150464    2532 flags.go:64] FLAG: --node-ip="0.0.0.0"
Sep 18 20:01:10.157586 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:10.150467    2532 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Sep 18 20:01:10.157586 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:10.150474    2532 flags.go:64] FLAG: --node-status-max-images="50"
Sep 18 20:01:10.157586 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:10.150477    2532 flags.go:64] FLAG: --node-status-update-frequency="10s"
Sep 18 20:01:10.157586 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:10.150481    2532 flags.go:64] FLAG: --oom-score-adj="-999"
Sep 18 20:01:10.157586 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:10.150498    2532 flags.go:64] FLAG: --pod-cidr=""
Sep 18 20:01:10.157586 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:10.150501    2532 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:a6abe77af61c5f2e3985a83e7d43538aefc38b1b0444127da14ee0e8cf8f76fc"
Sep 18 20:01:10.157586 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:10.150507    2532 flags.go:64] FLAG: --pod-manifest-path=""
Sep 18 20:01:10.157586 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:10.150510    2532 flags.go:64] FLAG: --pod-max-pids="-1"
Sep 18 20:01:10.157586 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:10.150513    2532 flags.go:64] FLAG: --pods-per-core="0"
Sep 18 20:01:10.157586 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:10.150516    2532 flags.go:64] FLAG: --port="10250"
Sep 18 20:01:10.157586 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:10.150519    2532 flags.go:64] FLAG: --protect-kernel-defaults="false"
Sep 18 20:01:10.157586 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:10.150522    2532 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-092ba412d8cd39ff8"
Sep 18 20:01:10.157586 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:10.150525    2532 flags.go:64] FLAG: --qos-reserved=""
Sep 18 20:01:10.157586 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:10.150528    2532 flags.go:64] FLAG: --read-only-port="10255"
Sep 18 20:01:10.157586 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:10.150533    2532 flags.go:64] FLAG: --register-node="true"
Sep 18 20:01:10.157586 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:10.150536    2532 flags.go:64] FLAG: --register-with-taints=""
Sep 18 20:01:10.157586 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:10.150540    2532 flags.go:64] FLAG: --registry-burst="10"
Sep 18 20:01:10.157586 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:10.150543    2532 flags.go:64] FLAG: --registry-qps="5"
Sep 18 20:01:10.157586 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:10.150546    2532 flags.go:64] FLAG: --reserved-cpus=""
Sep 18 20:01:10.157586 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:10.150549    2532 flags.go:64] FLAG: --reserved-memory=""
Sep 18 20:01:10.157586 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:10.150552    2532 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Sep 18 20:01:10.157586 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:10.150555    2532 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Sep 18 20:01:10.157586 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:10.150558    2532 flags.go:64] FLAG: --rotate-certificates="false"
Sep 18 20:01:10.158195 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:10.150561    2532 flags.go:64] FLAG: --rotate-server-certificates="false"
Sep 18 20:01:10.158195 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:10.150564    2532 flags.go:64] FLAG: --runonce="false"
Sep 18 20:01:10.158195 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:10.150567    2532 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Sep 18 20:01:10.158195 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:10.150570    2532 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Sep 18 20:01:10.158195 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:10.150573    2532 flags.go:64] FLAG: --seccomp-default="false"
Sep 18 20:01:10.158195 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:10.150576    2532 flags.go:64] FLAG: --serialize-image-pulls="true"
Sep 18 20:01:10.158195 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:10.150580    2532 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Sep 18 20:01:10.158195 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:10.150583    2532 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Sep 18 20:01:10.158195 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:10.150586    2532 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Sep 18 20:01:10.158195 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:10.150589    2532 flags.go:64] FLAG: --storage-driver-password="root"
Sep 18 20:01:10.158195 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:10.150592    2532 flags.go:64] FLAG: --storage-driver-secure="false"
Sep 18 20:01:10.158195 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:10.150595    2532 flags.go:64] FLAG: --storage-driver-table="stats"
Sep 18 20:01:10.158195 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:10.150598    2532 flags.go:64] FLAG: --storage-driver-user="root"
Sep 18 20:01:10.158195 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:10.150600    2532 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Sep 18 20:01:10.158195 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:10.150603    2532 flags.go:64] FLAG: --sync-frequency="1m0s"
Sep 18 20:01:10.158195 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:10.150606    2532 flags.go:64] FLAG: --system-cgroups=""
Sep 18 20:01:10.158195 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:10.150609    2532 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Sep 18 20:01:10.158195 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:10.150614    2532 flags.go:64] FLAG: --system-reserved-cgroup=""
Sep 18 20:01:10.158195 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:10.150618    2532 flags.go:64] FLAG: --tls-cert-file=""
Sep 18 20:01:10.158195 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:10.150620    2532 flags.go:64] FLAG: --tls-cipher-suites="[]"
Sep 18 20:01:10.158195 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:10.150625    2532 flags.go:64] FLAG: --tls-min-version=""
Sep 18 20:01:10.158195 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:10.150628    2532 flags.go:64] FLAG: --tls-private-key-file=""
Sep 18 20:01:10.158195 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:10.150631    2532 flags.go:64] FLAG: --topology-manager-policy="none"
Sep 18 20:01:10.158195 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:10.150635    2532 flags.go:64] FLAG: --topology-manager-policy-options=""
Sep 18 20:01:10.158195 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:10.150638    2532 flags.go:64] FLAG: --topology-manager-scope="container"
Sep 18 20:01:10.159043 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:10.150641    2532 flags.go:64] FLAG: --v="2"
Sep 18 20:01:10.159043 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:10.150645    2532 flags.go:64] FLAG: --version="false"
Sep 18 20:01:10.159043 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:10.150649    2532 flags.go:64] FLAG: --vmodule=""
Sep 18 20:01:10.159043 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:10.150653    2532 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Sep 18 20:01:10.159043 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:10.150656    2532 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Sep 18 20:01:10.159043 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.150771    2532 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 18 20:01:10.159043 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.150774    2532 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 18 20:01:10.159043 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.150777    2532 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 18 20:01:10.159043 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.150780    2532 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 18 20:01:10.159043 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.150783    2532 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 18 20:01:10.159043 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.150786    2532 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 18 20:01:10.159043 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.150789    2532 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 18 20:01:10.159043 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.150791    2532 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 18 20:01:10.159043 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.150795    2532 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 18 20:01:10.159043 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.150798    2532 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 18 20:01:10.159043 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.150801    2532 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 18 20:01:10.159043 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.150803    2532 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 18 20:01:10.159043 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.150806    2532 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 18 20:01:10.159043 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.150809    2532 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 18 20:01:10.159043 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.150811    2532 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 18 20:01:10.159043 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.150814    2532 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 18 20:01:10.159750 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.150817    2532 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 18 20:01:10.159750 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.150819    2532 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 18 20:01:10.159750 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.150822    2532 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 18 20:01:10.159750 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.150825    2532 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 18 20:01:10.159750 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.150828    2532 feature_gate.go:328] unrecognized feature gate: Example
Sep 18 20:01:10.159750 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.150831    2532 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 18 20:01:10.159750 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.150834    2532 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 18 20:01:10.159750 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.150837    2532 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 18 20:01:10.159750 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.150840    2532 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 18 20:01:10.159750 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.150844    2532 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 18 20:01:10.159750 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.150847    2532 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 18 20:01:10.159750 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.150849    2532 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 18 20:01:10.159750 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.150852    2532 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 18 20:01:10.159750 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.150855    2532 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 18 20:01:10.159750 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.150858    2532 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 18 20:01:10.159750 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.150860    2532 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 18 20:01:10.159750 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.150863    2532 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 18 20:01:10.159750 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.150866    2532 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 18 20:01:10.159750 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.150868    2532 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 18 20:01:10.159750 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.150871    2532 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 18 20:01:10.160243 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.150874    2532 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 18 20:01:10.160243 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.150876    2532 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 18 20:01:10.160243 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.150879    2532 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 18 20:01:10.160243 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.150881    2532 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 18 20:01:10.160243 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.150885    2532 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 18 20:01:10.160243 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.150888    2532 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 18 20:01:10.160243 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.150892    2532 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 18 20:01:10.160243 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.150895    2532 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 18 20:01:10.160243 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.150898    2532 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 18 20:01:10.160243 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.150901    2532 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 18 20:01:10.160243 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.150903    2532 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 18 20:01:10.160243 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.150906    2532 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 18 20:01:10.160243 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.150908    2532 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 18 20:01:10.160243 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.150911    2532 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 18 20:01:10.160243 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.150914    2532 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 18 20:01:10.160243 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.150916    2532 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 18 20:01:10.160243 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.150919    2532 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 18 20:01:10.160243 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.150922    2532 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 18 20:01:10.160243 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.150925    2532 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 18 20:01:10.160243 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.150927    2532 feature_gate.go:328] unrecognized feature gate: Example2
Sep 18 20:01:10.160797 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.150930    2532 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 18 20:01:10.160797 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.150936    2532 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 18 20:01:10.160797 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.150939    2532 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 18 20:01:10.160797 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.150943    2532 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 18 20:01:10.160797 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.150946    2532 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 18 20:01:10.160797 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.150949    2532 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 18 20:01:10.160797 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.150952    2532 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 18 20:01:10.160797 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.150954    2532 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 18 20:01:10.160797 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.150957    2532 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 18 20:01:10.160797 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.150959    2532 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 18 20:01:10.160797 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.150962    2532 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 18 20:01:10.160797 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.150965    2532 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 18 20:01:10.160797 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.150967    2532 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 18 20:01:10.160797 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.150970    2532 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 18 20:01:10.160797 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.150973    2532 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 18 20:01:10.160797 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.150975    2532 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 18 20:01:10.160797 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.150979    2532 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 18 20:01:10.160797 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.150982    2532 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 18 20:01:10.160797 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.150985    2532 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 18 20:01:10.161281 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.150987    2532 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 18 20:01:10.161281 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.150990    2532 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 18 20:01:10.161281 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.150993    2532 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 18 20:01:10.161281 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.150996    2532 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 18 20:01:10.161281 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.150998    2532 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 18 20:01:10.161281 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.151001    2532 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 18 20:01:10.161281 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.151004    2532 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 18 20:01:10.161281 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.151006    2532 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 18 20:01:10.161281 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.151009    2532 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 18 20:01:10.161281 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.151011    2532 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 18 20:01:10.161281 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.151014    2532 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 18 20:01:10.161281 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.151017    2532 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 18 20:01:10.161281 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.151020    2532 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 18 20:01:10.161281 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.151022    2532 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 18 20:01:10.161281 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.151026    2532 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 18 20:01:10.161281 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.151029    2532 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 18 20:01:10.161281 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.151033    2532 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 18 20:01:10.161281 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.151036    2532 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 18 20:01:10.161281 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.151039    2532 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 18 20:01:10.161766 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.151042    2532 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 18 20:01:10.161766 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.151045    2532 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 18 20:01:10.161766 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.151048    2532 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 18 20:01:10.161766 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.151051    2532 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 18 20:01:10.161766 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.151054    2532 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 18 20:01:10.161766 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:10.151059    2532 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 18 20:01:10.161766 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:10.160087    2532 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Sep 18 20:01:10.161766 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:10.160109    2532 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Sep 18 20:01:10.161766 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:10.160129    2532 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Sep 18 20:01:10.161766 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:10.160139    2532 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Sep 18 20:01:10.161766 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.160197    2532 feature_gate.go:328] unrecognized feature gate: Example
Sep 18 20:01:10.161766 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.160202    2532 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 18 20:01:10.161766 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.160205    2532 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 18 20:01:10.161766 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.160208    2532 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 18 20:01:10.161766 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.160212    2532 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 18 20:01:10.162153 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.160215    2532 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 18 20:01:10.162153 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.160218    2532 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 18 20:01:10.162153 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.160221    2532 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 18 20:01:10.162153 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.160223    2532 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 18 20:01:10.162153 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.160226    2532 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 18 20:01:10.162153 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.160229    2532 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 18 20:01:10.162153 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.160232    2532 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 18 20:01:10.162153 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.160234    2532 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 18 20:01:10.162153 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.160237    2532 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 18 20:01:10.162153 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.160240    2532 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 18 20:01:10.162153 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.160242    2532 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 18 20:01:10.162153 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.160245    2532 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 18 20:01:10.162153 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.160248    2532 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 18 20:01:10.162153 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.160251    2532 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 18 20:01:10.162153 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.160253    2532 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 18 20:01:10.162153 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.160256    2532 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 18 20:01:10.162153 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.160259    2532 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 18 20:01:10.162153 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.160262    2532 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 18 20:01:10.162153 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.160265    2532 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 18 20:01:10.162682 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.160267    2532 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 18 20:01:10.162682 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.160270    2532 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 18 20:01:10.162682 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.160273    2532 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 18 20:01:10.162682 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.160276    2532 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 18 20:01:10.162682 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.160279    2532 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 18 20:01:10.162682 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.160282    2532 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 18 20:01:10.162682 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.160285    2532 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 18 20:01:10.162682 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.160287    2532 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 18 20:01:10.162682 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.160290    2532 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 18 20:01:10.162682 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.160293    2532 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 18 20:01:10.162682 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.160295    2532 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 18 20:01:10.162682 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.160298    2532 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 18 20:01:10.162682 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.160301    2532 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 18 20:01:10.162682 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.160303    2532 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 18 20:01:10.162682 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.160306    2532 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 18 20:01:10.162682 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.160309    2532 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 18 20:01:10.162682 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.160311    2532 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 18 20:01:10.162682 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.160314    2532 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 18 20:01:10.162682 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.160316    2532 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 18 20:01:10.162682 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.160319    2532 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 18 20:01:10.163168 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.160321    2532 feature_gate.go:328] unrecognized feature gate: Example2
Sep 18 20:01:10.163168 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.160324    2532 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 18 20:01:10.163168 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.160326    2532 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 18 20:01:10.163168 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.160329    2532 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 18 20:01:10.163168 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.160333    2532 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 18 20:01:10.163168 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.160337    2532 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 18 20:01:10.163168 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.160340    2532 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 18 20:01:10.163168 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.160343    2532 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 18 20:01:10.163168 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.160346    2532 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 18 20:01:10.163168 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.160350    2532 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 18 20:01:10.163168 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.160354    2532 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 18 20:01:10.163168 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.160357    2532 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 18 20:01:10.163168 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.160361    2532 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 18 20:01:10.163168 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.160364    2532 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 18 20:01:10.163168 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.160367    2532 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 18 20:01:10.163168 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.160370    2532 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 18 20:01:10.163168 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.160373    2532 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 18 20:01:10.163168 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.160376    2532 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 18 20:01:10.163168 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.160379    2532 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 18 20:01:10.163659 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.160381    2532 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 18 20:01:10.163659 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.160384    2532 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 18 20:01:10.163659 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.160387    2532 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 18 20:01:10.163659 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.160390    2532 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 18 20:01:10.163659 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.160392    2532 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 18 20:01:10.163659 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.160395    2532 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 18 20:01:10.163659 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.160398    2532 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 18 20:01:10.163659 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.160400    2532 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 18 20:01:10.163659 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.160403    2532 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 18 20:01:10.163659 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.160406    2532 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 18 20:01:10.163659 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.160410    2532 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 18 20:01:10.163659 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.160413    2532 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 18 20:01:10.163659 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.160416    2532 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 18 20:01:10.163659 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.160418    2532 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 18 20:01:10.163659 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.160422    2532 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 18 20:01:10.163659 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.160425    2532 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 18 20:01:10.163659 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.160429    2532 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 18 20:01:10.163659 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.160432    2532 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 18 20:01:10.163659 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.160434    2532 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 18 20:01:10.164161 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.160437    2532 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 18 20:01:10.164161 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.160440    2532 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 18 20:01:10.164161 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.160442    2532 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 18 20:01:10.164161 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.160445    2532 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 18 20:01:10.164161 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.160447    2532 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 18 20:01:10.164161 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.160450    2532 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 18 20:01:10.164161 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.160453    2532 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 18 20:01:10.164161 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.160455    2532 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 18 20:01:10.164161 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.160458    2532 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 18 20:01:10.164161 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.160461    2532 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 18 20:01:10.164161 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.160464    2532 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 18 20:01:10.164161 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.160467    2532 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 18 20:01:10.164161 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.160469    2532 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 18 20:01:10.164161 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.160472    2532 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 18 20:01:10.164161 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.160474    2532 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 18 20:01:10.164161 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.160477    2532 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 18 20:01:10.164161 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.160480    2532 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 18 20:01:10.164583 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:10.160502    2532 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 18 20:01:10.164583 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.160639    2532 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 18 20:01:10.164583 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.160645    2532 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 18 20:01:10.164583 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.160648    2532 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 18 20:01:10.164583 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.160651    2532 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 18 20:01:10.164583 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.160654    2532 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 18 20:01:10.164583 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.160657    2532 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 18 20:01:10.164583 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.160660    2532 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 18 20:01:10.164583 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.160662    2532 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 18 20:01:10.164583 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.160665    2532 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 18 20:01:10.164583 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.160667    2532 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 18 20:01:10.164583 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.160670    2532 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 18 20:01:10.164583 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.160673    2532 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 18 20:01:10.164583 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.160676    2532 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 18 20:01:10.164583 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.160679    2532 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 18 20:01:10.164957 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.160681    2532 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 18 20:01:10.164957 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.160686    2532 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 18 20:01:10.164957 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.160690    2532 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 18 20:01:10.164957 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.160693    2532 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 18 20:01:10.164957 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.160696    2532 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 18 20:01:10.164957 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.160699    2532 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 18 20:01:10.164957 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.160702    2532 feature_gate.go:328] unrecognized feature gate: Example2
Sep 18 20:01:10.164957 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.160705    2532 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 18 20:01:10.164957 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.160707    2532 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 18 20:01:10.164957 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.160710    2532 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 18 20:01:10.164957 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.160712    2532 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 18 20:01:10.164957 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.160716    2532 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 18 20:01:10.164957 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.160719    2532 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 18 20:01:10.164957 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.160722    2532 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 18 20:01:10.164957 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.160726    2532 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 18 20:01:10.164957 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.160729    2532 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 18 20:01:10.164957 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.160732    2532 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 18 20:01:10.164957 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.160735    2532 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 18 20:01:10.164957 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.160737    2532 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 18 20:01:10.165429 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.160740    2532 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 18 20:01:10.165429 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.160742    2532 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 18 20:01:10.165429 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.160745    2532 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 18 20:01:10.165429 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.160748    2532 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 18 20:01:10.165429 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.160750    2532 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 18 20:01:10.165429 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.160753    2532 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 18 20:01:10.165429 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.160755    2532 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 18 20:01:10.165429 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.160758    2532 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 18 20:01:10.165429 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.160761    2532 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 18 20:01:10.165429 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.160763    2532 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 18 20:01:10.165429 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.160766    2532 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 18 20:01:10.165429 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.160768    2532 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 18 20:01:10.165429 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.160771    2532 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 18 20:01:10.165429 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.160774    2532 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 18 20:01:10.165429 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.160777    2532 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 18 20:01:10.165429 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.160780    2532 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 18 20:01:10.165429 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.160782    2532 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 18 20:01:10.165429 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.160785    2532 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 18 20:01:10.165429 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.160787    2532 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 18 20:01:10.165429 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.160790    2532 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 18 20:01:10.165922 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.160792    2532 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 18 20:01:10.165922 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.160795    2532 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 18 20:01:10.165922 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.160797    2532 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 18 20:01:10.165922 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.160800    2532 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 18 20:01:10.165922 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.160804    2532 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 18 20:01:10.165922 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.160806    2532 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 18 20:01:10.165922 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.160809    2532 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 18 20:01:10.165922 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.160812    2532 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 18 20:01:10.165922 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.160814    2532 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 18 20:01:10.165922 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.160817    2532 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 18 20:01:10.165922 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.160820    2532 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 18 20:01:10.165922 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.160822    2532 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 18 20:01:10.165922 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.160825    2532 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 18 20:01:10.165922 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.160828    2532 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 18 20:01:10.165922 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.160830    2532 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 18 20:01:10.165922 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.160832    2532 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 18 20:01:10.165922 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.160835    2532 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 18 20:01:10.165922 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.160838    2532 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 18 20:01:10.165922 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.160841    2532 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 18 20:01:10.165922 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.160844    2532 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 18 20:01:10.166415 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.160846    2532 feature_gate.go:328] unrecognized feature gate: Example
Sep 18 20:01:10.166415 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.160849    2532 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 18 20:01:10.166415 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.160852    2532 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 18 20:01:10.166415 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.160855    2532 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 18 20:01:10.166415 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.160858    2532 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 18 20:01:10.166415 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.160860    2532 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 18 20:01:10.166415 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.160863    2532 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 18 20:01:10.166415 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.160866    2532 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 18 20:01:10.166415 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.160869    2532 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 18 20:01:10.166415 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.160871    2532 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 18 20:01:10.166415 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.160874    2532 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 18 20:01:10.166415 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.160876    2532 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 18 20:01:10.166415 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.160879    2532 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 18 20:01:10.166415 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.160881    2532 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 18 20:01:10.166415 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.160884    2532 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 18 20:01:10.166415 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.160887    2532 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 18 20:01:10.166415 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.160890    2532 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 18 20:01:10.166415 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.160892    2532 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 18 20:01:10.166415 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.160894    2532 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 18 20:01:10.166415 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.160897    2532 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 18 20:01:10.166933 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.160900    2532 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 18 20:01:10.166933 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.160902    2532 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 18 20:01:10.166933 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.160904    2532 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 18 20:01:10.166933 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.160908    2532 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 18 20:01:10.166933 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.160911    2532 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 18 20:01:10.166933 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:10.160914    2532 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 18 20:01:10.166933 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:10.160919    2532 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 18 20:01:10.166933 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:10.161036    2532 server.go:962] "Client rotation is on, will bootstrap in background"
Sep 18 20:01:10.166933 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:10.163021    2532 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Sep 18 20:01:10.166933 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:10.163918    2532 server.go:1019] "Starting client certificate rotation"
Sep 18 20:01:10.166933 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:10.164017    2532 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 18 20:01:10.166933 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:10.164048    2532 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 18 20:01:10.166933 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:10.166510    2532 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 18 20:01:10.167666 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:10.167648    2532 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 18 20:01:10.181305 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:10.181284    2532 log.go:25] "Validated CRI v1 runtime API"
Sep 18 20:01:10.183021 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:10.183005    2532 log.go:25] "Validated CRI v1 image API"
Sep 18 20:01:10.183861 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:10.183847    2532 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Sep 18 20:01:10.186035 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:10.186016    2532 fs.go:135] Filesystem UUIDs: map[11c2268d-6af5-4aaa-b632-04c3a672ea06:/dev/nvme0n1p3 1f26156d-bcaf-48a6-823c-23be09f942c4:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2]
Sep 18 20:01:10.186098 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:10.186034    2532 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Sep 18 20:01:10.190868 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:10.190760    2532 manager.go:216] Machine: {Timestamp:2026-09-18 20:01:10.190280495 +0000 UTC m=+1.955669885 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3100136 MemoryCapacity:33164480512 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2ee6b942cbccaf7cbf45d63b30cb45 SystemUUID:ec2ee6b9-42cb-ccaf-7cbf-45d63b30cb45 BootID:970e819b-ee1a-488b-b658-030231851c81 Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582238208 Type:vfs Inodes:4048398 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582242304 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:ff:e2:15:10:53 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:ff:e2:15:10:53 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:5a:09:74:29:d1:ac Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164480512 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Sep 18 20:01:10.190868 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:10.190857    2532 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Sep 18 20:01:10.191007 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:10.190928    2532 manager.go:232] Version: {KernelVersion:5.14.0-570.141.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260914-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Sep 18 20:01:10.192550 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:10.192523    2532 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Sep 18 20:01:10.192722 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:10.192558    2532 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-130-81.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Sep 18 20:01:10.193769 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:10.193759    2532 topology_manager.go:138] "Creating topology manager with none policy"
Sep 18 20:01:10.193809 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:10.193773    2532 container_manager_linux.go:309] "Creating device plugin manager"
Sep 18 20:01:10.193809 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:10.193781    2532 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 18 20:01:10.193809 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:10.193795    2532 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 18 20:01:10.193888 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:10.193865    2532 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Sep 18 20:01:10.197443 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:10.197430    2532 state_mem.go:36] "Initialized new in-memory state store"
Sep 18 20:01:10.197609 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:10.197598    2532 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Sep 18 20:01:10.197650 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:10.197637    2532 kubelet.go:486] "Attempting to sync node with API server"
Sep 18 20:01:10.197678 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:10.197652    2532 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Sep 18 20:01:10.197678 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:10.197666    2532 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Sep 18 20:01:10.197760 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:10.197678    2532 kubelet.go:398] "Adding apiserver pod source"
Sep 18 20:01:10.197760 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:10.197690    2532 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Sep 18 20:01:10.199012 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:10.198983    2532 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 18 20:01:10.199096 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:10.199017    2532 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 18 20:01:10.201906 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:10.201884    2532 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.13-6.rhaos4.21.git6253162.el9" apiVersion="v1"
Sep 18 20:01:10.202291 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:10.202272    2532 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 18 20:01:10.204044 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:10.204030    2532 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Sep 18 20:01:10.204087 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:10.204061    2532 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Sep 18 20:01:10.205028 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:10.205017    2532 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Sep 18 20:01:10.205502 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:10.205037    2532 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Sep 18 20:01:10.205502 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:10.205044    2532 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Sep 18 20:01:10.205502 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:10.205049    2532 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Sep 18 20:01:10.205502 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:10.205058    2532 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Sep 18 20:01:10.205502 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:10.205064    2532 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Sep 18 20:01:10.205502 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:10.205077    2532 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Sep 18 20:01:10.205502 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:10.205083    2532 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Sep 18 20:01:10.205502 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:10.205089    2532 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Sep 18 20:01:10.205502 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:10.205095    2532 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Sep 18 20:01:10.205502 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:10.205104    2532 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Sep 18 20:01:10.205502 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:10.205113    2532 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Sep 18 20:01:10.205502 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:10.205139    2532 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Sep 18 20:01:10.205502 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:10.205144    2532 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Sep 18 20:01:10.209344 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:10.209329    2532 server.go:1268] "Started kubelet"
Sep 18 20:01:10.209463 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:10.209437    2532 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Sep 18 20:01:10.209534 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:10.209439    2532 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Sep 18 20:01:10.209534 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:10.209519    2532 server_v1.go:49] "podresources" method="list" useActivePods=true
Sep 18 20:01:10.209854 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:10.209836    2532 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-130-81.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Sep 18 20:01:10.210203 ip-10-0-130-81 kubenswrapper[2532]: E0918 20:01:10.210176    2532 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-130-81.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Sep 18 20:01:10.210203 ip-10-0-130-81 kubenswrapper[2532]: E0918 20:01:10.210187    2532 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Sep 18 20:01:10.210252 ip-10-0-130-81 systemd[1]: Started Kubernetes Kubelet.
Sep 18 20:01:10.210866 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:10.210521    2532 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Sep 18 20:01:10.211140 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:10.211077    2532 server.go:310] "Adding debug handlers to kubelet server"
Sep 18 20:01:10.213597 ip-10-0-130-81 kubenswrapper[2532]: E0918 20:01:10.213577    2532 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Sep 18 20:01:10.216633 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:10.216615    2532 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Sep 18 20:01:10.216704 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:10.216623    2532 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Sep 18 20:01:10.216806 ip-10-0-130-81 kubenswrapper[2532]: E0918 20:01:10.216771    2532 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-81.ec2.internal\" not found"
Sep 18 20:01:10.217279 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:10.217261    2532 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Sep 18 20:01:10.217279 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:10.217261    2532 volume_manager.go:311] "The desired_state_of_world populator starts"
Sep 18 20:01:10.217420 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:10.217290    2532 volume_manager.go:313] "Starting Kubelet Volume Manager"
Sep 18 20:01:10.217420 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:10.217415    2532 reconstruct.go:97] "Volume reconstruction finished"
Sep 18 20:01:10.217540 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:10.217424    2532 reconciler.go:29] "Reconciler: start to sync state"
Sep 18 20:01:10.217540 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:10.217473    2532 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Sep 18 20:01:10.217540 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:10.217500    2532 factory.go:55] Registering systemd factory
Sep 18 20:01:10.217540 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:10.217509    2532 factory.go:223] Registration of the systemd container factory successfully
Sep 18 20:01:10.217810 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:10.217794    2532 factory.go:153] Registering CRI-O factory
Sep 18 20:01:10.217810 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:10.217810    2532 factory.go:223] Registration of the crio container factory successfully
Sep 18 20:01:10.217949 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:10.217824    2532 factory.go:103] Registering Raw factory
Sep 18 20:01:10.217949 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:10.217834    2532 manager.go:1196] Started watching for new ooms in manager
Sep 18 20:01:10.218328 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:10.218309    2532 manager.go:349] Starting recovery of all containers
Sep 18 20:01:10.218575 ip-10-0-130-81 kubenswrapper[2532]: E0918 20:01:10.218552    2532 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-130-81.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Sep 18 20:01:10.218664 ip-10-0-130-81 kubenswrapper[2532]: E0918 20:01:10.218645    2532 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Sep 18 20:01:10.220920 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:10.220895    2532 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-tkgs5"
Sep 18 20:01:10.240539 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:10.240359    2532 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-tkgs5"
Sep 18 20:01:10.240841 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:10.240799    2532 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Sep 18 20:01:10.245084 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:10.245068    2532 manager.go:354] Recovery completed
Sep 18 20:01:10.249267 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:10.249254    2532 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 18 20:01:10.251179 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:10.251161    2532 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-81.ec2.internal" event="NodeHasSufficientMemory"
Sep 18 20:01:10.251252 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:10.251191    2532 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-81.ec2.internal" event="NodeHasNoDiskPressure"
Sep 18 20:01:10.251252 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:10.251204    2532 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-81.ec2.internal" event="NodeHasSufficientPID"
Sep 18 20:01:10.251765 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:10.251752    2532 cpu_manager.go:222] "Starting CPU manager" policy="none"
Sep 18 20:01:10.251765 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:10.251761    2532 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 18 20:01:10.251885 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:10.251763    2532 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Sep 18 20:01:10.251885 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:10.251828    2532 state_mem.go:36] "Initialized new in-memory state store"
Sep 18 20:01:10.252397 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:10.252384    2532 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-81.ec2.internal" event="NodeHasSufficientMemory"
Sep 18 20:01:10.252459 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:10.252412    2532 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-81.ec2.internal" event="NodeHasNoDiskPressure"
Sep 18 20:01:10.252459 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:10.252426    2532 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-81.ec2.internal" event="NodeHasSufficientPID"
Sep 18 20:01:10.253838 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:10.253825    2532 policy_none.go:49] "None policy: Start"
Sep 18 20:01:10.253838 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:10.253840    2532 memory_manager.go:187] "Starting memorymanager" policy="None"
Sep 18 20:01:10.253924 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:10.253850    2532 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Sep 18 20:01:10.255764 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:10.255753    2532 policy_none.go:47] "Start"
Sep 18 20:01:10.289090 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:10.289074    2532 manager.go:337] "Starting Device Plugin manager"
Sep 18 20:01:10.310466 ip-10-0-130-81 kubenswrapper[2532]: E0918 20:01:10.289116    2532 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Sep 18 20:01:10.310466 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:10.289127    2532 server.go:87] "Starting device plugin registration server"
Sep 18 20:01:10.310466 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:10.289388    2532 eviction_manager.go:189] "Eviction manager: starting control loop"
Sep 18 20:01:10.310466 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:10.289407    2532 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Sep 18 20:01:10.310466 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:10.289470    2532 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Sep 18 20:01:10.310466 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:10.289592    2532 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Sep 18 20:01:10.310466 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:10.289599    2532 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Sep 18 20:01:10.310466 ip-10-0-130-81 kubenswrapper[2532]: E0918 20:01:10.290254    2532 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Sep 18 20:01:10.310466 ip-10-0-130-81 kubenswrapper[2532]: E0918 20:01:10.290295    2532 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-130-81.ec2.internal\" not found"
Sep 18 20:01:10.310466 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:10.299434    2532 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Sep 18 20:01:10.310466 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:10.299513    2532 status_manager.go:244] "Starting to sync pod status with apiserver"
Sep 18 20:01:10.310466 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:10.299540    2532 kubelet.go:2448] "Starting kubelet main sync loop"
Sep 18 20:01:10.310466 ip-10-0-130-81 kubenswrapper[2532]: E0918 20:01:10.299582    2532 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Sep 18 20:01:10.310466 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:10.301727    2532 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 18 20:01:10.390554 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:10.390510    2532 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 18 20:01:10.391509 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:10.391475    2532 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-81.ec2.internal" event="NodeHasSufficientMemory"
Sep 18 20:01:10.391654 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:10.391525    2532 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-81.ec2.internal" event="NodeHasNoDiskPressure"
Sep 18 20:01:10.391654 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:10.391536    2532 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-81.ec2.internal" event="NodeHasSufficientPID"
Sep 18 20:01:10.391654 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:10.391559    2532 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-130-81.ec2.internal"
Sep 18 20:01:10.400142 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:10.400098    2532 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-130-81.ec2.internal"
Sep 18 20:01:10.400142 ip-10-0-130-81 kubenswrapper[2532]: E0918 20:01:10.400123    2532 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-130-81.ec2.internal\": node \"ip-10-0-130-81.ec2.internal\" not found"
Sep 18 20:01:10.400269 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:10.400252    2532 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-81.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-130-81.ec2.internal"]
Sep 18 20:01:10.400326 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:10.400316    2532 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 18 20:01:10.401192 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:10.401176    2532 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-81.ec2.internal" event="NodeHasSufficientMemory"
Sep 18 20:01:10.401291 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:10.401208    2532 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-81.ec2.internal" event="NodeHasNoDiskPressure"
Sep 18 20:01:10.401291 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:10.401222    2532 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-81.ec2.internal" event="NodeHasSufficientPID"
Sep 18 20:01:10.403518 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:10.403497    2532 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 18 20:01:10.403618 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:10.403593    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-81.ec2.internal"
Sep 18 20:01:10.403668 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:10.403625    2532 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 18 20:01:10.404175 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:10.404160    2532 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-81.ec2.internal" event="NodeHasSufficientMemory"
Sep 18 20:01:10.404256 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:10.404186    2532 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-81.ec2.internal" event="NodeHasNoDiskPressure"
Sep 18 20:01:10.404256 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:10.404200    2532 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-81.ec2.internal" event="NodeHasSufficientPID"
Sep 18 20:01:10.404256 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:10.404164    2532 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-81.ec2.internal" event="NodeHasSufficientMemory"
Sep 18 20:01:10.404402 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:10.404264    2532 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-81.ec2.internal" event="NodeHasNoDiskPressure"
Sep 18 20:01:10.404402 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:10.404276    2532 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-81.ec2.internal" event="NodeHasSufficientPID"
Sep 18 20:01:10.406335 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:10.406321    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-81.ec2.internal"
Sep 18 20:01:10.406393 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:10.406345    2532 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 18 20:01:10.406985 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:10.406969    2532 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-81.ec2.internal" event="NodeHasSufficientMemory"
Sep 18 20:01:10.407050 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:10.406997    2532 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-81.ec2.internal" event="NodeHasNoDiskPressure"
Sep 18 20:01:10.407050 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:10.407009    2532 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-81.ec2.internal" event="NodeHasSufficientPID"
Sep 18 20:01:10.410445 ip-10-0-130-81 kubenswrapper[2532]: E0918 20:01:10.410428    2532 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-81.ec2.internal\" not found"
Sep 18 20:01:10.418611 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:10.418590    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ae7090a051a4a9229a0e9951ab9e2daf-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-130-81.ec2.internal\" (UID: \"ae7090a051a4a9229a0e9951ab9e2daf\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-81.ec2.internal"
Sep 18 20:01:10.418672 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:10.418617    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/7719088cb6c9e3ddb918a0f82975ee3c-config\") pod \"kube-apiserver-proxy-ip-10-0-130-81.ec2.internal\" (UID: \"7719088cb6c9e3ddb918a0f82975ee3c\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-130-81.ec2.internal"
Sep 18 20:01:10.418672 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:10.418633    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/ae7090a051a4a9229a0e9951ab9e2daf-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-130-81.ec2.internal\" (UID: \"ae7090a051a4a9229a0e9951ab9e2daf\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-81.ec2.internal"
Sep 18 20:01:10.428227 ip-10-0-130-81 kubenswrapper[2532]: E0918 20:01:10.428205    2532 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-130-81.ec2.internal\" not found" node="ip-10-0-130-81.ec2.internal"
Sep 18 20:01:10.432649 ip-10-0-130-81 kubenswrapper[2532]: E0918 20:01:10.432634    2532 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-130-81.ec2.internal\" not found" node="ip-10-0-130-81.ec2.internal"
Sep 18 20:01:10.511336 ip-10-0-130-81 kubenswrapper[2532]: E0918 20:01:10.511302    2532 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-81.ec2.internal\" not found"
Sep 18 20:01:10.519745 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:10.519716    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/7719088cb6c9e3ddb918a0f82975ee3c-config\") pod \"kube-apiserver-proxy-ip-10-0-130-81.ec2.internal\" (UID: \"7719088cb6c9e3ddb918a0f82975ee3c\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-130-81.ec2.internal"
Sep 18 20:01:10.519745 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:10.519681    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/7719088cb6c9e3ddb918a0f82975ee3c-config\") pod \"kube-apiserver-proxy-ip-10-0-130-81.ec2.internal\" (UID: \"7719088cb6c9e3ddb918a0f82975ee3c\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-130-81.ec2.internal"
Sep 18 20:01:10.519903 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:10.519790    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/ae7090a051a4a9229a0e9951ab9e2daf-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-130-81.ec2.internal\" (UID: \"ae7090a051a4a9229a0e9951ab9e2daf\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-81.ec2.internal"
Sep 18 20:01:10.519903 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:10.519812    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ae7090a051a4a9229a0e9951ab9e2daf-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-130-81.ec2.internal\" (UID: \"ae7090a051a4a9229a0e9951ab9e2daf\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-81.ec2.internal"
Sep 18 20:01:10.519903 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:10.519838    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ae7090a051a4a9229a0e9951ab9e2daf-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-130-81.ec2.internal\" (UID: \"ae7090a051a4a9229a0e9951ab9e2daf\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-81.ec2.internal"
Sep 18 20:01:10.519903 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:10.519871    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/ae7090a051a4a9229a0e9951ab9e2daf-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-130-81.ec2.internal\" (UID: \"ae7090a051a4a9229a0e9951ab9e2daf\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-81.ec2.internal"
Sep 18 20:01:10.612397 ip-10-0-130-81 kubenswrapper[2532]: E0918 20:01:10.612347    2532 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-81.ec2.internal\" not found"
Sep 18 20:01:10.713171 ip-10-0-130-81 kubenswrapper[2532]: E0918 20:01:10.713092    2532 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-81.ec2.internal\" not found"
Sep 18 20:01:10.731644 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:10.731618    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-81.ec2.internal"
Sep 18 20:01:10.735548 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:10.735523    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-81.ec2.internal"
Sep 18 20:01:10.813561 ip-10-0-130-81 kubenswrapper[2532]: E0918 20:01:10.813529    2532 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-81.ec2.internal\" not found"
Sep 18 20:01:10.914039 ip-10-0-130-81 kubenswrapper[2532]: E0918 20:01:10.914004    2532 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-81.ec2.internal\" not found"
Sep 18 20:01:11.014627 ip-10-0-130-81 kubenswrapper[2532]: E0918 20:01:11.014541    2532 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-81.ec2.internal\" not found"
Sep 18 20:01:11.115069 ip-10-0-130-81 kubenswrapper[2532]: E0918 20:01:11.115027    2532 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-81.ec2.internal\" not found"
Sep 18 20:01:11.163564 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:11.163536    2532 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Sep 18 20:01:11.164046 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:11.163675    2532 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 18 20:01:11.215981 ip-10-0-130-81 kubenswrapper[2532]: E0918 20:01:11.215952    2532 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-81.ec2.internal\" not found"
Sep 18 20:01:11.244623 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:11.244602    2532 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 18 20:01:11.246358 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:11.246321    2532 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-09-17 19:56:10 +0000 UTC" deadline="2028-06-09 10:36:46.26899538 +0000 UTC"
Sep 18 20:01:11.246358 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:11.246356    2532 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15110h35m35.022641926s"
Sep 18 20:01:11.316081 ip-10-0-130-81 kubenswrapper[2532]: E0918 20:01:11.316053    2532 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-81.ec2.internal\" not found"
Sep 18 20:01:11.354795 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:11.354766    2532 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 18 20:01:11.416660 ip-10-0-130-81 kubenswrapper[2532]: E0918 20:01:11.416626    2532 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-81.ec2.internal\" not found"
Sep 18 20:01:11.427157 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:11.427126    2532 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7719088cb6c9e3ddb918a0f82975ee3c.slice/crio-1ccff460ecbfebcf662a2145555ddfe713e870e3b7c984215400bb340561aae1 WatchSource:0}: Error finding container 1ccff460ecbfebcf662a2145555ddfe713e870e3b7c984215400bb340561aae1: Status 404 returned error can't find the container with id 1ccff460ecbfebcf662a2145555ddfe713e870e3b7c984215400bb340561aae1
Sep 18 20:01:11.427572 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:11.427555    2532 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podae7090a051a4a9229a0e9951ab9e2daf.slice/crio-3b1883c47cf5d28a65a23d3abc53b07a7fb46035bebe3cf4cb0e3385c970e75b WatchSource:0}: Error finding container 3b1883c47cf5d28a65a23d3abc53b07a7fb46035bebe3cf4cb0e3385c970e75b: Status 404 returned error can't find the container with id 3b1883c47cf5d28a65a23d3abc53b07a7fb46035bebe3cf4cb0e3385c970e75b
Sep 18 20:01:11.432991 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:11.432975    2532 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 18 20:01:11.516899 ip-10-0-130-81 kubenswrapper[2532]: E0918 20:01:11.516871    2532 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-81.ec2.internal\" not found"
Sep 18 20:01:11.617344 ip-10-0-130-81 kubenswrapper[2532]: E0918 20:01:11.617273    2532 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-81.ec2.internal\" not found"
Sep 18 20:01:11.718021 ip-10-0-130-81 kubenswrapper[2532]: E0918 20:01:11.717987    2532 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-81.ec2.internal\" not found"
Sep 18 20:01:11.747402 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:11.747377    2532 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 18 20:01:11.817694 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:11.817656    2532 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-81.ec2.internal"
Sep 18 20:01:11.827820 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:11.827794    2532 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 18 20:01:11.828782 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:11.828769    2532 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-81.ec2.internal"
Sep 18 20:01:11.836446 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:11.836430    2532 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 18 20:01:12.024650 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:12.024609    2532 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 18 20:01:12.198970 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:12.198694    2532 apiserver.go:52] "Watching apiserver"
Sep 18 20:01:12.206441 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:12.205520    2532 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Sep 18 20:01:12.206441 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:12.205992    2532 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/kube-apiserver-proxy-ip-10-0-130-81.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nbw5k","openshift-multus/multus-additional-cni-plugins-pn4rk","openshift-multus/multus-jbmm2","openshift-multus/network-metrics-daemon-xx7f7","kube-system/konnectivity-agent-g749g","openshift-cluster-node-tuning-operator/tuned-spc6q","openshift-dns/node-resolver-b8jbz","openshift-image-registry/node-ca-wkwnc","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-81.ec2.internal","openshift-network-diagnostics/network-check-target-gd24n","openshift-network-operator/iptables-alerter-dxcqj","openshift-ovn-kubernetes/ovnkube-node-rtw9n","kube-system/global-pull-secret-syncer-gd9qz"]
Sep 18 20:01:12.209852 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:12.209050    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-spc6q"
Sep 18 20:01:12.211666 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:12.211635    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-rtw9n"
Sep 18 20:01:12.215525 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:12.211929    2532 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Sep 18 20:01:12.215525 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:12.212195    2532 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-vdtd8\""
Sep 18 20:01:12.215525 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:12.212833    2532 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Sep 18 20:01:12.215525 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:12.214053    2532 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Sep 18 20:01:12.215525 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:12.214261    2532 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Sep 18 20:01:12.215525 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:12.214368    2532 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Sep 18 20:01:12.215525 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:12.214447    2532 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-wmmll\""
Sep 18 20:01:12.215525 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:12.214582    2532 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Sep 18 20:01:12.215525 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:12.214806    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-pn4rk"
Sep 18 20:01:12.215525 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:12.214872    2532 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Sep 18 20:01:12.215525 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:12.215297    2532 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Sep 18 20:01:12.217066 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:12.217044    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-jbmm2"
Sep 18 20:01:12.217666 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:12.217648    2532 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Sep 18 20:01:12.217982 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:12.217962    2532 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Sep 18 20:01:12.218869 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:12.218060    2532 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Sep 18 20:01:12.218869 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:12.218167    2532 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-kfm8w\""
Sep 18 20:01:12.218869 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:12.218336    2532 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Sep 18 20:01:12.218869 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:12.218338    2532 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Sep 18 20:01:12.218869 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:12.218552    2532 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Sep 18 20:01:12.219178 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:12.219058    2532 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Sep 18 20:01:12.220720 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:12.219213    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xx7f7"
Sep 18 20:01:12.220720 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:12.219313    2532 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-czwrm\""
Sep 18 20:01:12.220720 ip-10-0-130-81 kubenswrapper[2532]: E0918 20:01:12.219308    2532 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xx7f7" podUID="96167e3f-fb44-48e0-b86e-419a4a3b117c"
Sep 18 20:01:12.223376 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:12.221901    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-g749g"
Sep 18 20:01:12.225274 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:12.224668    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-wkwnc"
Sep 18 20:01:12.227759 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:12.227742    2532 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Sep 18 20:01:12.227854 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:12.227817    2532 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Sep 18 20:01:12.227985 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:12.227967    2532 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-pbs5l\""
Sep 18 20:01:12.229700 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:12.228867    2532 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-f2jnd\""
Sep 18 20:01:12.229700 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:12.229266    2532 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Sep 18 20:01:12.229840 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:12.229749    2532 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Sep 18 20:01:12.230333 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:12.230309    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/73fbf16b-3549-4a6c-a959-b633237ebd0c-systemd-units\") pod \"ovnkube-node-rtw9n\" (UID: \"73fbf16b-3549-4a6c-a959-b633237ebd0c\") " pod="openshift-ovn-kubernetes/ovnkube-node-rtw9n"
Sep 18 20:01:12.230408 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:12.230357    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/73fbf16b-3549-4a6c-a959-b633237ebd0c-var-lib-openvswitch\") pod \"ovnkube-node-rtw9n\" (UID: \"73fbf16b-3549-4a6c-a959-b633237ebd0c\") " pod="openshift-ovn-kubernetes/ovnkube-node-rtw9n"
Sep 18 20:01:12.230408 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:12.230394    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/5c8c76fd-1219-40d0-b010-cd89d714a3fe-cni-binary-copy\") pod \"multus-additional-cni-plugins-pn4rk\" (UID: \"5c8c76fd-1219-40d0-b010-cd89d714a3fe\") " pod="openshift-multus/multus-additional-cni-plugins-pn4rk"
Sep 18 20:01:12.230477 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:12.230320    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gd9qz"
Sep 18 20:01:12.230575 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:12.230558    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/bf9010d7-90b5-4f95-8dea-8af5839c0f87-hostroot\") pod \"multus-jbmm2\" (UID: \"bf9010d7-90b5-4f95-8dea-8af5839c0f87\") " pod="openshift-multus/multus-jbmm2"
Sep 18 20:01:12.230616 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:12.230595    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bf9010d7-90b5-4f95-8dea-8af5839c0f87-etc-kubernetes\") pod \"multus-jbmm2\" (UID: \"bf9010d7-90b5-4f95-8dea-8af5839c0f87\") " pod="openshift-multus/multus-jbmm2"
Sep 18 20:01:12.230616 ip-10-0-130-81 kubenswrapper[2532]: E0918 20:01:12.230603    2532 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gd9qz" podUID="b377a9e6-dbbb-4023-8997-70178a63034f"
Sep 18 20:01:12.230684 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:12.230629    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xnvlz\" (UniqueName: \"kubernetes.io/projected/96167e3f-fb44-48e0-b86e-419a4a3b117c-kube-api-access-xnvlz\") pod \"network-metrics-daemon-xx7f7\" (UID: \"96167e3f-fb44-48e0-b86e-419a4a3b117c\") " pod="openshift-multus/network-metrics-daemon-xx7f7"
Sep 18 20:01:12.230684 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:12.230664    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/73fbf16b-3549-4a6c-a959-b633237ebd0c-node-log\") pod \"ovnkube-node-rtw9n\" (UID: \"73fbf16b-3549-4a6c-a959-b633237ebd0c\") " pod="openshift-ovn-kubernetes/ovnkube-node-rtw9n"
Sep 18 20:01:12.230755 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:12.230694    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/6b3e5758-70d0-496d-90a9-35d371e8e1b6-etc-modprobe-d\") pod \"tuned-spc6q\" (UID: \"6b3e5758-70d0-496d-90a9-35d371e8e1b6\") " pod="openshift-cluster-node-tuning-operator/tuned-spc6q"
Sep 18 20:01:12.230942 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:12.230722    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6b3e5758-70d0-496d-90a9-35d371e8e1b6-sys\") pod \"tuned-spc6q\" (UID: \"6b3e5758-70d0-496d-90a9-35d371e8e1b6\") " pod="openshift-cluster-node-tuning-operator/tuned-spc6q"
Sep 18 20:01:12.231021 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:12.230956    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/73fbf16b-3549-4a6c-a959-b633237ebd0c-run-systemd\") pod \"ovnkube-node-rtw9n\" (UID: \"73fbf16b-3549-4a6c-a959-b633237ebd0c\") " pod="openshift-ovn-kubernetes/ovnkube-node-rtw9n"
Sep 18 20:01:12.231021 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:12.230994    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/73fbf16b-3549-4a6c-a959-b633237ebd0c-host-cni-netd\") pod \"ovnkube-node-rtw9n\" (UID: \"73fbf16b-3549-4a6c-a959-b633237ebd0c\") " pod="openshift-ovn-kubernetes/ovnkube-node-rtw9n"
Sep 18 20:01:12.231117 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:12.231024    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/bf9010d7-90b5-4f95-8dea-8af5839c0f87-host-run-k8s-cni-cncf-io\") pod \"multus-jbmm2\" (UID: \"bf9010d7-90b5-4f95-8dea-8af5839c0f87\") " pod="openshift-multus/multus-jbmm2"
Sep 18 20:01:12.231117 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:12.231053    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/bf9010d7-90b5-4f95-8dea-8af5839c0f87-host-var-lib-cni-multus\") pod \"multus-jbmm2\" (UID: \"bf9010d7-90b5-4f95-8dea-8af5839c0f87\") " pod="openshift-multus/multus-jbmm2"
Sep 18 20:01:12.231117 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:12.231082    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/73fbf16b-3549-4a6c-a959-b633237ebd0c-host-run-netns\") pod \"ovnkube-node-rtw9n\" (UID: \"73fbf16b-3549-4a6c-a959-b633237ebd0c\") " pod="openshift-ovn-kubernetes/ovnkube-node-rtw9n"
Sep 18 20:01:12.231322 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:12.231111    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/bf9010d7-90b5-4f95-8dea-8af5839c0f87-multus-socket-dir-parent\") pod \"multus-jbmm2\" (UID: \"bf9010d7-90b5-4f95-8dea-8af5839c0f87\") " pod="openshift-multus/multus-jbmm2"
Sep 18 20:01:12.231417 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:12.231335    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/6b3e5758-70d0-496d-90a9-35d371e8e1b6-etc-systemd\") pod \"tuned-spc6q\" (UID: \"6b3e5758-70d0-496d-90a9-35d371e8e1b6\") " pod="openshift-cluster-node-tuning-operator/tuned-spc6q"
Sep 18 20:01:12.231417 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:12.231368    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/6b3e5758-70d0-496d-90a9-35d371e8e1b6-lib-modules\") pod \"tuned-spc6q\" (UID: \"6b3e5758-70d0-496d-90a9-35d371e8e1b6\") " pod="openshift-cluster-node-tuning-operator/tuned-spc6q"
Sep 18 20:01:12.231417 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:12.231399    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/73fbf16b-3549-4a6c-a959-b633237ebd0c-log-socket\") pod \"ovnkube-node-rtw9n\" (UID: \"73fbf16b-3549-4a6c-a959-b633237ebd0c\") " pod="openshift-ovn-kubernetes/ovnkube-node-rtw9n"
Sep 18 20:01:12.231592 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:12.231430    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/5c8c76fd-1219-40d0-b010-cd89d714a3fe-tuning-conf-dir\") pod \"multus-additional-cni-plugins-pn4rk\" (UID: \"5c8c76fd-1219-40d0-b010-cd89d714a3fe\") " pod="openshift-multus/multus-additional-cni-plugins-pn4rk"
Sep 18 20:01:12.231592 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:12.231457    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/6b3e5758-70d0-496d-90a9-35d371e8e1b6-etc-tuned\") pod \"tuned-spc6q\" (UID: \"6b3e5758-70d0-496d-90a9-35d371e8e1b6\") " pod="openshift-cluster-node-tuning-operator/tuned-spc6q"
Sep 18 20:01:12.231592 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:12.231578    2532 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Sep 18 20:01:12.231736 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:12.231609    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-blmfw\" (UniqueName: \"kubernetes.io/projected/6b3e5758-70d0-496d-90a9-35d371e8e1b6-kube-api-access-blmfw\") pod \"tuned-spc6q\" (UID: \"6b3e5758-70d0-496d-90a9-35d371e8e1b6\") " pod="openshift-cluster-node-tuning-operator/tuned-spc6q"
Sep 18 20:01:12.231781 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:12.231760    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/bf9010d7-90b5-4f95-8dea-8af5839c0f87-multus-cni-dir\") pod \"multus-jbmm2\" (UID: \"bf9010d7-90b5-4f95-8dea-8af5839c0f87\") " pod="openshift-multus/multus-jbmm2"
Sep 18 20:01:12.231817 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:12.231782    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-trvsb\" (UniqueName: \"kubernetes.io/projected/73fbf16b-3549-4a6c-a959-b633237ebd0c-kube-api-access-trvsb\") pod \"ovnkube-node-rtw9n\" (UID: \"73fbf16b-3549-4a6c-a959-b633237ebd0c\") " pod="openshift-ovn-kubernetes/ovnkube-node-rtw9n"
Sep 18 20:01:12.231887 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:12.231869    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/6b3e5758-70d0-496d-90a9-35d371e8e1b6-etc-sysctl-d\") pod \"tuned-spc6q\" (UID: \"6b3e5758-70d0-496d-90a9-35d371e8e1b6\") " pod="openshift-cluster-node-tuning-operator/tuned-spc6q"
Sep 18 20:01:12.231944 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:12.231899    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6b3e5758-70d0-496d-90a9-35d371e8e1b6-tmp\") pod \"tuned-spc6q\" (UID: \"6b3e5758-70d0-496d-90a9-35d371e8e1b6\") " pod="openshift-cluster-node-tuning-operator/tuned-spc6q"
Sep 18 20:01:12.231997 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:12.231951    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/73fbf16b-3549-4a6c-a959-b633237ebd0c-etc-openvswitch\") pod \"ovnkube-node-rtw9n\" (UID: \"73fbf16b-3549-4a6c-a959-b633237ebd0c\") " pod="openshift-ovn-kubernetes/ovnkube-node-rtw9n"
Sep 18 20:01:12.231997 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:12.231986    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/73fbf16b-3549-4a6c-a959-b633237ebd0c-host-run-ovn-kubernetes\") pod \"ovnkube-node-rtw9n\" (UID: \"73fbf16b-3549-4a6c-a959-b633237ebd0c\") " pod="openshift-ovn-kubernetes/ovnkube-node-rtw9n"
Sep 18 20:01:12.232186 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:12.232168    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/73fbf16b-3549-4a6c-a959-b633237ebd0c-env-overrides\") pod \"ovnkube-node-rtw9n\" (UID: \"73fbf16b-3549-4a6c-a959-b633237ebd0c\") " pod="openshift-ovn-kubernetes/ovnkube-node-rtw9n"
Sep 18 20:01:12.232231 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:12.232210    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/6b3e5758-70d0-496d-90a9-35d371e8e1b6-etc-sysctl-conf\") pod \"tuned-spc6q\" (UID: \"6b3e5758-70d0-496d-90a9-35d371e8e1b6\") " pod="openshift-cluster-node-tuning-operator/tuned-spc6q"
Sep 18 20:01:12.232366 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:12.232351    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6b3e5758-70d0-496d-90a9-35d371e8e1b6-var-lib-kubelet\") pod \"tuned-spc6q\" (UID: \"6b3e5758-70d0-496d-90a9-35d371e8e1b6\") " pod="openshift-cluster-node-tuning-operator/tuned-spc6q"
Sep 18 20:01:12.232403 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:12.232379    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/6b3e5758-70d0-496d-90a9-35d371e8e1b6-host\") pod \"tuned-spc6q\" (UID: \"6b3e5758-70d0-496d-90a9-35d371e8e1b6\") " pod="openshift-cluster-node-tuning-operator/tuned-spc6q"
Sep 18 20:01:12.232403 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:12.232396    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4h9pz\" (UniqueName: \"kubernetes.io/projected/5c8c76fd-1219-40d0-b010-cd89d714a3fe-kube-api-access-4h9pz\") pod \"multus-additional-cni-plugins-pn4rk\" (UID: \"5c8c76fd-1219-40d0-b010-cd89d714a3fe\") " pod="openshift-multus/multus-additional-cni-plugins-pn4rk"
Sep 18 20:01:12.232469 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:12.232439    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/bf9010d7-90b5-4f95-8dea-8af5839c0f87-host-run-netns\") pod \"multus-jbmm2\" (UID: \"bf9010d7-90b5-4f95-8dea-8af5839c0f87\") " pod="openshift-multus/multus-jbmm2"
Sep 18 20:01:12.232802 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:12.232645    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/bf9010d7-90b5-4f95-8dea-8af5839c0f87-host-run-multus-certs\") pod \"multus-jbmm2\" (UID: \"bf9010d7-90b5-4f95-8dea-8af5839c0f87\") " pod="openshift-multus/multus-jbmm2"
Sep 18 20:01:12.232802 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:12.232679    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/96167e3f-fb44-48e0-b86e-419a4a3b117c-metrics-certs\") pod \"network-metrics-daemon-xx7f7\" (UID: \"96167e3f-fb44-48e0-b86e-419a4a3b117c\") " pod="openshift-multus/network-metrics-daemon-xx7f7"
Sep 18 20:01:12.232802 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:12.232709    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/6b3e5758-70d0-496d-90a9-35d371e8e1b6-run\") pod \"tuned-spc6q\" (UID: \"6b3e5758-70d0-496d-90a9-35d371e8e1b6\") " pod="openshift-cluster-node-tuning-operator/tuned-spc6q"
Sep 18 20:01:12.232964 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:12.232856    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/73fbf16b-3549-4a6c-a959-b633237ebd0c-host-slash\") pod \"ovnkube-node-rtw9n\" (UID: \"73fbf16b-3549-4a6c-a959-b633237ebd0c\") " pod="openshift-ovn-kubernetes/ovnkube-node-rtw9n"
Sep 18 20:01:12.232964 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:12.232881    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/73fbf16b-3549-4a6c-a959-b633237ebd0c-run-openvswitch\") pod \"ovnkube-node-rtw9n\" (UID: \"73fbf16b-3549-4a6c-a959-b633237ebd0c\") " pod="openshift-ovn-kubernetes/ovnkube-node-rtw9n"
Sep 18 20:01:12.232964 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:12.232909    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/bf9010d7-90b5-4f95-8dea-8af5839c0f87-cni-binary-copy\") pod \"multus-jbmm2\" (UID: \"bf9010d7-90b5-4f95-8dea-8af5839c0f87\") " pod="openshift-multus/multus-jbmm2"
Sep 18 20:01:12.232964 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:12.232956    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/6b3e5758-70d0-496d-90a9-35d371e8e1b6-etc-sysconfig\") pod \"tuned-spc6q\" (UID: \"6b3e5758-70d0-496d-90a9-35d371e8e1b6\") " pod="openshift-cluster-node-tuning-operator/tuned-spc6q"
Sep 18 20:01:12.233252 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:12.233013    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/73fbf16b-3549-4a6c-a959-b633237ebd0c-ovn-node-metrics-cert\") pod \"ovnkube-node-rtw9n\" (UID: \"73fbf16b-3549-4a6c-a959-b633237ebd0c\") " pod="openshift-ovn-kubernetes/ovnkube-node-rtw9n"
Sep 18 20:01:12.233252 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:12.233049    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/73fbf16b-3549-4a6c-a959-b633237ebd0c-ovnkube-config\") pod \"ovnkube-node-rtw9n\" (UID: \"73fbf16b-3549-4a6c-a959-b633237ebd0c\") " pod="openshift-ovn-kubernetes/ovnkube-node-rtw9n"
Sep 18 20:01:12.233252 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:12.233079    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/bf9010d7-90b5-4f95-8dea-8af5839c0f87-system-cni-dir\") pod \"multus-jbmm2\" (UID: \"bf9010d7-90b5-4f95-8dea-8af5839c0f87\") " pod="openshift-multus/multus-jbmm2"
Sep 18 20:01:12.233252 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:12.233108    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/5c8c76fd-1219-40d0-b010-cd89d714a3fe-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-pn4rk\" (UID: \"5c8c76fd-1219-40d0-b010-cd89d714a3fe\") " pod="openshift-multus/multus-additional-cni-plugins-pn4rk"
Sep 18 20:01:12.233439 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:12.233253    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/5c8c76fd-1219-40d0-b010-cd89d714a3fe-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-pn4rk\" (UID: \"5c8c76fd-1219-40d0-b010-cd89d714a3fe\") " pod="openshift-multus/multus-additional-cni-plugins-pn4rk"
Sep 18 20:01:12.233439 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:12.233290    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/73fbf16b-3549-4a6c-a959-b633237ebd0c-host-cni-bin\") pod \"ovnkube-node-rtw9n\" (UID: \"73fbf16b-3549-4a6c-a959-b633237ebd0c\") " pod="openshift-ovn-kubernetes/ovnkube-node-rtw9n"
Sep 18 20:01:12.233439 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:12.233315    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/bf9010d7-90b5-4f95-8dea-8af5839c0f87-cnibin\") pod \"multus-jbmm2\" (UID: \"bf9010d7-90b5-4f95-8dea-8af5839c0f87\") " pod="openshift-multus/multus-jbmm2"
Sep 18 20:01:12.233439 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:12.233346    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/bf9010d7-90b5-4f95-8dea-8af5839c0f87-multus-conf-dir\") pod \"multus-jbmm2\" (UID: \"bf9010d7-90b5-4f95-8dea-8af5839c0f87\") " pod="openshift-multus/multus-jbmm2"
Sep 18 20:01:12.233683 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:12.233662    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/bf9010d7-90b5-4f95-8dea-8af5839c0f87-host-var-lib-kubelet\") pod \"multus-jbmm2\" (UID: \"bf9010d7-90b5-4f95-8dea-8af5839c0f87\") " pod="openshift-multus/multus-jbmm2"
Sep 18 20:01:12.233746 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:12.233699    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qq7pb\" (UniqueName: \"kubernetes.io/projected/bf9010d7-90b5-4f95-8dea-8af5839c0f87-kube-api-access-qq7pb\") pod \"multus-jbmm2\" (UID: \"bf9010d7-90b5-4f95-8dea-8af5839c0f87\") " pod="openshift-multus/multus-jbmm2"
Sep 18 20:01:12.233746 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:12.233725    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/73fbf16b-3549-4a6c-a959-b633237ebd0c-ovnkube-script-lib\") pod \"ovnkube-node-rtw9n\" (UID: \"73fbf16b-3549-4a6c-a959-b633237ebd0c\") " pod="openshift-ovn-kubernetes/ovnkube-node-rtw9n"
Sep 18 20:01:12.233837 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:12.233754    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5c8c76fd-1219-40d0-b010-cd89d714a3fe-system-cni-dir\") pod \"multus-additional-cni-plugins-pn4rk\" (UID: \"5c8c76fd-1219-40d0-b010-cd89d714a3fe\") " pod="openshift-multus/multus-additional-cni-plugins-pn4rk"
Sep 18 20:01:12.233837 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:12.233790    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/73fbf16b-3549-4a6c-a959-b633237ebd0c-host-kubelet\") pod \"ovnkube-node-rtw9n\" (UID: \"73fbf16b-3549-4a6c-a959-b633237ebd0c\") " pod="openshift-ovn-kubernetes/ovnkube-node-rtw9n"
Sep 18 20:01:12.233837 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:12.233819    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/73fbf16b-3549-4a6c-a959-b633237ebd0c-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-rtw9n\" (UID: \"73fbf16b-3549-4a6c-a959-b633237ebd0c\") " pod="openshift-ovn-kubernetes/ovnkube-node-rtw9n"
Sep 18 20:01:12.233975 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:12.233850    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/bf9010d7-90b5-4f95-8dea-8af5839c0f87-multus-daemon-config\") pod \"multus-jbmm2\" (UID: \"bf9010d7-90b5-4f95-8dea-8af5839c0f87\") " pod="openshift-multus/multus-jbmm2"
Sep 18 20:01:12.234310 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:12.234284    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6b3e5758-70d0-496d-90a9-35d371e8e1b6-etc-kubernetes\") pod \"tuned-spc6q\" (UID: \"6b3e5758-70d0-496d-90a9-35d371e8e1b6\") " pod="openshift-cluster-node-tuning-operator/tuned-spc6q"
Sep 18 20:01:12.234419 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:12.234340    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/73fbf16b-3549-4a6c-a959-b633237ebd0c-run-ovn\") pod \"ovnkube-node-rtw9n\" (UID: \"73fbf16b-3549-4a6c-a959-b633237ebd0c\") " pod="openshift-ovn-kubernetes/ovnkube-node-rtw9n"
Sep 18 20:01:12.234419 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:12.234368    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/5c8c76fd-1219-40d0-b010-cd89d714a3fe-cnibin\") pod \"multus-additional-cni-plugins-pn4rk\" (UID: \"5c8c76fd-1219-40d0-b010-cd89d714a3fe\") " pod="openshift-multus/multus-additional-cni-plugins-pn4rk"
Sep 18 20:01:12.234419 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:12.234403    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/bf9010d7-90b5-4f95-8dea-8af5839c0f87-host-var-lib-cni-bin\") pod \"multus-jbmm2\" (UID: \"bf9010d7-90b5-4f95-8dea-8af5839c0f87\") " pod="openshift-multus/multus-jbmm2"
Sep 18 20:01:12.235006 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:12.234653    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nbw5k"
Sep 18 20:01:12.235367 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:12.235351    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-b8jbz"
Sep 18 20:01:12.237620 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:12.237193    2532 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Sep 18 20:01:12.237620 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:12.237237    2532 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-vp97k\""
Sep 18 20:01:12.237770 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:12.237654    2532 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Sep 18 20:01:12.238727 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:12.238086    2532 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Sep 18 20:01:12.238727 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:12.238215    2532 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Sep 18 20:01:12.238727 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:12.238287    2532 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-c52tj\""
Sep 18 20:01:12.238727 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:12.238528    2532 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Sep 18 20:01:12.239254 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:12.238971    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gd24n"
Sep 18 20:01:12.239254 ip-10-0-130-81 kubenswrapper[2532]: E0918 20:01:12.239044    2532 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gd24n" podUID="a966f963-b4ae-46ba-b206-2a27ad401463"
Sep 18 20:01:12.239393 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:12.239335    2532 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 18 20:01:12.242075 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:12.242055    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-dxcqj"
Sep 18 20:01:12.244331 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:12.244313    2532 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Sep 18 20:01:12.244530 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:12.244514    2532 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Sep 18 20:01:12.245011 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:12.244692    2532 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-nkq5k\""
Sep 18 20:01:12.245011 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:12.244885    2532 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Sep 18 20:01:12.261255 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:12.261237    2532 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-krfrs"
Sep 18 20:01:12.268036 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:12.268011    2532 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-krfrs"
Sep 18 20:01:12.304500 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:12.304445    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-81.ec2.internal" event={"ID":"ae7090a051a4a9229a0e9951ab9e2daf","Type":"ContainerStarted","Data":"3b1883c47cf5d28a65a23d3abc53b07a7fb46035bebe3cf4cb0e3385c970e75b"}
Sep 18 20:01:12.305631 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:12.305604    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-81.ec2.internal" event={"ID":"7719088cb6c9e3ddb918a0f82975ee3c","Type":"ContainerStarted","Data":"1ccff460ecbfebcf662a2145555ddfe713e870e3b7c984215400bb340561aae1"}
Sep 18 20:01:12.318936 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:12.318910    2532 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Sep 18 20:01:12.335060 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:12.335037    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/bf9010d7-90b5-4f95-8dea-8af5839c0f87-multus-cni-dir\") pod \"multus-jbmm2\" (UID: \"bf9010d7-90b5-4f95-8dea-8af5839c0f87\") " pod="openshift-multus/multus-jbmm2"
Sep 18 20:01:12.335213 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:12.335073    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-trvsb\" (UniqueName: \"kubernetes.io/projected/73fbf16b-3549-4a6c-a959-b633237ebd0c-kube-api-access-trvsb\") pod \"ovnkube-node-rtw9n\" (UID: \"73fbf16b-3549-4a6c-a959-b633237ebd0c\") " pod="openshift-ovn-kubernetes/ovnkube-node-rtw9n"
Sep 18 20:01:12.335213 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:12.335100    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/6b3e5758-70d0-496d-90a9-35d371e8e1b6-etc-sysctl-d\") pod \"tuned-spc6q\" (UID: \"6b3e5758-70d0-496d-90a9-35d371e8e1b6\") " pod="openshift-cluster-node-tuning-operator/tuned-spc6q"
Sep 18 20:01:12.335213 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:12.335119    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6b3e5758-70d0-496d-90a9-35d371e8e1b6-tmp\") pod \"tuned-spc6q\" (UID: \"6b3e5758-70d0-496d-90a9-35d371e8e1b6\") " pod="openshift-cluster-node-tuning-operator/tuned-spc6q"
Sep 18 20:01:12.335213 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:12.335141    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/73fbf16b-3549-4a6c-a959-b633237ebd0c-etc-openvswitch\") pod \"ovnkube-node-rtw9n\" (UID: \"73fbf16b-3549-4a6c-a959-b633237ebd0c\") " pod="openshift-ovn-kubernetes/ovnkube-node-rtw9n"
Sep 18 20:01:12.335213 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:12.335161    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/73fbf16b-3549-4a6c-a959-b633237ebd0c-host-run-ovn-kubernetes\") pod \"ovnkube-node-rtw9n\" (UID: \"73fbf16b-3549-4a6c-a959-b633237ebd0c\") " pod="openshift-ovn-kubernetes/ovnkube-node-rtw9n"
Sep 18 20:01:12.335213 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:12.335205    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/73fbf16b-3549-4a6c-a959-b633237ebd0c-env-overrides\") pod \"ovnkube-node-rtw9n\" (UID: \"73fbf16b-3549-4a6c-a959-b633237ebd0c\") " pod="openshift-ovn-kubernetes/ovnkube-node-rtw9n"
Sep 18 20:01:12.335534 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:12.335233    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-slbwq\" (UniqueName: \"kubernetes.io/projected/a966f963-b4ae-46ba-b206-2a27ad401463-kube-api-access-slbwq\") pod \"network-check-target-gd24n\" (UID: \"a966f963-b4ae-46ba-b206-2a27ad401463\") " pod="openshift-network-diagnostics/network-check-target-gd24n"
Sep 18 20:01:12.335534 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:12.335253    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/bf9010d7-90b5-4f95-8dea-8af5839c0f87-multus-cni-dir\") pod \"multus-jbmm2\" (UID: \"bf9010d7-90b5-4f95-8dea-8af5839c0f87\") " pod="openshift-multus/multus-jbmm2"
Sep 18 20:01:12.335534 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:12.335264    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/6b3e5758-70d0-496d-90a9-35d371e8e1b6-etc-sysctl-conf\") pod \"tuned-spc6q\" (UID: \"6b3e5758-70d0-496d-90a9-35d371e8e1b6\") " pod="openshift-cluster-node-tuning-operator/tuned-spc6q"
Sep 18 20:01:12.335534 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:12.335313    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6b3e5758-70d0-496d-90a9-35d371e8e1b6-var-lib-kubelet\") pod \"tuned-spc6q\" (UID: \"6b3e5758-70d0-496d-90a9-35d371e8e1b6\") " pod="openshift-cluster-node-tuning-operator/tuned-spc6q"
Sep 18 20:01:12.335534 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:12.335340    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/6b3e5758-70d0-496d-90a9-35d371e8e1b6-host\") pod \"tuned-spc6q\" (UID: \"6b3e5758-70d0-496d-90a9-35d371e8e1b6\") " pod="openshift-cluster-node-tuning-operator/tuned-spc6q"
Sep 18 20:01:12.335534 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:12.335367    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4h9pz\" (UniqueName: \"kubernetes.io/projected/5c8c76fd-1219-40d0-b010-cd89d714a3fe-kube-api-access-4h9pz\") pod \"multus-additional-cni-plugins-pn4rk\" (UID: \"5c8c76fd-1219-40d0-b010-cd89d714a3fe\") " pod="openshift-multus/multus-additional-cni-plugins-pn4rk"
Sep 18 20:01:12.335534 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:12.335397    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/6b3e5758-70d0-496d-90a9-35d371e8e1b6-etc-sysctl-conf\") pod \"tuned-spc6q\" (UID: \"6b3e5758-70d0-496d-90a9-35d371e8e1b6\") " pod="openshift-cluster-node-tuning-operator/tuned-spc6q"
Sep 18 20:01:12.335534 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:12.335406    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/bf9010d7-90b5-4f95-8dea-8af5839c0f87-host-run-netns\") pod \"multus-jbmm2\" (UID: \"bf9010d7-90b5-4f95-8dea-8af5839c0f87\") " pod="openshift-multus/multus-jbmm2"
Sep 18 20:01:12.335534 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:12.335431    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/6b3e5758-70d0-496d-90a9-35d371e8e1b6-host\") pod \"tuned-spc6q\" (UID: \"6b3e5758-70d0-496d-90a9-35d371e8e1b6\") " pod="openshift-cluster-node-tuning-operator/tuned-spc6q"
Sep 18 20:01:12.335534 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:12.335439    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/bf9010d7-90b5-4f95-8dea-8af5839c0f87-host-run-multus-certs\") pod \"multus-jbmm2\" (UID: \"bf9010d7-90b5-4f95-8dea-8af5839c0f87\") " pod="openshift-multus/multus-jbmm2"
Sep 18 20:01:12.335534 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:12.335446    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/bf9010d7-90b5-4f95-8dea-8af5839c0f87-host-run-netns\") pod \"multus-jbmm2\" (UID: \"bf9010d7-90b5-4f95-8dea-8af5839c0f87\") " pod="openshift-multus/multus-jbmm2"
Sep 18 20:01:12.335534 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:12.335467    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/96167e3f-fb44-48e0-b86e-419a4a3b117c-metrics-certs\") pod \"network-metrics-daemon-xx7f7\" (UID: \"96167e3f-fb44-48e0-b86e-419a4a3b117c\") " pod="openshift-multus/network-metrics-daemon-xx7f7"
Sep 18 20:01:12.336126 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:12.335548    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/73fbf16b-3549-4a6c-a959-b633237ebd0c-host-run-ovn-kubernetes\") pod \"ovnkube-node-rtw9n\" (UID: \"73fbf16b-3549-4a6c-a959-b633237ebd0c\") " pod="openshift-ovn-kubernetes/ovnkube-node-rtw9n"
Sep 18 20:01:12.336126 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:12.335553    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6b3e5758-70d0-496d-90a9-35d371e8e1b6-var-lib-kubelet\") pod \"tuned-spc6q\" (UID: \"6b3e5758-70d0-496d-90a9-35d371e8e1b6\") " pod="openshift-cluster-node-tuning-operator/tuned-spc6q"
Sep 18 20:01:12.336126 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:12.335580    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/6b3e5758-70d0-496d-90a9-35d371e8e1b6-run\") pod \"tuned-spc6q\" (UID: \"6b3e5758-70d0-496d-90a9-35d371e8e1b6\") " pod="openshift-cluster-node-tuning-operator/tuned-spc6q"
Sep 18 20:01:12.336126 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:12.335606    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/73fbf16b-3549-4a6c-a959-b633237ebd0c-host-slash\") pod \"ovnkube-node-rtw9n\" (UID: \"73fbf16b-3549-4a6c-a959-b633237ebd0c\") " pod="openshift-ovn-kubernetes/ovnkube-node-rtw9n"
Sep 18 20:01:12.336126 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:12.335640    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/6b3e5758-70d0-496d-90a9-35d371e8e1b6-etc-sysctl-d\") pod \"tuned-spc6q\" (UID: \"6b3e5758-70d0-496d-90a9-35d371e8e1b6\") " pod="openshift-cluster-node-tuning-operator/tuned-spc6q"
Sep 18 20:01:12.336126 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:12.335653    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/73fbf16b-3549-4a6c-a959-b633237ebd0c-run-openvswitch\") pod \"ovnkube-node-rtw9n\" (UID: \"73fbf16b-3549-4a6c-a959-b633237ebd0c\") " pod="openshift-ovn-kubernetes/ovnkube-node-rtw9n"
Sep 18 20:01:12.336126 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:12.335683    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/bf9010d7-90b5-4f95-8dea-8af5839c0f87-cni-binary-copy\") pod \"multus-jbmm2\" (UID: \"bf9010d7-90b5-4f95-8dea-8af5839c0f87\") " pod="openshift-multus/multus-jbmm2"
Sep 18 20:01:12.336126 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:12.335715    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/d94756ff-5c71-43bc-ab39-b0fde15a36a2-agent-certs\") pod \"konnectivity-agent-g749g\" (UID: \"d94756ff-5c71-43bc-ab39-b0fde15a36a2\") " pod="kube-system/konnectivity-agent-g749g"
Sep 18 20:01:12.336126 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:12.335776    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/d94756ff-5c71-43bc-ab39-b0fde15a36a2-konnectivity-ca\") pod \"konnectivity-agent-g749g\" (UID: \"d94756ff-5c71-43bc-ab39-b0fde15a36a2\") " pod="kube-system/konnectivity-agent-g749g"
Sep 18 20:01:12.336126 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:12.335937    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/7f063a5b-9649-46fc-8278-3f8bb49700ed-device-dir\") pod \"aws-ebs-csi-driver-node-nbw5k\" (UID: \"7f063a5b-9649-46fc-8278-3f8bb49700ed\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nbw5k"
Sep 18 20:01:12.336126 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:12.335957    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/73fbf16b-3549-4a6c-a959-b633237ebd0c-run-openvswitch\") pod \"ovnkube-node-rtw9n\" (UID: \"73fbf16b-3549-4a6c-a959-b633237ebd0c\") " pod="openshift-ovn-kubernetes/ovnkube-node-rtw9n"
Sep 18 20:01:12.336126 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:12.335986    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h84rm\" (UniqueName: \"kubernetes.io/projected/7f063a5b-9649-46fc-8278-3f8bb49700ed-kube-api-access-h84rm\") pod \"aws-ebs-csi-driver-node-nbw5k\" (UID: \"7f063a5b-9649-46fc-8278-3f8bb49700ed\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nbw5k"
Sep 18 20:01:12.336126 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:12.336014    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/6b3e5758-70d0-496d-90a9-35d371e8e1b6-etc-sysconfig\") pod \"tuned-spc6q\" (UID: \"6b3e5758-70d0-496d-90a9-35d371e8e1b6\") " pod="openshift-cluster-node-tuning-operator/tuned-spc6q"
Sep 18 20:01:12.336126 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:12.336016    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/6b3e5758-70d0-496d-90a9-35d371e8e1b6-run\") pod \"tuned-spc6q\" (UID: \"6b3e5758-70d0-496d-90a9-35d371e8e1b6\") " pod="openshift-cluster-node-tuning-operator/tuned-spc6q"
Sep 18 20:01:12.336126 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:12.336038    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/73fbf16b-3549-4a6c-a959-b633237ebd0c-ovn-node-metrics-cert\") pod \"ovnkube-node-rtw9n\" (UID: \"73fbf16b-3549-4a6c-a959-b633237ebd0c\") " pod="openshift-ovn-kubernetes/ovnkube-node-rtw9n"
Sep 18 20:01:12.336126 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:12.336051    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/73fbf16b-3549-4a6c-a959-b633237ebd0c-host-slash\") pod \"ovnkube-node-rtw9n\" (UID: \"73fbf16b-3549-4a6c-a959-b633237ebd0c\") " pod="openshift-ovn-kubernetes/ovnkube-node-rtw9n"
Sep 18 20:01:12.336126 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:12.336054    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/73fbf16b-3549-4a6c-a959-b633237ebd0c-env-overrides\") pod \"ovnkube-node-rtw9n\" (UID: \"73fbf16b-3549-4a6c-a959-b633237ebd0c\") " pod="openshift-ovn-kubernetes/ovnkube-node-rtw9n"
Sep 18 20:01:12.336992 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:12.336123    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/bf9010d7-90b5-4f95-8dea-8af5839c0f87-host-run-multus-certs\") pod \"multus-jbmm2\" (UID: \"bf9010d7-90b5-4f95-8dea-8af5839c0f87\") " pod="openshift-multus/multus-jbmm2"
Sep 18 20:01:12.336992 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:12.336145    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/6b3e5758-70d0-496d-90a9-35d371e8e1b6-etc-sysconfig\") pod \"tuned-spc6q\" (UID: \"6b3e5758-70d0-496d-90a9-35d371e8e1b6\") " pod="openshift-cluster-node-tuning-operator/tuned-spc6q"
Sep 18 20:01:12.336992 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:12.336130    2532 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Sep 18 20:01:12.336992 ip-10-0-130-81 kubenswrapper[2532]: E0918 20:01:12.336247    2532 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 18 20:01:12.336992 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:12.336253    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/7f063a5b-9649-46fc-8278-3f8bb49700ed-sys-fs\") pod \"aws-ebs-csi-driver-node-nbw5k\" (UID: \"7f063a5b-9649-46fc-8278-3f8bb49700ed\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nbw5k"
Sep 18 20:01:12.336992 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:12.336306    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/73fbf16b-3549-4a6c-a959-b633237ebd0c-etc-openvswitch\") pod \"ovnkube-node-rtw9n\" (UID: \"73fbf16b-3549-4a6c-a959-b633237ebd0c\") " pod="openshift-ovn-kubernetes/ovnkube-node-rtw9n"
Sep 18 20:01:12.336992 ip-10-0-130-81 kubenswrapper[2532]: E0918 20:01:12.336312    2532 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/96167e3f-fb44-48e0-b86e-419a4a3b117c-metrics-certs podName:96167e3f-fb44-48e0-b86e-419a4a3b117c nodeName:}" failed. No retries permitted until 2026-09-18 20:01:12.836298137 +0000 UTC m=+4.601687514 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/96167e3f-fb44-48e0-b86e-419a4a3b117c-metrics-certs") pod "network-metrics-daemon-xx7f7" (UID: "96167e3f-fb44-48e0-b86e-419a4a3b117c") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 18 20:01:12.336992 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:12.336358    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/87c4bfcc-4ad2-4166-b5c9-1051bdd2717d-hosts-file\") pod \"node-resolver-b8jbz\" (UID: \"87c4bfcc-4ad2-4166-b5c9-1051bdd2717d\") " pod="openshift-dns/node-resolver-b8jbz"
Sep 18 20:01:12.336992 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:12.336411    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/73fbf16b-3549-4a6c-a959-b633237ebd0c-ovnkube-config\") pod \"ovnkube-node-rtw9n\" (UID: \"73fbf16b-3549-4a6c-a959-b633237ebd0c\") " pod="openshift-ovn-kubernetes/ovnkube-node-rtw9n"
Sep 18 20:01:12.336992 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:12.336437    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/bf9010d7-90b5-4f95-8dea-8af5839c0f87-system-cni-dir\") pod \"multus-jbmm2\" (UID: \"bf9010d7-90b5-4f95-8dea-8af5839c0f87\") " pod="openshift-multus/multus-jbmm2"
Sep 18 20:01:12.336992 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:12.336511    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/4dc79459-39e4-4afe-b537-b657780854fb-host-slash\") pod \"iptables-alerter-dxcqj\" (UID: \"4dc79459-39e4-4afe-b537-b657780854fb\") " pod="openshift-network-operator/iptables-alerter-dxcqj"
Sep 18 20:01:12.336992 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:12.336543    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/5c8c76fd-1219-40d0-b010-cd89d714a3fe-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-pn4rk\" (UID: \"5c8c76fd-1219-40d0-b010-cd89d714a3fe\") " pod="openshift-multus/multus-additional-cni-plugins-pn4rk"
Sep 18 20:01:12.336992 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:12.336568    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/bf9010d7-90b5-4f95-8dea-8af5839c0f87-system-cni-dir\") pod \"multus-jbmm2\" (UID: \"bf9010d7-90b5-4f95-8dea-8af5839c0f87\") " pod="openshift-multus/multus-jbmm2"
Sep 18 20:01:12.336992 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:12.336601    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/5c8c76fd-1219-40d0-b010-cd89d714a3fe-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-pn4rk\" (UID: \"5c8c76fd-1219-40d0-b010-cd89d714a3fe\") " pod="openshift-multus/multus-additional-cni-plugins-pn4rk"
Sep 18 20:01:12.336992 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:12.336626    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b771ff0e-edf5-4d81-adf7-6350fe1ea781-host\") pod \"node-ca-wkwnc\" (UID: \"b771ff0e-edf5-4d81-adf7-6350fe1ea781\") " pod="openshift-image-registry/node-ca-wkwnc"
Sep 18 20:01:12.336992 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:12.336817    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/bf9010d7-90b5-4f95-8dea-8af5839c0f87-cni-binary-copy\") pod \"multus-jbmm2\" (UID: \"bf9010d7-90b5-4f95-8dea-8af5839c0f87\") " pod="openshift-multus/multus-jbmm2"
Sep 18 20:01:12.336992 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:12.336879    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/4dc79459-39e4-4afe-b537-b657780854fb-iptables-alerter-script\") pod \"iptables-alerter-dxcqj\" (UID: \"4dc79459-39e4-4afe-b537-b657780854fb\") " pod="openshift-network-operator/iptables-alerter-dxcqj"
Sep 18 20:01:12.337833 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:12.336910    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hvm5x\" (UniqueName: \"kubernetes.io/projected/4dc79459-39e4-4afe-b537-b657780854fb-kube-api-access-hvm5x\") pod \"iptables-alerter-dxcqj\" (UID: \"4dc79459-39e4-4afe-b537-b657780854fb\") " pod="openshift-network-operator/iptables-alerter-dxcqj"
Sep 18 20:01:12.337833 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:12.336935    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/73fbf16b-3549-4a6c-a959-b633237ebd0c-host-cni-bin\") pod \"ovnkube-node-rtw9n\" (UID: \"73fbf16b-3549-4a6c-a959-b633237ebd0c\") " pod="openshift-ovn-kubernetes/ovnkube-node-rtw9n"
Sep 18 20:01:12.337833 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:12.336960    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/bf9010d7-90b5-4f95-8dea-8af5839c0f87-cnibin\") pod \"multus-jbmm2\" (UID: \"bf9010d7-90b5-4f95-8dea-8af5839c0f87\") " pod="openshift-multus/multus-jbmm2"
Sep 18 20:01:12.337833 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:12.337020    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/bf9010d7-90b5-4f95-8dea-8af5839c0f87-multus-conf-dir\") pod \"multus-jbmm2\" (UID: \"bf9010d7-90b5-4f95-8dea-8af5839c0f87\") " pod="openshift-multus/multus-jbmm2"
Sep 18 20:01:12.337833 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:12.337047    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qr7vd\" (UniqueName: \"kubernetes.io/projected/b771ff0e-edf5-4d81-adf7-6350fe1ea781-kube-api-access-qr7vd\") pod \"node-ca-wkwnc\" (UID: \"b771ff0e-edf5-4d81-adf7-6350fe1ea781\") " pod="openshift-image-registry/node-ca-wkwnc"
Sep 18 20:01:12.337833 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:12.337075    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/bf9010d7-90b5-4f95-8dea-8af5839c0f87-host-var-lib-kubelet\") pod \"multus-jbmm2\" (UID: \"bf9010d7-90b5-4f95-8dea-8af5839c0f87\") " pod="openshift-multus/multus-jbmm2"
Sep 18 20:01:12.337833 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:12.337102    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qq7pb\" (UniqueName: \"kubernetes.io/projected/bf9010d7-90b5-4f95-8dea-8af5839c0f87-kube-api-access-qq7pb\") pod \"multus-jbmm2\" (UID: \"bf9010d7-90b5-4f95-8dea-8af5839c0f87\") " pod="openshift-multus/multus-jbmm2"
Sep 18 20:01:12.337833 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:12.337096    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/5c8c76fd-1219-40d0-b010-cd89d714a3fe-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-pn4rk\" (UID: \"5c8c76fd-1219-40d0-b010-cd89d714a3fe\") " pod="openshift-multus/multus-additional-cni-plugins-pn4rk"
Sep 18 20:01:12.337833 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:12.337114    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/73fbf16b-3549-4a6c-a959-b633237ebd0c-ovnkube-config\") pod \"ovnkube-node-rtw9n\" (UID: \"73fbf16b-3549-4a6c-a959-b633237ebd0c\") " pod="openshift-ovn-kubernetes/ovnkube-node-rtw9n"
Sep 18 20:01:12.337833 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:12.337129    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/b771ff0e-edf5-4d81-adf7-6350fe1ea781-serviceca\") pod \"node-ca-wkwnc\" (UID: \"b771ff0e-edf5-4d81-adf7-6350fe1ea781\") " pod="openshift-image-registry/node-ca-wkwnc"
Sep 18 20:01:12.337833 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:12.337155    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/b377a9e6-dbbb-4023-8997-70178a63034f-dbus\") pod \"global-pull-secret-syncer-gd9qz\" (UID: \"b377a9e6-dbbb-4023-8997-70178a63034f\") " pod="kube-system/global-pull-secret-syncer-gd9qz"
Sep 18 20:01:12.337833 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:12.337175    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/bf9010d7-90b5-4f95-8dea-8af5839c0f87-cnibin\") pod \"multus-jbmm2\" (UID: \"bf9010d7-90b5-4f95-8dea-8af5839c0f87\") " pod="openshift-multus/multus-jbmm2"
Sep 18 20:01:12.337833 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:12.337240    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/73fbf16b-3549-4a6c-a959-b633237ebd0c-host-cni-bin\") pod \"ovnkube-node-rtw9n\" (UID: \"73fbf16b-3549-4a6c-a959-b633237ebd0c\") " pod="openshift-ovn-kubernetes/ovnkube-node-rtw9n"
Sep 18 20:01:12.337833 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:12.337276    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/7f063a5b-9649-46fc-8278-3f8bb49700ed-kubelet-dir\") pod \"aws-ebs-csi-driver-node-nbw5k\" (UID: \"7f063a5b-9649-46fc-8278-3f8bb49700ed\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nbw5k"
Sep 18 20:01:12.337833 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:12.337278    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/bf9010d7-90b5-4f95-8dea-8af5839c0f87-host-var-lib-kubelet\") pod \"multus-jbmm2\" (UID: \"bf9010d7-90b5-4f95-8dea-8af5839c0f87\") " pod="openshift-multus/multus-jbmm2"
Sep 18 20:01:12.337833 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:12.337304    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/73fbf16b-3549-4a6c-a959-b633237ebd0c-ovnkube-script-lib\") pod \"ovnkube-node-rtw9n\" (UID: \"73fbf16b-3549-4a6c-a959-b633237ebd0c\") " pod="openshift-ovn-kubernetes/ovnkube-node-rtw9n"
Sep 18 20:01:12.337833 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:12.337318    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/bf9010d7-90b5-4f95-8dea-8af5839c0f87-multus-conf-dir\") pod \"multus-jbmm2\" (UID: \"bf9010d7-90b5-4f95-8dea-8af5839c0f87\") " pod="openshift-multus/multus-jbmm2"
Sep 18 20:01:12.338707 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:12.337329    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5c8c76fd-1219-40d0-b010-cd89d714a3fe-system-cni-dir\") pod \"multus-additional-cni-plugins-pn4rk\" (UID: \"5c8c76fd-1219-40d0-b010-cd89d714a3fe\") " pod="openshift-multus/multus-additional-cni-plugins-pn4rk"
Sep 18 20:01:12.338707 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:12.337369    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/73fbf16b-3549-4a6c-a959-b633237ebd0c-host-kubelet\") pod \"ovnkube-node-rtw9n\" (UID: \"73fbf16b-3549-4a6c-a959-b633237ebd0c\") " pod="openshift-ovn-kubernetes/ovnkube-node-rtw9n"
Sep 18 20:01:12.338707 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:12.337397    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/73fbf16b-3549-4a6c-a959-b633237ebd0c-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-rtw9n\" (UID: \"73fbf16b-3549-4a6c-a959-b633237ebd0c\") " pod="openshift-ovn-kubernetes/ovnkube-node-rtw9n"
Sep 18 20:01:12.338707 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:12.337470    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/bf9010d7-90b5-4f95-8dea-8af5839c0f87-multus-daemon-config\") pod \"multus-jbmm2\" (UID: \"bf9010d7-90b5-4f95-8dea-8af5839c0f87\") " pod="openshift-multus/multus-jbmm2"
Sep 18 20:01:12.338707 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:12.337524    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6b3e5758-70d0-496d-90a9-35d371e8e1b6-etc-kubernetes\") pod \"tuned-spc6q\" (UID: \"6b3e5758-70d0-496d-90a9-35d371e8e1b6\") " pod="openshift-cluster-node-tuning-operator/tuned-spc6q"
Sep 18 20:01:12.338707 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:12.337573    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/73fbf16b-3549-4a6c-a959-b633237ebd0c-run-ovn\") pod \"ovnkube-node-rtw9n\" (UID: \"73fbf16b-3549-4a6c-a959-b633237ebd0c\") " pod="openshift-ovn-kubernetes/ovnkube-node-rtw9n"
Sep 18 20:01:12.338707 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:12.337597    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/5c8c76fd-1219-40d0-b010-cd89d714a3fe-cnibin\") pod \"multus-additional-cni-plugins-pn4rk\" (UID: \"5c8c76fd-1219-40d0-b010-cd89d714a3fe\") " pod="openshift-multus/multus-additional-cni-plugins-pn4rk"
Sep 18 20:01:12.338707 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:12.337680    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/bf9010d7-90b5-4f95-8dea-8af5839c0f87-host-var-lib-cni-bin\") pod \"multus-jbmm2\" (UID: \"bf9010d7-90b5-4f95-8dea-8af5839c0f87\") " pod="openshift-multus/multus-jbmm2"
Sep 18 20:01:12.338707 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:12.337740    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/7f063a5b-9649-46fc-8278-3f8bb49700ed-socket-dir\") pod \"aws-ebs-csi-driver-node-nbw5k\" (UID: \"7f063a5b-9649-46fc-8278-3f8bb49700ed\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nbw5k"
Sep 18 20:01:12.338707 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:12.337767    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b377a9e6-dbbb-4023-8997-70178a63034f-original-pull-secret\") pod \"global-pull-secret-syncer-gd9qz\" (UID: \"b377a9e6-dbbb-4023-8997-70178a63034f\") " pod="kube-system/global-pull-secret-syncer-gd9qz"
Sep 18 20:01:12.338707 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:12.337793    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/73fbf16b-3549-4a6c-a959-b633237ebd0c-systemd-units\") pod \"ovnkube-node-rtw9n\" (UID: \"73fbf16b-3549-4a6c-a959-b633237ebd0c\") " pod="openshift-ovn-kubernetes/ovnkube-node-rtw9n"
Sep 18 20:01:12.338707 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:12.337819    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/73fbf16b-3549-4a6c-a959-b633237ebd0c-var-lib-openvswitch\") pod \"ovnkube-node-rtw9n\" (UID: \"73fbf16b-3549-4a6c-a959-b633237ebd0c\") " pod="openshift-ovn-kubernetes/ovnkube-node-rtw9n"
Sep 18 20:01:12.338707 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:12.337856    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/5c8c76fd-1219-40d0-b010-cd89d714a3fe-cni-binary-copy\") pod \"multus-additional-cni-plugins-pn4rk\" (UID: \"5c8c76fd-1219-40d0-b010-cd89d714a3fe\") " pod="openshift-multus/multus-additional-cni-plugins-pn4rk"
Sep 18 20:01:12.338707 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:12.337881    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/bf9010d7-90b5-4f95-8dea-8af5839c0f87-hostroot\") pod \"multus-jbmm2\" (UID: \"bf9010d7-90b5-4f95-8dea-8af5839c0f87\") " pod="openshift-multus/multus-jbmm2"
Sep 18 20:01:12.338707 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:12.337892    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/5c8c76fd-1219-40d0-b010-cd89d714a3fe-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-pn4rk\" (UID: \"5c8c76fd-1219-40d0-b010-cd89d714a3fe\") " pod="openshift-multus/multus-additional-cni-plugins-pn4rk"
Sep 18 20:01:12.338707 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:12.337907    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bf9010d7-90b5-4f95-8dea-8af5839c0f87-etc-kubernetes\") pod \"multus-jbmm2\" (UID: \"bf9010d7-90b5-4f95-8dea-8af5839c0f87\") " pod="openshift-multus/multus-jbmm2"
Sep 18 20:01:12.338707 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:12.338037    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/73fbf16b-3549-4a6c-a959-b633237ebd0c-ovnkube-script-lib\") pod \"ovnkube-node-rtw9n\" (UID: \"73fbf16b-3549-4a6c-a959-b633237ebd0c\") " pod="openshift-ovn-kubernetes/ovnkube-node-rtw9n"
Sep 18 20:01:12.339505 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:12.338127    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/73fbf16b-3549-4a6c-a959-b633237ebd0c-run-ovn\") pod \"ovnkube-node-rtw9n\" (UID: \"73fbf16b-3549-4a6c-a959-b633237ebd0c\") " pod="openshift-ovn-kubernetes/ovnkube-node-rtw9n"
Sep 18 20:01:12.339505 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:12.338124    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xnvlz\" (UniqueName: \"kubernetes.io/projected/96167e3f-fb44-48e0-b86e-419a4a3b117c-kube-api-access-xnvlz\") pod \"network-metrics-daemon-xx7f7\" (UID: \"96167e3f-fb44-48e0-b86e-419a4a3b117c\") " pod="openshift-multus/network-metrics-daemon-xx7f7"
Sep 18 20:01:12.339505 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:12.338161    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bf9010d7-90b5-4f95-8dea-8af5839c0f87-etc-kubernetes\") pod \"multus-jbmm2\" (UID: \"bf9010d7-90b5-4f95-8dea-8af5839c0f87\") " pod="openshift-multus/multus-jbmm2"
Sep 18 20:01:12.339505 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:12.338172    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6b3e5758-70d0-496d-90a9-35d371e8e1b6-etc-kubernetes\") pod \"tuned-spc6q\" (UID: \"6b3e5758-70d0-496d-90a9-35d371e8e1b6\") " pod="openshift-cluster-node-tuning-operator/tuned-spc6q"
Sep 18 20:01:12.339505 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:12.338182    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/b377a9e6-dbbb-4023-8997-70178a63034f-kubelet-config\") pod \"global-pull-secret-syncer-gd9qz\" (UID: \"b377a9e6-dbbb-4023-8997-70178a63034f\") " pod="kube-system/global-pull-secret-syncer-gd9qz"
Sep 18 20:01:12.339505 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:12.338227    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/7f063a5b-9649-46fc-8278-3f8bb49700ed-registration-dir\") pod \"aws-ebs-csi-driver-node-nbw5k\" (UID: \"7f063a5b-9649-46fc-8278-3f8bb49700ed\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nbw5k"
Sep 18 20:01:12.339505 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:12.338267    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5c8c76fd-1219-40d0-b010-cd89d714a3fe-system-cni-dir\") pod \"multus-additional-cni-plugins-pn4rk\" (UID: \"5c8c76fd-1219-40d0-b010-cd89d714a3fe\") " pod="openshift-multus/multus-additional-cni-plugins-pn4rk"
Sep 18 20:01:12.339505 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:12.338275    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/73fbf16b-3549-4a6c-a959-b633237ebd0c-node-log\") pod \"ovnkube-node-rtw9n\" (UID: \"73fbf16b-3549-4a6c-a959-b633237ebd0c\") " pod="openshift-ovn-kubernetes/ovnkube-node-rtw9n"
Sep 18 20:01:12.339505 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:12.338315    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/6b3e5758-70d0-496d-90a9-35d371e8e1b6-etc-modprobe-d\") pod \"tuned-spc6q\" (UID: \"6b3e5758-70d0-496d-90a9-35d371e8e1b6\") " pod="openshift-cluster-node-tuning-operator/tuned-spc6q"
Sep 18 20:01:12.339505 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:12.338341    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6b3e5758-70d0-496d-90a9-35d371e8e1b6-sys\") pod \"tuned-spc6q\" (UID: \"6b3e5758-70d0-496d-90a9-35d371e8e1b6\") " pod="openshift-cluster-node-tuning-operator/tuned-spc6q"
Sep 18 20:01:12.339505 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:12.338382    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/73fbf16b-3549-4a6c-a959-b633237ebd0c-run-systemd\") pod \"ovnkube-node-rtw9n\" (UID: \"73fbf16b-3549-4a6c-a959-b633237ebd0c\") " pod="openshift-ovn-kubernetes/ovnkube-node-rtw9n"
Sep 18 20:01:12.339505 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:12.338448    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/73fbf16b-3549-4a6c-a959-b633237ebd0c-host-cni-netd\") pod \"ovnkube-node-rtw9n\" (UID: \"73fbf16b-3549-4a6c-a959-b633237ebd0c\") " pod="openshift-ovn-kubernetes/ovnkube-node-rtw9n"
Sep 18 20:01:12.339505 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:12.338475    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/bf9010d7-90b5-4f95-8dea-8af5839c0f87-host-run-k8s-cni-cncf-io\") pod \"multus-jbmm2\" (UID: \"bf9010d7-90b5-4f95-8dea-8af5839c0f87\") " pod="openshift-multus/multus-jbmm2"
Sep 18 20:01:12.339505 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:12.338315    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/73fbf16b-3549-4a6c-a959-b633237ebd0c-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-rtw9n\" (UID: \"73fbf16b-3549-4a6c-a959-b633237ebd0c\") " pod="openshift-ovn-kubernetes/ovnkube-node-rtw9n"
Sep 18 20:01:12.339505 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:12.338639    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/bf9010d7-90b5-4f95-8dea-8af5839c0f87-host-var-lib-cni-multus\") pod \"multus-jbmm2\" (UID: \"bf9010d7-90b5-4f95-8dea-8af5839c0f87\") " pod="openshift-multus/multus-jbmm2"
Sep 18 20:01:12.339505 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:12.338677    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/7f063a5b-9649-46fc-8278-3f8bb49700ed-etc-selinux\") pod \"aws-ebs-csi-driver-node-nbw5k\" (UID: \"7f063a5b-9649-46fc-8278-3f8bb49700ed\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nbw5k"
Sep 18 20:01:12.339505 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:12.338706    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/73fbf16b-3549-4a6c-a959-b633237ebd0c-host-run-netns\") pod \"ovnkube-node-rtw9n\" (UID: \"73fbf16b-3549-4a6c-a959-b633237ebd0c\") " pod="openshift-ovn-kubernetes/ovnkube-node-rtw9n"
Sep 18 20:01:12.340258 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:12.338701    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6b3e5758-70d0-496d-90a9-35d371e8e1b6-sys\") pod \"tuned-spc6q\" (UID: \"6b3e5758-70d0-496d-90a9-35d371e8e1b6\") " pod="openshift-cluster-node-tuning-operator/tuned-spc6q"
Sep 18 20:01:12.340258 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:12.338711    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/bf9010d7-90b5-4f95-8dea-8af5839c0f87-multus-daemon-config\") pod \"multus-jbmm2\" (UID: \"bf9010d7-90b5-4f95-8dea-8af5839c0f87\") " pod="openshift-multus/multus-jbmm2"
Sep 18 20:01:12.340258 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:12.338754    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/bf9010d7-90b5-4f95-8dea-8af5839c0f87-host-run-k8s-cni-cncf-io\") pod \"multus-jbmm2\" (UID: \"bf9010d7-90b5-4f95-8dea-8af5839c0f87\") " pod="openshift-multus/multus-jbmm2"
Sep 18 20:01:12.340258 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:12.338761    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/73fbf16b-3549-4a6c-a959-b633237ebd0c-systemd-units\") pod \"ovnkube-node-rtw9n\" (UID: \"73fbf16b-3549-4a6c-a959-b633237ebd0c\") " pod="openshift-ovn-kubernetes/ovnkube-node-rtw9n"
Sep 18 20:01:12.340258 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:12.338768    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/73fbf16b-3549-4a6c-a959-b633237ebd0c-node-log\") pod \"ovnkube-node-rtw9n\" (UID: \"73fbf16b-3549-4a6c-a959-b633237ebd0c\") " pod="openshift-ovn-kubernetes/ovnkube-node-rtw9n"
Sep 18 20:01:12.340258 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:12.338806    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/bf9010d7-90b5-4f95-8dea-8af5839c0f87-multus-socket-dir-parent\") pod \"multus-jbmm2\" (UID: \"bf9010d7-90b5-4f95-8dea-8af5839c0f87\") " pod="openshift-multus/multus-jbmm2"
Sep 18 20:01:12.340258 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:12.338808    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/bf9010d7-90b5-4f95-8dea-8af5839c0f87-host-var-lib-cni-bin\") pod \"multus-jbmm2\" (UID: \"bf9010d7-90b5-4f95-8dea-8af5839c0f87\") " pod="openshift-multus/multus-jbmm2"
Sep 18 20:01:12.340258 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:12.338849    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/6b3e5758-70d0-496d-90a9-35d371e8e1b6-etc-modprobe-d\") pod \"tuned-spc6q\" (UID: \"6b3e5758-70d0-496d-90a9-35d371e8e1b6\") " pod="openshift-cluster-node-tuning-operator/tuned-spc6q"
Sep 18 20:01:12.340258 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:12.338844    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/87c4bfcc-4ad2-4166-b5c9-1051bdd2717d-tmp-dir\") pod \"node-resolver-b8jbz\" (UID: \"87c4bfcc-4ad2-4166-b5c9-1051bdd2717d\") " pod="openshift-dns/node-resolver-b8jbz"
Sep 18 20:01:12.340258 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:12.338855    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/bf9010d7-90b5-4f95-8dea-8af5839c0f87-hostroot\") pod \"multus-jbmm2\" (UID: \"bf9010d7-90b5-4f95-8dea-8af5839c0f87\") " pod="openshift-multus/multus-jbmm2"
Sep 18 20:01:12.340258 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:12.338934    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/bf9010d7-90b5-4f95-8dea-8af5839c0f87-multus-socket-dir-parent\") pod \"multus-jbmm2\" (UID: \"bf9010d7-90b5-4f95-8dea-8af5839c0f87\") " pod="openshift-multus/multus-jbmm2"
Sep 18 20:01:12.340258 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:12.338964    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/6b3e5758-70d0-496d-90a9-35d371e8e1b6-etc-systemd\") pod \"tuned-spc6q\" (UID: \"6b3e5758-70d0-496d-90a9-35d371e8e1b6\") " pod="openshift-cluster-node-tuning-operator/tuned-spc6q"
Sep 18 20:01:12.340258 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:12.338976    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/73fbf16b-3549-4a6c-a959-b633237ebd0c-var-lib-openvswitch\") pod \"ovnkube-node-rtw9n\" (UID: \"73fbf16b-3549-4a6c-a959-b633237ebd0c\") " pod="openshift-ovn-kubernetes/ovnkube-node-rtw9n"
Sep 18 20:01:12.340258 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:12.338977    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/5c8c76fd-1219-40d0-b010-cd89d714a3fe-cnibin\") pod \"multus-additional-cni-plugins-pn4rk\" (UID: \"5c8c76fd-1219-40d0-b010-cd89d714a3fe\") " pod="openshift-multus/multus-additional-cni-plugins-pn4rk"
Sep 18 20:01:12.340258 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:12.339016    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/73fbf16b-3549-4a6c-a959-b633237ebd0c-host-cni-netd\") pod \"ovnkube-node-rtw9n\" (UID: \"73fbf16b-3549-4a6c-a959-b633237ebd0c\") " pod="openshift-ovn-kubernetes/ovnkube-node-rtw9n"
Sep 18 20:01:12.340258 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:12.339027    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/6b3e5758-70d0-496d-90a9-35d371e8e1b6-etc-systemd\") pod \"tuned-spc6q\" (UID: \"6b3e5758-70d0-496d-90a9-35d371e8e1b6\") " pod="openshift-cluster-node-tuning-operator/tuned-spc6q"
Sep 18 20:01:12.340258 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:12.339036    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/73fbf16b-3549-4a6c-a959-b633237ebd0c-host-kubelet\") pod \"ovnkube-node-rtw9n\" (UID: \"73fbf16b-3549-4a6c-a959-b633237ebd0c\") " pod="openshift-ovn-kubernetes/ovnkube-node-rtw9n"
Sep 18 20:01:12.340258 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:12.339047    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/6b3e5758-70d0-496d-90a9-35d371e8e1b6-lib-modules\") pod \"tuned-spc6q\" (UID: \"6b3e5758-70d0-496d-90a9-35d371e8e1b6\") " pod="openshift-cluster-node-tuning-operator/tuned-spc6q"
Sep 18 20:01:12.340964 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:12.339067    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/73fbf16b-3549-4a6c-a959-b633237ebd0c-run-systemd\") pod \"ovnkube-node-rtw9n\" (UID: \"73fbf16b-3549-4a6c-a959-b633237ebd0c\") " pod="openshift-ovn-kubernetes/ovnkube-node-rtw9n"
Sep 18 20:01:12.340964 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:12.339082    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/73fbf16b-3549-4a6c-a959-b633237ebd0c-host-run-netns\") pod \"ovnkube-node-rtw9n\" (UID: \"73fbf16b-3549-4a6c-a959-b633237ebd0c\") " pod="openshift-ovn-kubernetes/ovnkube-node-rtw9n"
Sep 18 20:01:12.340964 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:12.339111    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/73fbf16b-3549-4a6c-a959-b633237ebd0c-log-socket\") pod \"ovnkube-node-rtw9n\" (UID: \"73fbf16b-3549-4a6c-a959-b633237ebd0c\") " pod="openshift-ovn-kubernetes/ovnkube-node-rtw9n"
Sep 18 20:01:12.340964 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:12.339174    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/5c8c76fd-1219-40d0-b010-cd89d714a3fe-tuning-conf-dir\") pod \"multus-additional-cni-plugins-pn4rk\" (UID: \"5c8c76fd-1219-40d0-b010-cd89d714a3fe\") " pod="openshift-multus/multus-additional-cni-plugins-pn4rk"
Sep 18 20:01:12.340964 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:12.339177    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/6b3e5758-70d0-496d-90a9-35d371e8e1b6-lib-modules\") pod \"tuned-spc6q\" (UID: \"6b3e5758-70d0-496d-90a9-35d371e8e1b6\") " pod="openshift-cluster-node-tuning-operator/tuned-spc6q"
Sep 18 20:01:12.340964 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:12.339211    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s7k29\" (UniqueName: \"kubernetes.io/projected/87c4bfcc-4ad2-4166-b5c9-1051bdd2717d-kube-api-access-s7k29\") pod \"node-resolver-b8jbz\" (UID: \"87c4bfcc-4ad2-4166-b5c9-1051bdd2717d\") " pod="openshift-dns/node-resolver-b8jbz"
Sep 18 20:01:12.340964 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:12.339247    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/6b3e5758-70d0-496d-90a9-35d371e8e1b6-etc-tuned\") pod \"tuned-spc6q\" (UID: \"6b3e5758-70d0-496d-90a9-35d371e8e1b6\") " pod="openshift-cluster-node-tuning-operator/tuned-spc6q"
Sep 18 20:01:12.340964 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:12.339255    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/bf9010d7-90b5-4f95-8dea-8af5839c0f87-host-var-lib-cni-multus\") pod \"multus-jbmm2\" (UID: \"bf9010d7-90b5-4f95-8dea-8af5839c0f87\") " pod="openshift-multus/multus-jbmm2"
Sep 18 20:01:12.340964 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:12.339261    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/73fbf16b-3549-4a6c-a959-b633237ebd0c-log-socket\") pod \"ovnkube-node-rtw9n\" (UID: \"73fbf16b-3549-4a6c-a959-b633237ebd0c\") " pod="openshift-ovn-kubernetes/ovnkube-node-rtw9n"
Sep 18 20:01:12.340964 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:12.339298    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-blmfw\" (UniqueName: \"kubernetes.io/projected/6b3e5758-70d0-496d-90a9-35d371e8e1b6-kube-api-access-blmfw\") pod \"tuned-spc6q\" (UID: \"6b3e5758-70d0-496d-90a9-35d371e8e1b6\") " pod="openshift-cluster-node-tuning-operator/tuned-spc6q"
Sep 18 20:01:12.340964 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:12.339425    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/5c8c76fd-1219-40d0-b010-cd89d714a3fe-tuning-conf-dir\") pod \"multus-additional-cni-plugins-pn4rk\" (UID: \"5c8c76fd-1219-40d0-b010-cd89d714a3fe\") " pod="openshift-multus/multus-additional-cni-plugins-pn4rk"
Sep 18 20:01:12.340964 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:12.339576    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/5c8c76fd-1219-40d0-b010-cd89d714a3fe-cni-binary-copy\") pod \"multus-additional-cni-plugins-pn4rk\" (UID: \"5c8c76fd-1219-40d0-b010-cd89d714a3fe\") " pod="openshift-multus/multus-additional-cni-plugins-pn4rk"
Sep 18 20:01:12.340964 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:12.340282    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6b3e5758-70d0-496d-90a9-35d371e8e1b6-tmp\") pod \"tuned-spc6q\" (UID: \"6b3e5758-70d0-496d-90a9-35d371e8e1b6\") " pod="openshift-cluster-node-tuning-operator/tuned-spc6q"
Sep 18 20:01:12.340964 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:12.340527    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/73fbf16b-3549-4a6c-a959-b633237ebd0c-ovn-node-metrics-cert\") pod \"ovnkube-node-rtw9n\" (UID: \"73fbf16b-3549-4a6c-a959-b633237ebd0c\") " pod="openshift-ovn-kubernetes/ovnkube-node-rtw9n"
Sep 18 20:01:12.341649 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:12.341533    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/6b3e5758-70d0-496d-90a9-35d371e8e1b6-etc-tuned\") pod \"tuned-spc6q\" (UID: \"6b3e5758-70d0-496d-90a9-35d371e8e1b6\") " pod="openshift-cluster-node-tuning-operator/tuned-spc6q"
Sep 18 20:01:12.343866 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:12.343843    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4h9pz\" (UniqueName: \"kubernetes.io/projected/5c8c76fd-1219-40d0-b010-cd89d714a3fe-kube-api-access-4h9pz\") pod \"multus-additional-cni-plugins-pn4rk\" (UID: \"5c8c76fd-1219-40d0-b010-cd89d714a3fe\") " pod="openshift-multus/multus-additional-cni-plugins-pn4rk"
Sep 18 20:01:12.345413 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:12.345394    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-trvsb\" (UniqueName: \"kubernetes.io/projected/73fbf16b-3549-4a6c-a959-b633237ebd0c-kube-api-access-trvsb\") pod \"ovnkube-node-rtw9n\" (UID: \"73fbf16b-3549-4a6c-a959-b633237ebd0c\") " pod="openshift-ovn-kubernetes/ovnkube-node-rtw9n"
Sep 18 20:01:12.346528 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:12.346506    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-blmfw\" (UniqueName: \"kubernetes.io/projected/6b3e5758-70d0-496d-90a9-35d371e8e1b6-kube-api-access-blmfw\") pod \"tuned-spc6q\" (UID: \"6b3e5758-70d0-496d-90a9-35d371e8e1b6\") " pod="openshift-cluster-node-tuning-operator/tuned-spc6q"
Sep 18 20:01:12.347021 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:12.346994    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xnvlz\" (UniqueName: \"kubernetes.io/projected/96167e3f-fb44-48e0-b86e-419a4a3b117c-kube-api-access-xnvlz\") pod \"network-metrics-daemon-xx7f7\" (UID: \"96167e3f-fb44-48e0-b86e-419a4a3b117c\") " pod="openshift-multus/network-metrics-daemon-xx7f7"
Sep 18 20:01:12.347441 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:12.347417    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qq7pb\" (UniqueName: \"kubernetes.io/projected/bf9010d7-90b5-4f95-8dea-8af5839c0f87-kube-api-access-qq7pb\") pod \"multus-jbmm2\" (UID: \"bf9010d7-90b5-4f95-8dea-8af5839c0f87\") " pod="openshift-multus/multus-jbmm2"
Sep 18 20:01:12.440381 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:12.440346    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-slbwq\" (UniqueName: \"kubernetes.io/projected/a966f963-b4ae-46ba-b206-2a27ad401463-kube-api-access-slbwq\") pod \"network-check-target-gd24n\" (UID: \"a966f963-b4ae-46ba-b206-2a27ad401463\") " pod="openshift-network-diagnostics/network-check-target-gd24n"
Sep 18 20:01:12.440579 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:12.440409    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/d94756ff-5c71-43bc-ab39-b0fde15a36a2-agent-certs\") pod \"konnectivity-agent-g749g\" (UID: \"d94756ff-5c71-43bc-ab39-b0fde15a36a2\") " pod="kube-system/konnectivity-agent-g749g"
Sep 18 20:01:12.440579 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:12.440511    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/d94756ff-5c71-43bc-ab39-b0fde15a36a2-konnectivity-ca\") pod \"konnectivity-agent-g749g\" (UID: \"d94756ff-5c71-43bc-ab39-b0fde15a36a2\") " pod="kube-system/konnectivity-agent-g749g"
Sep 18 20:01:12.440579 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:12.440535    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/7f063a5b-9649-46fc-8278-3f8bb49700ed-device-dir\") pod \"aws-ebs-csi-driver-node-nbw5k\" (UID: \"7f063a5b-9649-46fc-8278-3f8bb49700ed\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nbw5k"
Sep 18 20:01:12.440579 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:12.440557    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h84rm\" (UniqueName: \"kubernetes.io/projected/7f063a5b-9649-46fc-8278-3f8bb49700ed-kube-api-access-h84rm\") pod \"aws-ebs-csi-driver-node-nbw5k\" (UID: \"7f063a5b-9649-46fc-8278-3f8bb49700ed\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nbw5k"
Sep 18 20:01:12.440579 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:12.440579    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/7f063a5b-9649-46fc-8278-3f8bb49700ed-sys-fs\") pod \"aws-ebs-csi-driver-node-nbw5k\" (UID: \"7f063a5b-9649-46fc-8278-3f8bb49700ed\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nbw5k"
Sep 18 20:01:12.440817 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:12.440651    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/7f063a5b-9649-46fc-8278-3f8bb49700ed-device-dir\") pod \"aws-ebs-csi-driver-node-nbw5k\" (UID: \"7f063a5b-9649-46fc-8278-3f8bb49700ed\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nbw5k"
Sep 18 20:01:12.440817 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:12.440701    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/87c4bfcc-4ad2-4166-b5c9-1051bdd2717d-hosts-file\") pod \"node-resolver-b8jbz\" (UID: \"87c4bfcc-4ad2-4166-b5c9-1051bdd2717d\") " pod="openshift-dns/node-resolver-b8jbz"
Sep 18 20:01:12.440817 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:12.440717    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/7f063a5b-9649-46fc-8278-3f8bb49700ed-sys-fs\") pod \"aws-ebs-csi-driver-node-nbw5k\" (UID: \"7f063a5b-9649-46fc-8278-3f8bb49700ed\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nbw5k"
Sep 18 20:01:12.440817 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:12.440742    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/4dc79459-39e4-4afe-b537-b657780854fb-host-slash\") pod \"iptables-alerter-dxcqj\" (UID: \"4dc79459-39e4-4afe-b537-b657780854fb\") " pod="openshift-network-operator/iptables-alerter-dxcqj"
Sep 18 20:01:12.440817 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:12.440769    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b771ff0e-edf5-4d81-adf7-6350fe1ea781-host\") pod \"node-ca-wkwnc\" (UID: \"b771ff0e-edf5-4d81-adf7-6350fe1ea781\") " pod="openshift-image-registry/node-ca-wkwnc"
Sep 18 20:01:12.440817 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:12.440795    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/4dc79459-39e4-4afe-b537-b657780854fb-iptables-alerter-script\") pod \"iptables-alerter-dxcqj\" (UID: \"4dc79459-39e4-4afe-b537-b657780854fb\") " pod="openshift-network-operator/iptables-alerter-dxcqj"
Sep 18 20:01:12.440817 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:12.440819    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hvm5x\" (UniqueName: \"kubernetes.io/projected/4dc79459-39e4-4afe-b537-b657780854fb-kube-api-access-hvm5x\") pod \"iptables-alerter-dxcqj\" (UID: \"4dc79459-39e4-4afe-b537-b657780854fb\") " pod="openshift-network-operator/iptables-alerter-dxcqj"
Sep 18 20:01:12.441046 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:12.440828    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/4dc79459-39e4-4afe-b537-b657780854fb-host-slash\") pod \"iptables-alerter-dxcqj\" (UID: \"4dc79459-39e4-4afe-b537-b657780854fb\") " pod="openshift-network-operator/iptables-alerter-dxcqj"
Sep 18 20:01:12.441046 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:12.440845    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b771ff0e-edf5-4d81-adf7-6350fe1ea781-host\") pod \"node-ca-wkwnc\" (UID: \"b771ff0e-edf5-4d81-adf7-6350fe1ea781\") " pod="openshift-image-registry/node-ca-wkwnc"
Sep 18 20:01:12.441046 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:12.440859    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/87c4bfcc-4ad2-4166-b5c9-1051bdd2717d-hosts-file\") pod \"node-resolver-b8jbz\" (UID: \"87c4bfcc-4ad2-4166-b5c9-1051bdd2717d\") " pod="openshift-dns/node-resolver-b8jbz"
Sep 18 20:01:12.441046 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:12.440848    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qr7vd\" (UniqueName: \"kubernetes.io/projected/b771ff0e-edf5-4d81-adf7-6350fe1ea781-kube-api-access-qr7vd\") pod \"node-ca-wkwnc\" (UID: \"b771ff0e-edf5-4d81-adf7-6350fe1ea781\") " pod="openshift-image-registry/node-ca-wkwnc"
Sep 18 20:01:12.441046 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:12.440909    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/b771ff0e-edf5-4d81-adf7-6350fe1ea781-serviceca\") pod \"node-ca-wkwnc\" (UID: \"b771ff0e-edf5-4d81-adf7-6350fe1ea781\") " pod="openshift-image-registry/node-ca-wkwnc"
Sep 18 20:01:12.441046 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:12.440935    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/b377a9e6-dbbb-4023-8997-70178a63034f-dbus\") pod \"global-pull-secret-syncer-gd9qz\" (UID: \"b377a9e6-dbbb-4023-8997-70178a63034f\") " pod="kube-system/global-pull-secret-syncer-gd9qz"
Sep 18 20:01:12.441046 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:12.440960    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/7f063a5b-9649-46fc-8278-3f8bb49700ed-kubelet-dir\") pod \"aws-ebs-csi-driver-node-nbw5k\" (UID: \"7f063a5b-9649-46fc-8278-3f8bb49700ed\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nbw5k"
Sep 18 20:01:12.441046 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:12.440996    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/7f063a5b-9649-46fc-8278-3f8bb49700ed-socket-dir\") pod \"aws-ebs-csi-driver-node-nbw5k\" (UID: \"7f063a5b-9649-46fc-8278-3f8bb49700ed\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nbw5k"
Sep 18 20:01:12.441046 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:12.441019    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b377a9e6-dbbb-4023-8997-70178a63034f-original-pull-secret\") pod \"global-pull-secret-syncer-gd9qz\" (UID: \"b377a9e6-dbbb-4023-8997-70178a63034f\") " pod="kube-system/global-pull-secret-syncer-gd9qz"
Sep 18 20:01:12.441324 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:12.441049    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/b377a9e6-dbbb-4023-8997-70178a63034f-kubelet-config\") pod \"global-pull-secret-syncer-gd9qz\" (UID: \"b377a9e6-dbbb-4023-8997-70178a63034f\") " pod="kube-system/global-pull-secret-syncer-gd9qz"
Sep 18 20:01:12.441324 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:12.441074    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/7f063a5b-9649-46fc-8278-3f8bb49700ed-registration-dir\") pod \"aws-ebs-csi-driver-node-nbw5k\" (UID: \"7f063a5b-9649-46fc-8278-3f8bb49700ed\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nbw5k"
Sep 18 20:01:12.441324 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:12.441105    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/7f063a5b-9649-46fc-8278-3f8bb49700ed-etc-selinux\") pod \"aws-ebs-csi-driver-node-nbw5k\" (UID: \"7f063a5b-9649-46fc-8278-3f8bb49700ed\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nbw5k"
Sep 18 20:01:12.441324 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:12.441129    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/87c4bfcc-4ad2-4166-b5c9-1051bdd2717d-tmp-dir\") pod \"node-resolver-b8jbz\" (UID: \"87c4bfcc-4ad2-4166-b5c9-1051bdd2717d\") " pod="openshift-dns/node-resolver-b8jbz"
Sep 18 20:01:12.441324 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:12.441149    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/d94756ff-5c71-43bc-ab39-b0fde15a36a2-konnectivity-ca\") pod \"konnectivity-agent-g749g\" (UID: \"d94756ff-5c71-43bc-ab39-b0fde15a36a2\") " pod="kube-system/konnectivity-agent-g749g"
Sep 18 20:01:12.441324 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:12.441189    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/7f063a5b-9649-46fc-8278-3f8bb49700ed-socket-dir\") pod \"aws-ebs-csi-driver-node-nbw5k\" (UID: \"7f063a5b-9649-46fc-8278-3f8bb49700ed\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nbw5k"
Sep 18 20:01:12.441529 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:12.441324    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/b377a9e6-dbbb-4023-8997-70178a63034f-dbus\") pod \"global-pull-secret-syncer-gd9qz\" (UID: \"b377a9e6-dbbb-4023-8997-70178a63034f\") " pod="kube-system/global-pull-secret-syncer-gd9qz"
Sep 18 20:01:12.441529 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:12.441155    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s7k29\" (UniqueName: \"kubernetes.io/projected/87c4bfcc-4ad2-4166-b5c9-1051bdd2717d-kube-api-access-s7k29\") pod \"node-resolver-b8jbz\" (UID: \"87c4bfcc-4ad2-4166-b5c9-1051bdd2717d\") " pod="openshift-dns/node-resolver-b8jbz"
Sep 18 20:01:12.441529 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:12.441400    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/7f063a5b-9649-46fc-8278-3f8bb49700ed-etc-selinux\") pod \"aws-ebs-csi-driver-node-nbw5k\" (UID: \"7f063a5b-9649-46fc-8278-3f8bb49700ed\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nbw5k"
Sep 18 20:01:12.441529 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:12.441446    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/b377a9e6-dbbb-4023-8997-70178a63034f-kubelet-config\") pod \"global-pull-secret-syncer-gd9qz\" (UID: \"b377a9e6-dbbb-4023-8997-70178a63034f\") " pod="kube-system/global-pull-secret-syncer-gd9qz"
Sep 18 20:01:12.441529 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:12.441452    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/b771ff0e-edf5-4d81-adf7-6350fe1ea781-serviceca\") pod \"node-ca-wkwnc\" (UID: \"b771ff0e-edf5-4d81-adf7-6350fe1ea781\") " pod="openshift-image-registry/node-ca-wkwnc"
Sep 18 20:01:12.441529 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:12.441461    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/4dc79459-39e4-4afe-b537-b657780854fb-iptables-alerter-script\") pod \"iptables-alerter-dxcqj\" (UID: \"4dc79459-39e4-4afe-b537-b657780854fb\") " pod="openshift-network-operator/iptables-alerter-dxcqj"
Sep 18 20:01:12.441529 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:12.441468    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/7f063a5b-9649-46fc-8278-3f8bb49700ed-kubelet-dir\") pod \"aws-ebs-csi-driver-node-nbw5k\" (UID: \"7f063a5b-9649-46fc-8278-3f8bb49700ed\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nbw5k"
Sep 18 20:01:12.441824 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:12.441530    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/87c4bfcc-4ad2-4166-b5c9-1051bdd2717d-tmp-dir\") pod \"node-resolver-b8jbz\" (UID: \"87c4bfcc-4ad2-4166-b5c9-1051bdd2717d\") " pod="openshift-dns/node-resolver-b8jbz"
Sep 18 20:01:12.441824 ip-10-0-130-81 kubenswrapper[2532]: E0918 20:01:12.441533    2532 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 18 20:01:12.441824 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:12.441553    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/7f063a5b-9649-46fc-8278-3f8bb49700ed-registration-dir\") pod \"aws-ebs-csi-driver-node-nbw5k\" (UID: \"7f063a5b-9649-46fc-8278-3f8bb49700ed\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nbw5k"
Sep 18 20:01:12.441824 ip-10-0-130-81 kubenswrapper[2532]: E0918 20:01:12.441622    2532 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b377a9e6-dbbb-4023-8997-70178a63034f-original-pull-secret podName:b377a9e6-dbbb-4023-8997-70178a63034f nodeName:}" failed. No retries permitted until 2026-09-18 20:01:12.941602414 +0000 UTC m=+4.706991796 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b377a9e6-dbbb-4023-8997-70178a63034f-original-pull-secret") pod "global-pull-secret-syncer-gd9qz" (UID: "b377a9e6-dbbb-4023-8997-70178a63034f") : object "kube-system"/"original-pull-secret" not registered
Sep 18 20:01:12.443223 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:12.443196    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/d94756ff-5c71-43bc-ab39-b0fde15a36a2-agent-certs\") pod \"konnectivity-agent-g749g\" (UID: \"d94756ff-5c71-43bc-ab39-b0fde15a36a2\") " pod="kube-system/konnectivity-agent-g749g"
Sep 18 20:01:12.447115 ip-10-0-130-81 kubenswrapper[2532]: E0918 20:01:12.446830    2532 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 18 20:01:12.447115 ip-10-0-130-81 kubenswrapper[2532]: E0918 20:01:12.446850    2532 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 18 20:01:12.447115 ip-10-0-130-81 kubenswrapper[2532]: E0918 20:01:12.446862    2532 projected.go:196] Error preparing data for projected volume kube-api-access-slbwq for pod openshift-network-diagnostics/network-check-target-gd24n: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 18 20:01:12.447115 ip-10-0-130-81 kubenswrapper[2532]: E0918 20:01:12.446923    2532 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a966f963-b4ae-46ba-b206-2a27ad401463-kube-api-access-slbwq podName:a966f963-b4ae-46ba-b206-2a27ad401463 nodeName:}" failed. No retries permitted until 2026-09-18 20:01:12.94690709 +0000 UTC m=+4.712296474 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-slbwq" (UniqueName: "kubernetes.io/projected/a966f963-b4ae-46ba-b206-2a27ad401463-kube-api-access-slbwq") pod "network-check-target-gd24n" (UID: "a966f963-b4ae-46ba-b206-2a27ad401463") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 18 20:01:12.449055 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:12.449014    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qr7vd\" (UniqueName: \"kubernetes.io/projected/b771ff0e-edf5-4d81-adf7-6350fe1ea781-kube-api-access-qr7vd\") pod \"node-ca-wkwnc\" (UID: \"b771ff0e-edf5-4d81-adf7-6350fe1ea781\") " pod="openshift-image-registry/node-ca-wkwnc"
Sep 18 20:01:12.450063 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:12.450038    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s7k29\" (UniqueName: \"kubernetes.io/projected/87c4bfcc-4ad2-4166-b5c9-1051bdd2717d-kube-api-access-s7k29\") pod \"node-resolver-b8jbz\" (UID: \"87c4bfcc-4ad2-4166-b5c9-1051bdd2717d\") " pod="openshift-dns/node-resolver-b8jbz"
Sep 18 20:01:12.450354 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:12.450336    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hvm5x\" (UniqueName: \"kubernetes.io/projected/4dc79459-39e4-4afe-b537-b657780854fb-kube-api-access-hvm5x\") pod \"iptables-alerter-dxcqj\" (UID: \"4dc79459-39e4-4afe-b537-b657780854fb\") " pod="openshift-network-operator/iptables-alerter-dxcqj"
Sep 18 20:01:12.450454 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:12.450432    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h84rm\" (UniqueName: \"kubernetes.io/projected/7f063a5b-9649-46fc-8278-3f8bb49700ed-kube-api-access-h84rm\") pod \"aws-ebs-csi-driver-node-nbw5k\" (UID: \"7f063a5b-9649-46fc-8278-3f8bb49700ed\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nbw5k"
Sep 18 20:01:12.528212 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:12.528170    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-spc6q"
Sep 18 20:01:12.541541 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:12.541518    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-rtw9n"
Sep 18 20:01:12.556368 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:12.556346    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-pn4rk"
Sep 18 20:01:12.562142 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:12.562123    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-jbmm2"
Sep 18 20:01:12.568711 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:12.568692    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-g749g"
Sep 18 20:01:12.576306 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:12.576287    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-wkwnc"
Sep 18 20:01:12.583824 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:12.583807    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nbw5k"
Sep 18 20:01:12.591279 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:12.591264    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-b8jbz"
Sep 18 20:01:12.597717 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:12.597699    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-dxcqj"
Sep 18 20:01:12.843920 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:12.843838    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/96167e3f-fb44-48e0-b86e-419a4a3b117c-metrics-certs\") pod \"network-metrics-daemon-xx7f7\" (UID: \"96167e3f-fb44-48e0-b86e-419a4a3b117c\") " pod="openshift-multus/network-metrics-daemon-xx7f7"
Sep 18 20:01:12.844072 ip-10-0-130-81 kubenswrapper[2532]: E0918 20:01:12.843969    2532 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 18 20:01:12.844072 ip-10-0-130-81 kubenswrapper[2532]: E0918 20:01:12.844032    2532 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/96167e3f-fb44-48e0-b86e-419a4a3b117c-metrics-certs podName:96167e3f-fb44-48e0-b86e-419a4a3b117c nodeName:}" failed. No retries permitted until 2026-09-18 20:01:13.844013293 +0000 UTC m=+5.609402693 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/96167e3f-fb44-48e0-b86e-419a4a3b117c-metrics-certs") pod "network-metrics-daemon-xx7f7" (UID: "96167e3f-fb44-48e0-b86e-419a4a3b117c") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 18 20:01:12.944996 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:12.944967    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b377a9e6-dbbb-4023-8997-70178a63034f-original-pull-secret\") pod \"global-pull-secret-syncer-gd9qz\" (UID: \"b377a9e6-dbbb-4023-8997-70178a63034f\") " pod="kube-system/global-pull-secret-syncer-gd9qz"
Sep 18 20:01:12.945135 ip-10-0-130-81 kubenswrapper[2532]: E0918 20:01:12.945099    2532 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 18 20:01:12.945184 ip-10-0-130-81 kubenswrapper[2532]: E0918 20:01:12.945154    2532 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b377a9e6-dbbb-4023-8997-70178a63034f-original-pull-secret podName:b377a9e6-dbbb-4023-8997-70178a63034f nodeName:}" failed. No retries permitted until 2026-09-18 20:01:13.945139232 +0000 UTC m=+5.710528609 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b377a9e6-dbbb-4023-8997-70178a63034f-original-pull-secret") pod "global-pull-secret-syncer-gd9qz" (UID: "b377a9e6-dbbb-4023-8997-70178a63034f") : object "kube-system"/"original-pull-secret" not registered
Sep 18 20:01:13.045913 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:13.045879    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-slbwq\" (UniqueName: \"kubernetes.io/projected/a966f963-b4ae-46ba-b206-2a27ad401463-kube-api-access-slbwq\") pod \"network-check-target-gd24n\" (UID: \"a966f963-b4ae-46ba-b206-2a27ad401463\") " pod="openshift-network-diagnostics/network-check-target-gd24n"
Sep 18 20:01:13.046076 ip-10-0-130-81 kubenswrapper[2532]: E0918 20:01:13.046031    2532 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 18 20:01:13.046076 ip-10-0-130-81 kubenswrapper[2532]: E0918 20:01:13.046047    2532 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 18 20:01:13.046076 ip-10-0-130-81 kubenswrapper[2532]: E0918 20:01:13.046059    2532 projected.go:196] Error preparing data for projected volume kube-api-access-slbwq for pod openshift-network-diagnostics/network-check-target-gd24n: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 18 20:01:13.046213 ip-10-0-130-81 kubenswrapper[2532]: E0918 20:01:13.046120    2532 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a966f963-b4ae-46ba-b206-2a27ad401463-kube-api-access-slbwq podName:a966f963-b4ae-46ba-b206-2a27ad401463 nodeName:}" failed. No retries permitted until 2026-09-18 20:01:14.046104451 +0000 UTC m=+5.811493830 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-slbwq" (UniqueName: "kubernetes.io/projected/a966f963-b4ae-46ba-b206-2a27ad401463-kube-api-access-slbwq") pod "network-check-target-gd24n" (UID: "a966f963-b4ae-46ba-b206-2a27ad401463") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 18 20:01:13.271188 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:13.271155    2532 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-17 19:56:12 +0000 UTC" deadline="2028-06-24 13:23:40.42413921 +0000 UTC"
Sep 18 20:01:13.271188 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:13.271185    2532 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15473h22m27.152958236s"
Sep 18 20:01:13.300617 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:13.300587    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gd9qz"
Sep 18 20:01:13.300617 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:13.300603    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xx7f7"
Sep 18 20:01:13.300804 ip-10-0-130-81 kubenswrapper[2532]: E0918 20:01:13.300690    2532 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gd9qz" podUID="b377a9e6-dbbb-4023-8997-70178a63034f"
Sep 18 20:01:13.300860 ip-10-0-130-81 kubenswrapper[2532]: E0918 20:01:13.300840    2532 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xx7f7" podUID="96167e3f-fb44-48e0-b86e-419a4a3b117c"
Sep 18 20:01:13.327007 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:13.326984    2532 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod73fbf16b_3549_4a6c_a959_b633237ebd0c.slice/crio-43c93666a10673cf4002871e565017bc21b771d0f0c3ba61885cc2637c6645dd WatchSource:0}: Error finding container 43c93666a10673cf4002871e565017bc21b771d0f0c3ba61885cc2637c6645dd: Status 404 returned error can't find the container with id 43c93666a10673cf4002871e565017bc21b771d0f0c3ba61885cc2637c6645dd
Sep 18 20:01:13.332666 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:13.332639    2532 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod87c4bfcc_4ad2_4166_b5c9_1051bdd2717d.slice/crio-926dbe3b1924c32967680c7713ccc052b4cc580c53c3eee20c74d617bf31720f WatchSource:0}: Error finding container 926dbe3b1924c32967680c7713ccc052b4cc580c53c3eee20c74d617bf31720f: Status 404 returned error can't find the container with id 926dbe3b1924c32967680c7713ccc052b4cc580c53c3eee20c74d617bf31720f
Sep 18 20:01:13.333176 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:13.333147    2532 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd94756ff_5c71_43bc_ab39_b0fde15a36a2.slice/crio-6d9498f05210f3a4d8115bfaba5be1ec6b871899d35c1e706b28666a7ca3f4a2 WatchSource:0}: Error finding container 6d9498f05210f3a4d8115bfaba5be1ec6b871899d35c1e706b28666a7ca3f4a2: Status 404 returned error can't find the container with id 6d9498f05210f3a4d8115bfaba5be1ec6b871899d35c1e706b28666a7ca3f4a2
Sep 18 20:01:13.334461 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:13.334441    2532 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5c8c76fd_1219_40d0_b010_cd89d714a3fe.slice/crio-9adba7bd76c22fb772fa6bd8b7e120dec959c2a2952b37d4518f69d350ad3216 WatchSource:0}: Error finding container 9adba7bd76c22fb772fa6bd8b7e120dec959c2a2952b37d4518f69d350ad3216: Status 404 returned error can't find the container with id 9adba7bd76c22fb772fa6bd8b7e120dec959c2a2952b37d4518f69d350ad3216
Sep 18 20:01:13.335667 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:13.335373    2532 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7f063a5b_9649_46fc_8278_3f8bb49700ed.slice/crio-034417031cade8867145efc338a70d76c81983176afc65bc0c1bd86d18fa0fd4 WatchSource:0}: Error finding container 034417031cade8867145efc338a70d76c81983176afc65bc0c1bd86d18fa0fd4: Status 404 returned error can't find the container with id 034417031cade8867145efc338a70d76c81983176afc65bc0c1bd86d18fa0fd4
Sep 18 20:01:13.337725 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:13.337673    2532 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6b3e5758_70d0_496d_90a9_35d371e8e1b6.slice/crio-83cb75a09aa63484f9fbe54296126422c38af95cd09544f21edb3ecdec74eabb WatchSource:0}: Error finding container 83cb75a09aa63484f9fbe54296126422c38af95cd09544f21edb3ecdec74eabb: Status 404 returned error can't find the container with id 83cb75a09aa63484f9fbe54296126422c38af95cd09544f21edb3ecdec74eabb
Sep 18 20:01:13.339128 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:13.339076    2532 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb771ff0e_edf5_4d81_adf7_6350fe1ea781.slice/crio-a9a7416c2b26e0113663daf26792ca805f8942cae0f20c36b5db575f3a778cd2 WatchSource:0}: Error finding container a9a7416c2b26e0113663daf26792ca805f8942cae0f20c36b5db575f3a778cd2: Status 404 returned error can't find the container with id a9a7416c2b26e0113663daf26792ca805f8942cae0f20c36b5db575f3a778cd2
Sep 18 20:01:13.339453 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:13.339424    2532 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbf9010d7_90b5_4f95_8dea_8af5839c0f87.slice/crio-ba861788725044377a19013678625b61610576c26cfb701440ea92f58961f24b WatchSource:0}: Error finding container ba861788725044377a19013678625b61610576c26cfb701440ea92f58961f24b: Status 404 returned error can't find the container with id ba861788725044377a19013678625b61610576c26cfb701440ea92f58961f24b
Sep 18 20:01:13.340303 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:13.340282    2532 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4dc79459_39e4_4afe_b537_b657780854fb.slice/crio-4db6f9d7471fe1a67b7a1d6c4550ad108713b7d94ded9ddb387727b28d03a801 WatchSource:0}: Error finding container 4db6f9d7471fe1a67b7a1d6c4550ad108713b7d94ded9ddb387727b28d03a801: Status 404 returned error can't find the container with id 4db6f9d7471fe1a67b7a1d6c4550ad108713b7d94ded9ddb387727b28d03a801
Sep 18 20:01:13.857188 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:13.856887    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/96167e3f-fb44-48e0-b86e-419a4a3b117c-metrics-certs\") pod \"network-metrics-daemon-xx7f7\" (UID: \"96167e3f-fb44-48e0-b86e-419a4a3b117c\") " pod="openshift-multus/network-metrics-daemon-xx7f7"
Sep 18 20:01:13.857370 ip-10-0-130-81 kubenswrapper[2532]: E0918 20:01:13.857262    2532 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 18 20:01:13.857370 ip-10-0-130-81 kubenswrapper[2532]: E0918 20:01:13.857324    2532 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/96167e3f-fb44-48e0-b86e-419a4a3b117c-metrics-certs podName:96167e3f-fb44-48e0-b86e-419a4a3b117c nodeName:}" failed. No retries permitted until 2026-09-18 20:01:15.857304221 +0000 UTC m=+7.622693604 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/96167e3f-fb44-48e0-b86e-419a4a3b117c-metrics-certs") pod "network-metrics-daemon-xx7f7" (UID: "96167e3f-fb44-48e0-b86e-419a4a3b117c") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 18 20:01:13.958078 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:13.958044    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b377a9e6-dbbb-4023-8997-70178a63034f-original-pull-secret\") pod \"global-pull-secret-syncer-gd9qz\" (UID: \"b377a9e6-dbbb-4023-8997-70178a63034f\") " pod="kube-system/global-pull-secret-syncer-gd9qz"
Sep 18 20:01:13.958279 ip-10-0-130-81 kubenswrapper[2532]: E0918 20:01:13.958263    2532 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 18 20:01:13.958350 ip-10-0-130-81 kubenswrapper[2532]: E0918 20:01:13.958326    2532 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b377a9e6-dbbb-4023-8997-70178a63034f-original-pull-secret podName:b377a9e6-dbbb-4023-8997-70178a63034f nodeName:}" failed. No retries permitted until 2026-09-18 20:01:15.958307216 +0000 UTC m=+7.723696599 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b377a9e6-dbbb-4023-8997-70178a63034f-original-pull-secret") pod "global-pull-secret-syncer-gd9qz" (UID: "b377a9e6-dbbb-4023-8997-70178a63034f") : object "kube-system"/"original-pull-secret" not registered
Sep 18 20:01:14.059377 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:14.058775    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-slbwq\" (UniqueName: \"kubernetes.io/projected/a966f963-b4ae-46ba-b206-2a27ad401463-kube-api-access-slbwq\") pod \"network-check-target-gd24n\" (UID: \"a966f963-b4ae-46ba-b206-2a27ad401463\") " pod="openshift-network-diagnostics/network-check-target-gd24n"
Sep 18 20:01:14.059377 ip-10-0-130-81 kubenswrapper[2532]: E0918 20:01:14.058949    2532 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 18 20:01:14.059377 ip-10-0-130-81 kubenswrapper[2532]: E0918 20:01:14.058971    2532 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 18 20:01:14.059377 ip-10-0-130-81 kubenswrapper[2532]: E0918 20:01:14.058984    2532 projected.go:196] Error preparing data for projected volume kube-api-access-slbwq for pod openshift-network-diagnostics/network-check-target-gd24n: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 18 20:01:14.059377 ip-10-0-130-81 kubenswrapper[2532]: E0918 20:01:14.059041    2532 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a966f963-b4ae-46ba-b206-2a27ad401463-kube-api-access-slbwq podName:a966f963-b4ae-46ba-b206-2a27ad401463 nodeName:}" failed. No retries permitted until 2026-09-18 20:01:16.059024118 +0000 UTC m=+7.824413503 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-slbwq" (UniqueName: "kubernetes.io/projected/a966f963-b4ae-46ba-b206-2a27ad401463-kube-api-access-slbwq") pod "network-check-target-gd24n" (UID: "a966f963-b4ae-46ba-b206-2a27ad401463") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 18 20:01:14.272333 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:14.272210    2532 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-17 19:56:12 +0000 UTC" deadline="2028-02-14 05:52:14.283973864 +0000 UTC"
Sep 18 20:01:14.272333 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:14.272285    2532 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12321h51m0.011693292s"
Sep 18 20:01:14.300628 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:14.300598    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gd24n"
Sep 18 20:01:14.300812 ip-10-0-130-81 kubenswrapper[2532]: E0918 20:01:14.300759    2532 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gd24n" podUID="a966f963-b4ae-46ba-b206-2a27ad401463"
Sep 18 20:01:14.324963 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:14.324928    2532 generic.go:356] "Generic (PLEG): container finished" podID="ae7090a051a4a9229a0e9951ab9e2daf" containerID="8600d3d85e7ffb10c1c3fe15b5331b914f02a03a0092baf15d969fdd9bd5f54e" exitCode=0
Sep 18 20:01:14.325574 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:14.325549    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-81.ec2.internal" event={"ID":"ae7090a051a4a9229a0e9951ab9e2daf","Type":"ContainerDied","Data":"8600d3d85e7ffb10c1c3fe15b5331b914f02a03a0092baf15d969fdd9bd5f54e"}
Sep 18 20:01:14.342343 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:14.342312    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-81.ec2.internal" event={"ID":"7719088cb6c9e3ddb918a0f82975ee3c","Type":"ContainerStarted","Data":"0d6e082feb5b9428e366d6ca7b123d4f5235ebae821322408a7b5955730ae72e"}
Sep 18 20:01:14.348289 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:14.348203    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-wkwnc" event={"ID":"b771ff0e-edf5-4d81-adf7-6350fe1ea781","Type":"ContainerStarted","Data":"a9a7416c2b26e0113663daf26792ca805f8942cae0f20c36b5db575f3a778cd2"}
Sep 18 20:01:14.357875 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:14.357274    2532 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-81.ec2.internal" podStartSLOduration=3.357254613 podStartE2EDuration="3.357254613s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-18 20:01:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 20:01:14.357051406 +0000 UTC m=+6.122440806" watchObservedRunningTime="2026-09-18 20:01:14.357254613 +0000 UTC m=+6.122644015"
Sep 18 20:01:14.359426 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:14.359397    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nbw5k" event={"ID":"7f063a5b-9649-46fc-8278-3f8bb49700ed","Type":"ContainerStarted","Data":"034417031cade8867145efc338a70d76c81983176afc65bc0c1bd86d18fa0fd4"}
Sep 18 20:01:14.368820 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:14.364405    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-pn4rk" event={"ID":"5c8c76fd-1219-40d0-b010-cd89d714a3fe","Type":"ContainerStarted","Data":"9adba7bd76c22fb772fa6bd8b7e120dec959c2a2952b37d4518f69d350ad3216"}
Sep 18 20:01:14.374498 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:14.374433    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-b8jbz" event={"ID":"87c4bfcc-4ad2-4166-b5c9-1051bdd2717d","Type":"ContainerStarted","Data":"926dbe3b1924c32967680c7713ccc052b4cc580c53c3eee20c74d617bf31720f"}
Sep 18 20:01:14.380599 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:14.380575    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-rtw9n" event={"ID":"73fbf16b-3549-4a6c-a959-b633237ebd0c","Type":"ContainerStarted","Data":"43c93666a10673cf4002871e565017bc21b771d0f0c3ba61885cc2637c6645dd"}
Sep 18 20:01:14.392195 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:14.392166    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-dxcqj" event={"ID":"4dc79459-39e4-4afe-b537-b657780854fb","Type":"ContainerStarted","Data":"4db6f9d7471fe1a67b7a1d6c4550ad108713b7d94ded9ddb387727b28d03a801"}
Sep 18 20:01:14.406036 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:14.405968    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-jbmm2" event={"ID":"bf9010d7-90b5-4f95-8dea-8af5839c0f87","Type":"ContainerStarted","Data":"ba861788725044377a19013678625b61610576c26cfb701440ea92f58961f24b"}
Sep 18 20:01:14.415258 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:14.415183    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-spc6q" event={"ID":"6b3e5758-70d0-496d-90a9-35d371e8e1b6","Type":"ContainerStarted","Data":"83cb75a09aa63484f9fbe54296126422c38af95cd09544f21edb3ecdec74eabb"}
Sep 18 20:01:14.446287 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:14.446247    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-g749g" event={"ID":"d94756ff-5c71-43bc-ab39-b0fde15a36a2","Type":"ContainerStarted","Data":"6d9498f05210f3a4d8115bfaba5be1ec6b871899d35c1e706b28666a7ca3f4a2"}
Sep 18 20:01:15.299760 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:15.299727    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gd9qz"
Sep 18 20:01:15.300209 ip-10-0-130-81 kubenswrapper[2532]: E0918 20:01:15.299866    2532 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gd9qz" podUID="b377a9e6-dbbb-4023-8997-70178a63034f"
Sep 18 20:01:15.300284 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:15.300241    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xx7f7"
Sep 18 20:01:15.300394 ip-10-0-130-81 kubenswrapper[2532]: E0918 20:01:15.300351    2532 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xx7f7" podUID="96167e3f-fb44-48e0-b86e-419a4a3b117c"
Sep 18 20:01:15.469638 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:15.469550    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-81.ec2.internal" event={"ID":"ae7090a051a4a9229a0e9951ab9e2daf","Type":"ContainerStarted","Data":"f08627dd3b5fb6d74061762b6ba6c334221a221fb748269a6672ef1c5c164a40"}
Sep 18 20:01:15.483862 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:15.483809    2532 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-81.ec2.internal" podStartSLOduration=4.483788358 podStartE2EDuration="4.483788358s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-18 20:01:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 20:01:15.483448578 +0000 UTC m=+7.248837977" watchObservedRunningTime="2026-09-18 20:01:15.483788358 +0000 UTC m=+7.249177761"
Sep 18 20:01:15.875495 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:15.874838    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/96167e3f-fb44-48e0-b86e-419a4a3b117c-metrics-certs\") pod \"network-metrics-daemon-xx7f7\" (UID: \"96167e3f-fb44-48e0-b86e-419a4a3b117c\") " pod="openshift-multus/network-metrics-daemon-xx7f7"
Sep 18 20:01:15.875495 ip-10-0-130-81 kubenswrapper[2532]: E0918 20:01:15.874997    2532 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 18 20:01:15.875495 ip-10-0-130-81 kubenswrapper[2532]: E0918 20:01:15.875061    2532 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/96167e3f-fb44-48e0-b86e-419a4a3b117c-metrics-certs podName:96167e3f-fb44-48e0-b86e-419a4a3b117c nodeName:}" failed. No retries permitted until 2026-09-18 20:01:19.87504223 +0000 UTC m=+11.640431629 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/96167e3f-fb44-48e0-b86e-419a4a3b117c-metrics-certs") pod "network-metrics-daemon-xx7f7" (UID: "96167e3f-fb44-48e0-b86e-419a4a3b117c") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 18 20:01:15.976027 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:15.975404    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b377a9e6-dbbb-4023-8997-70178a63034f-original-pull-secret\") pod \"global-pull-secret-syncer-gd9qz\" (UID: \"b377a9e6-dbbb-4023-8997-70178a63034f\") " pod="kube-system/global-pull-secret-syncer-gd9qz"
Sep 18 20:01:15.976027 ip-10-0-130-81 kubenswrapper[2532]: E0918 20:01:15.975612    2532 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 18 20:01:15.976027 ip-10-0-130-81 kubenswrapper[2532]: E0918 20:01:15.975677    2532 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b377a9e6-dbbb-4023-8997-70178a63034f-original-pull-secret podName:b377a9e6-dbbb-4023-8997-70178a63034f nodeName:}" failed. No retries permitted until 2026-09-18 20:01:19.975659684 +0000 UTC m=+11.741049065 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b377a9e6-dbbb-4023-8997-70178a63034f-original-pull-secret") pod "global-pull-secret-syncer-gd9qz" (UID: "b377a9e6-dbbb-4023-8997-70178a63034f") : object "kube-system"/"original-pull-secret" not registered
Sep 18 20:01:16.076764 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:16.076118    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-slbwq\" (UniqueName: \"kubernetes.io/projected/a966f963-b4ae-46ba-b206-2a27ad401463-kube-api-access-slbwq\") pod \"network-check-target-gd24n\" (UID: \"a966f963-b4ae-46ba-b206-2a27ad401463\") " pod="openshift-network-diagnostics/network-check-target-gd24n"
Sep 18 20:01:16.076764 ip-10-0-130-81 kubenswrapper[2532]: E0918 20:01:16.076321    2532 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 18 20:01:16.076764 ip-10-0-130-81 kubenswrapper[2532]: E0918 20:01:16.076340    2532 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 18 20:01:16.076764 ip-10-0-130-81 kubenswrapper[2532]: E0918 20:01:16.076353    2532 projected.go:196] Error preparing data for projected volume kube-api-access-slbwq for pod openshift-network-diagnostics/network-check-target-gd24n: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 18 20:01:16.076764 ip-10-0-130-81 kubenswrapper[2532]: E0918 20:01:16.076405    2532 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a966f963-b4ae-46ba-b206-2a27ad401463-kube-api-access-slbwq podName:a966f963-b4ae-46ba-b206-2a27ad401463 nodeName:}" failed. No retries permitted until 2026-09-18 20:01:20.076389097 +0000 UTC m=+11.841778478 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-slbwq" (UniqueName: "kubernetes.io/projected/a966f963-b4ae-46ba-b206-2a27ad401463-kube-api-access-slbwq") pod "network-check-target-gd24n" (UID: "a966f963-b4ae-46ba-b206-2a27ad401463") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 18 20:01:16.303114 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:16.302597    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gd24n"
Sep 18 20:01:16.303114 ip-10-0-130-81 kubenswrapper[2532]: E0918 20:01:16.302725    2532 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gd24n" podUID="a966f963-b4ae-46ba-b206-2a27ad401463"
Sep 18 20:01:17.300519 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:17.300177    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gd9qz"
Sep 18 20:01:17.300519 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:17.300224    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xx7f7"
Sep 18 20:01:17.300519 ip-10-0-130-81 kubenswrapper[2532]: E0918 20:01:17.300309    2532 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gd9qz" podUID="b377a9e6-dbbb-4023-8997-70178a63034f"
Sep 18 20:01:17.300519 ip-10-0-130-81 kubenswrapper[2532]: E0918 20:01:17.300461    2532 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xx7f7" podUID="96167e3f-fb44-48e0-b86e-419a4a3b117c"
Sep 18 20:01:18.305212 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:18.305182    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gd24n"
Sep 18 20:01:18.305682 ip-10-0-130-81 kubenswrapper[2532]: E0918 20:01:18.305348    2532 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gd24n" podUID="a966f963-b4ae-46ba-b206-2a27ad401463"
Sep 18 20:01:19.299907 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:19.299867    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xx7f7"
Sep 18 20:01:19.299907 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:19.299908    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gd9qz"
Sep 18 20:01:19.300144 ip-10-0-130-81 kubenswrapper[2532]: E0918 20:01:19.300018    2532 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xx7f7" podUID="96167e3f-fb44-48e0-b86e-419a4a3b117c"
Sep 18 20:01:19.300199 ip-10-0-130-81 kubenswrapper[2532]: E0918 20:01:19.300142    2532 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gd9qz" podUID="b377a9e6-dbbb-4023-8997-70178a63034f"
Sep 18 20:01:19.910814 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:19.910776    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/96167e3f-fb44-48e0-b86e-419a4a3b117c-metrics-certs\") pod \"network-metrics-daemon-xx7f7\" (UID: \"96167e3f-fb44-48e0-b86e-419a4a3b117c\") " pod="openshift-multus/network-metrics-daemon-xx7f7"
Sep 18 20:01:19.911226 ip-10-0-130-81 kubenswrapper[2532]: E0918 20:01:19.910945    2532 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 18 20:01:19.911226 ip-10-0-130-81 kubenswrapper[2532]: E0918 20:01:19.911018    2532 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/96167e3f-fb44-48e0-b86e-419a4a3b117c-metrics-certs podName:96167e3f-fb44-48e0-b86e-419a4a3b117c nodeName:}" failed. No retries permitted until 2026-09-18 20:01:27.910999724 +0000 UTC m=+19.676389103 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/96167e3f-fb44-48e0-b86e-419a4a3b117c-metrics-certs") pod "network-metrics-daemon-xx7f7" (UID: "96167e3f-fb44-48e0-b86e-419a4a3b117c") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 18 20:01:20.011575 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:20.011539    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b377a9e6-dbbb-4023-8997-70178a63034f-original-pull-secret\") pod \"global-pull-secret-syncer-gd9qz\" (UID: \"b377a9e6-dbbb-4023-8997-70178a63034f\") " pod="kube-system/global-pull-secret-syncer-gd9qz"
Sep 18 20:01:20.011758 ip-10-0-130-81 kubenswrapper[2532]: E0918 20:01:20.011741    2532 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 18 20:01:20.011846 ip-10-0-130-81 kubenswrapper[2532]: E0918 20:01:20.011805    2532 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b377a9e6-dbbb-4023-8997-70178a63034f-original-pull-secret podName:b377a9e6-dbbb-4023-8997-70178a63034f nodeName:}" failed. No retries permitted until 2026-09-18 20:01:28.011786779 +0000 UTC m=+19.777176170 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b377a9e6-dbbb-4023-8997-70178a63034f-original-pull-secret") pod "global-pull-secret-syncer-gd9qz" (UID: "b377a9e6-dbbb-4023-8997-70178a63034f") : object "kube-system"/"original-pull-secret" not registered
Sep 18 20:01:20.112305 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:20.112251    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-slbwq\" (UniqueName: \"kubernetes.io/projected/a966f963-b4ae-46ba-b206-2a27ad401463-kube-api-access-slbwq\") pod \"network-check-target-gd24n\" (UID: \"a966f963-b4ae-46ba-b206-2a27ad401463\") " pod="openshift-network-diagnostics/network-check-target-gd24n"
Sep 18 20:01:20.112477 ip-10-0-130-81 kubenswrapper[2532]: E0918 20:01:20.112427    2532 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 18 20:01:20.112477 ip-10-0-130-81 kubenswrapper[2532]: E0918 20:01:20.112448    2532 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 18 20:01:20.112477 ip-10-0-130-81 kubenswrapper[2532]: E0918 20:01:20.112460    2532 projected.go:196] Error preparing data for projected volume kube-api-access-slbwq for pod openshift-network-diagnostics/network-check-target-gd24n: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 18 20:01:20.112675 ip-10-0-130-81 kubenswrapper[2532]: E0918 20:01:20.112533    2532 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a966f963-b4ae-46ba-b206-2a27ad401463-kube-api-access-slbwq podName:a966f963-b4ae-46ba-b206-2a27ad401463 nodeName:}" failed. No retries permitted until 2026-09-18 20:01:28.112515722 +0000 UTC m=+19.877905104 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-slbwq" (UniqueName: "kubernetes.io/projected/a966f963-b4ae-46ba-b206-2a27ad401463-kube-api-access-slbwq") pod "network-check-target-gd24n" (UID: "a966f963-b4ae-46ba-b206-2a27ad401463") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 18 20:01:20.300764 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:20.300731    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gd24n"
Sep 18 20:01:20.300932 ip-10-0-130-81 kubenswrapper[2532]: E0918 20:01:20.300860    2532 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gd24n" podUID="a966f963-b4ae-46ba-b206-2a27ad401463"
Sep 18 20:01:21.300324 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:21.300288    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gd9qz"
Sep 18 20:01:21.300324 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:21.300310    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xx7f7"
Sep 18 20:01:21.300789 ip-10-0-130-81 kubenswrapper[2532]: E0918 20:01:21.300398    2532 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gd9qz" podUID="b377a9e6-dbbb-4023-8997-70178a63034f"
Sep 18 20:01:21.300789 ip-10-0-130-81 kubenswrapper[2532]: E0918 20:01:21.300554    2532 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xx7f7" podUID="96167e3f-fb44-48e0-b86e-419a4a3b117c"
Sep 18 20:01:22.304144 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:22.304113    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gd24n"
Sep 18 20:01:22.304585 ip-10-0-130-81 kubenswrapper[2532]: E0918 20:01:22.304246    2532 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gd24n" podUID="a966f963-b4ae-46ba-b206-2a27ad401463"
Sep 18 20:01:23.299984 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:23.299956    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gd9qz"
Sep 18 20:01:23.300085 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:23.299959    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xx7f7"
Sep 18 20:01:23.300156 ip-10-0-130-81 kubenswrapper[2532]: E0918 20:01:23.300080    2532 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gd9qz" podUID="b377a9e6-dbbb-4023-8997-70178a63034f"
Sep 18 20:01:23.300156 ip-10-0-130-81 kubenswrapper[2532]: E0918 20:01:23.300127    2532 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xx7f7" podUID="96167e3f-fb44-48e0-b86e-419a4a3b117c"
Sep 18 20:01:24.299714 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:24.299672    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gd24n"
Sep 18 20:01:24.300242 ip-10-0-130-81 kubenswrapper[2532]: E0918 20:01:24.299808    2532 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gd24n" podUID="a966f963-b4ae-46ba-b206-2a27ad401463"
Sep 18 20:01:25.299905 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:25.299871    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xx7f7"
Sep 18 20:01:25.300359 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:25.299872    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gd9qz"
Sep 18 20:01:25.300359 ip-10-0-130-81 kubenswrapper[2532]: E0918 20:01:25.299996    2532 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xx7f7" podUID="96167e3f-fb44-48e0-b86e-419a4a3b117c"
Sep 18 20:01:25.300359 ip-10-0-130-81 kubenswrapper[2532]: E0918 20:01:25.300070    2532 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gd9qz" podUID="b377a9e6-dbbb-4023-8997-70178a63034f"
Sep 18 20:01:26.300823 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:26.300787    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gd24n"
Sep 18 20:01:26.301240 ip-10-0-130-81 kubenswrapper[2532]: E0918 20:01:26.300918    2532 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gd24n" podUID="a966f963-b4ae-46ba-b206-2a27ad401463"
Sep 18 20:01:27.299831 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:27.299795    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gd9qz"
Sep 18 20:01:27.300119 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:27.299840    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xx7f7"
Sep 18 20:01:27.300119 ip-10-0-130-81 kubenswrapper[2532]: E0918 20:01:27.299937    2532 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gd9qz" podUID="b377a9e6-dbbb-4023-8997-70178a63034f"
Sep 18 20:01:27.300119 ip-10-0-130-81 kubenswrapper[2532]: E0918 20:01:27.300063    2532 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xx7f7" podUID="96167e3f-fb44-48e0-b86e-419a4a3b117c"
Sep 18 20:01:27.965199 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:27.965159    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/96167e3f-fb44-48e0-b86e-419a4a3b117c-metrics-certs\") pod \"network-metrics-daemon-xx7f7\" (UID: \"96167e3f-fb44-48e0-b86e-419a4a3b117c\") " pod="openshift-multus/network-metrics-daemon-xx7f7"
Sep 18 20:01:27.965674 ip-10-0-130-81 kubenswrapper[2532]: E0918 20:01:27.965326    2532 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 18 20:01:27.965674 ip-10-0-130-81 kubenswrapper[2532]: E0918 20:01:27.965408    2532 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/96167e3f-fb44-48e0-b86e-419a4a3b117c-metrics-certs podName:96167e3f-fb44-48e0-b86e-419a4a3b117c nodeName:}" failed. No retries permitted until 2026-09-18 20:01:43.96538888 +0000 UTC m=+35.730778275 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/96167e3f-fb44-48e0-b86e-419a4a3b117c-metrics-certs") pod "network-metrics-daemon-xx7f7" (UID: "96167e3f-fb44-48e0-b86e-419a4a3b117c") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 18 20:01:28.066251 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:28.066214    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b377a9e6-dbbb-4023-8997-70178a63034f-original-pull-secret\") pod \"global-pull-secret-syncer-gd9qz\" (UID: \"b377a9e6-dbbb-4023-8997-70178a63034f\") " pod="kube-system/global-pull-secret-syncer-gd9qz"
Sep 18 20:01:28.066408 ip-10-0-130-81 kubenswrapper[2532]: E0918 20:01:28.066372    2532 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 18 20:01:28.066462 ip-10-0-130-81 kubenswrapper[2532]: E0918 20:01:28.066451    2532 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b377a9e6-dbbb-4023-8997-70178a63034f-original-pull-secret podName:b377a9e6-dbbb-4023-8997-70178a63034f nodeName:}" failed. No retries permitted until 2026-09-18 20:01:44.06643239 +0000 UTC m=+35.831821786 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b377a9e6-dbbb-4023-8997-70178a63034f-original-pull-secret") pod "global-pull-secret-syncer-gd9qz" (UID: "b377a9e6-dbbb-4023-8997-70178a63034f") : object "kube-system"/"original-pull-secret" not registered
Sep 18 20:01:28.167347 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:28.167303    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-slbwq\" (UniqueName: \"kubernetes.io/projected/a966f963-b4ae-46ba-b206-2a27ad401463-kube-api-access-slbwq\") pod \"network-check-target-gd24n\" (UID: \"a966f963-b4ae-46ba-b206-2a27ad401463\") " pod="openshift-network-diagnostics/network-check-target-gd24n"
Sep 18 20:01:28.167541 ip-10-0-130-81 kubenswrapper[2532]: E0918 20:01:28.167507    2532 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 18 20:01:28.167541 ip-10-0-130-81 kubenswrapper[2532]: E0918 20:01:28.167535    2532 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 18 20:01:28.167638 ip-10-0-130-81 kubenswrapper[2532]: E0918 20:01:28.167549    2532 projected.go:196] Error preparing data for projected volume kube-api-access-slbwq for pod openshift-network-diagnostics/network-check-target-gd24n: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 18 20:01:28.167638 ip-10-0-130-81 kubenswrapper[2532]: E0918 20:01:28.167610    2532 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a966f963-b4ae-46ba-b206-2a27ad401463-kube-api-access-slbwq podName:a966f963-b4ae-46ba-b206-2a27ad401463 nodeName:}" failed. No retries permitted until 2026-09-18 20:01:44.167588989 +0000 UTC m=+35.932978388 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-slbwq" (UniqueName: "kubernetes.io/projected/a966f963-b4ae-46ba-b206-2a27ad401463-kube-api-access-slbwq") pod "network-check-target-gd24n" (UID: "a966f963-b4ae-46ba-b206-2a27ad401463") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 18 20:01:28.300603 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:28.300530    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gd24n"
Sep 18 20:01:28.300749 ip-10-0-130-81 kubenswrapper[2532]: E0918 20:01:28.300663    2532 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gd24n" podUID="a966f963-b4ae-46ba-b206-2a27ad401463"
Sep 18 20:01:29.300703 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:29.300662    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gd9qz"
Sep 18 20:01:29.301173 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:29.300677    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xx7f7"
Sep 18 20:01:29.301173 ip-10-0-130-81 kubenswrapper[2532]: E0918 20:01:29.300804    2532 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gd9qz" podUID="b377a9e6-dbbb-4023-8997-70178a63034f"
Sep 18 20:01:29.301173 ip-10-0-130-81 kubenswrapper[2532]: E0918 20:01:29.300879    2532 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xx7f7" podUID="96167e3f-fb44-48e0-b86e-419a4a3b117c"
Sep 18 20:01:30.301076 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:30.301046    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gd24n"
Sep 18 20:01:30.301542 ip-10-0-130-81 kubenswrapper[2532]: E0918 20:01:30.301140    2532 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gd24n" podUID="a966f963-b4ae-46ba-b206-2a27ad401463"
Sep 18 20:01:31.300720 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:31.300542    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gd9qz"
Sep 18 20:01:31.300854 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:31.300542    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xx7f7"
Sep 18 20:01:31.300854 ip-10-0-130-81 kubenswrapper[2532]: E0918 20:01:31.300822    2532 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gd9qz" podUID="b377a9e6-dbbb-4023-8997-70178a63034f"
Sep 18 20:01:31.300953 ip-10-0-130-81 kubenswrapper[2532]: E0918 20:01:31.300862    2532 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xx7f7" podUID="96167e3f-fb44-48e0-b86e-419a4a3b117c"
Sep 18 20:01:31.504745 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:31.504717    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-wkwnc" event={"ID":"b771ff0e-edf5-4d81-adf7-6350fe1ea781","Type":"ContainerStarted","Data":"19a02f557817570b65e94285313e53be30d5377d68281adeda03f53f036091fe"}
Sep 18 20:01:31.506862 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:31.506836    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nbw5k" event={"ID":"7f063a5b-9649-46fc-8278-3f8bb49700ed","Type":"ContainerStarted","Data":"2232127886c42c46a2d0655ee746aaec97023d7c417c1ae3e2b296406397170d"}
Sep 18 20:01:31.508606 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:31.508584    2532 generic.go:356] "Generic (PLEG): container finished" podID="5c8c76fd-1219-40d0-b010-cd89d714a3fe" containerID="dd799cfff8c1c9c9c8f079221d0e3e42a0586fa9741861b8319ce114b4be1106" exitCode=0
Sep 18 20:01:31.508699 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:31.508650    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-pn4rk" event={"ID":"5c8c76fd-1219-40d0-b010-cd89d714a3fe","Type":"ContainerDied","Data":"dd799cfff8c1c9c9c8f079221d0e3e42a0586fa9741861b8319ce114b4be1106"}
Sep 18 20:01:31.510399 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:31.510378    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-b8jbz" event={"ID":"87c4bfcc-4ad2-4166-b5c9-1051bdd2717d","Type":"ContainerStarted","Data":"44bf44bf37f1d85f3dc4f27525b1e87d37df85ae2e509f22391d82b703eba2f9"}
Sep 18 20:01:31.514006 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:31.513979    2532 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rtw9n_73fbf16b-3549-4a6c-a959-b633237ebd0c/ovn-acl-logging/0.log"
Sep 18 20:01:31.517209 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:31.517132    2532 generic.go:356] "Generic (PLEG): container finished" podID="73fbf16b-3549-4a6c-a959-b633237ebd0c" containerID="0b333eb2a1c4f6c8c92644ab2c2c05a8674c4cf9b27ace122b1a584b163331b7" exitCode=1
Sep 18 20:01:31.517292 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:31.517248    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-rtw9n" event={"ID":"73fbf16b-3549-4a6c-a959-b633237ebd0c","Type":"ContainerStarted","Data":"92b12a0338db8802c7aeac3c94ff5f7c85b09bdc4662e00c85a46092c14f06f0"}
Sep 18 20:01:31.517346 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:31.517318    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-rtw9n" event={"ID":"73fbf16b-3549-4a6c-a959-b633237ebd0c","Type":"ContainerStarted","Data":"3645cfe63332ea974e92b4a67d3b96df4d066fcc1c35a2a91ce5297648a33310"}
Sep 18 20:01:31.517548 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:31.517391    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-rtw9n" event={"ID":"73fbf16b-3549-4a6c-a959-b633237ebd0c","Type":"ContainerStarted","Data":"c162ba9aed23cdd5decb7914df4c26b76d29f77f799946df07c2f1a96810714e"}
Sep 18 20:01:31.517548 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:31.517445    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-rtw9n" event={"ID":"73fbf16b-3549-4a6c-a959-b633237ebd0c","Type":"ContainerDied","Data":"0b333eb2a1c4f6c8c92644ab2c2c05a8674c4cf9b27ace122b1a584b163331b7"}
Sep 18 20:01:31.517548 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:31.517536    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-rtw9n" event={"ID":"73fbf16b-3549-4a6c-a959-b633237ebd0c","Type":"ContainerStarted","Data":"ab26af85ec58c568aa13b312e947b46efa6d0d7d39ab8f5bc9fed9ebc6566c47"}
Sep 18 20:01:31.519781 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:31.519644    2532 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-wkwnc" podStartSLOduration=4.112345329 podStartE2EDuration="21.519628009s" totalImagesPullingTime="17.40728268s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-18 20:01:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 20:01:31.519569642 +0000 UTC m=+23.284959042" watchObservedRunningTime="2026-09-18 20:01:31.519628009 +0000 UTC m=+23.285017409"
Sep 18 20:01:31.521180 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:31.521158    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-jbmm2" event={"ID":"bf9010d7-90b5-4f95-8dea-8af5839c0f87","Type":"ContainerStarted","Data":"13faf6efc0ed2e3b30b03db3891d36f0f9df95b8c6c5234bbd822fea2cb3dd0a"}
Sep 18 20:01:31.523337 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:31.523315    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-spc6q" event={"ID":"6b3e5758-70d0-496d-90a9-35d371e8e1b6","Type":"ContainerStarted","Data":"3bd9ccdc07a5f3040b3c34a3e176edc495c1020437cb427e75b36e889738397e"}
Sep 18 20:01:31.525017 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:31.524995    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-g749g" event={"ID":"d94756ff-5c71-43bc-ab39-b0fde15a36a2","Type":"ContainerStarted","Data":"a389a8ae7ad7f05e89e2be4c78f4814d93dccf3cf9d3355a960e5c9723e4dbeb"}
Sep 18 20:01:31.535588 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:31.535531    2532 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-b8jbz" podStartSLOduration=4.120279173 podStartE2EDuration="21.535519198s" totalImagesPullingTime="17.415240025s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-18 20:01:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 20:01:31.535025527 +0000 UTC m=+23.300414925" watchObservedRunningTime="2026-09-18 20:01:31.535519198 +0000 UTC m=+23.300908597"
Sep 18 20:01:31.581039 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:31.580997    2532 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-spc6q" podStartSLOduration=4.169311312 podStartE2EDuration="21.580981754s" totalImagesPullingTime="17.411670442s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-18 20:01:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 20:01:31.580599469 +0000 UTC m=+23.345988869" watchObservedRunningTime="2026-09-18 20:01:31.580981754 +0000 UTC m=+23.346371153"
Sep 18 20:01:31.596598 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:31.596550    2532 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-jbmm2" podStartSLOduration=4.144059011 podStartE2EDuration="21.596536216s" totalImagesPullingTime="17.452477205s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-18 20:01:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 20:01:31.596363738 +0000 UTC m=+23.361753136" watchObservedRunningTime="2026-09-18 20:01:31.596536216 +0000 UTC m=+23.361925614"
Sep 18 20:01:31.610960 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:31.610914    2532 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-g749g" podStartSLOduration=8.398648698 podStartE2EDuration="21.610898501s" totalImagesPullingTime="13.212249803s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 20:01:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 20:01:31.610395596 +0000 UTC m=+23.375784994" watchObservedRunningTime="2026-09-18 20:01:31.610898501 +0000 UTC m=+23.376287900"
Sep 18 20:01:31.667440 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:31.667401    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:01:31.730958 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:31.730878    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:01:31.831426 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:31.831396    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:01:32.106066 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:32.106032    2532 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Sep 18 20:01:32.300432 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:32.300358    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gd24n"
Sep 18 20:01:32.300606 ip-10-0-130-81 kubenswrapper[2532]: E0918 20:01:32.300481    2532 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gd24n" podUID="a966f963-b4ae-46ba-b206-2a27ad401463"
Sep 18 20:01:32.301343 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:32.301251    2532 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-09-18T20:01:32.106062027Z","UUID":"dc642cfd-df8f-49b7-bbf9-c0409bfabc1d","Handler":null,"Name":"","Endpoint":""}
Sep 18 20:01:32.302926 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:32.302891    2532 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Sep 18 20:01:32.303088 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:32.303071    2532 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Sep 18 20:01:32.529962 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:32.529939    2532 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rtw9n_73fbf16b-3549-4a6c-a959-b633237ebd0c/ovn-acl-logging/0.log"
Sep 18 20:01:32.530386 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:32.530322    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-rtw9n" event={"ID":"73fbf16b-3549-4a6c-a959-b633237ebd0c","Type":"ContainerStarted","Data":"9bc13d0b16a4baa04e08c7cc44043ba6edc562dd3de4e513f3a3d35908a178d1"}
Sep 18 20:01:32.531781 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:32.531758    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-dxcqj" event={"ID":"4dc79459-39e4-4afe-b537-b657780854fb","Type":"ContainerStarted","Data":"b74737e60ffd1be72165c234a755d92e78ecaa4dd9a3ed5e42331da036e736b5"}
Sep 18 20:01:32.533476 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:32.533451    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nbw5k" event={"ID":"7f063a5b-9649-46fc-8278-3f8bb49700ed","Type":"ContainerStarted","Data":"6f76a01598c9bec506d61fbea8c799b8b4362231ce093e0b87bd07365bdab530"}
Sep 18 20:01:32.545565 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:32.545519    2532 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-dxcqj" podStartSLOduration=5.137780394 podStartE2EDuration="22.545501672s" totalImagesPullingTime="17.407721278s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-18 20:01:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 20:01:32.545259998 +0000 UTC m=+24.310649398" watchObservedRunningTime="2026-09-18 20:01:32.545501672 +0000 UTC m=+24.310891065"
Sep 18 20:01:32.664067 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:32.663985    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:01:32.764502 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:32.764445    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:01:33.300032 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:33.299996    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gd9qz"
Sep 18 20:01:33.300217 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:33.300001    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xx7f7"
Sep 18 20:01:33.300217 ip-10-0-130-81 kubenswrapper[2532]: E0918 20:01:33.300122    2532 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gd9qz" podUID="b377a9e6-dbbb-4023-8997-70178a63034f"
Sep 18 20:01:33.300334 ip-10-0-130-81 kubenswrapper[2532]: E0918 20:01:33.300222    2532 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xx7f7" podUID="96167e3f-fb44-48e0-b86e-419a4a3b117c"
Sep 18 20:01:33.538437 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:33.538405    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nbw5k" event={"ID":"7f063a5b-9649-46fc-8278-3f8bb49700ed","Type":"ContainerStarted","Data":"fe9d991b724cf11eaed45fa189dda4853dcf910d79e5c0a6fdae1751ce3be90e"}
Sep 18 20:01:33.559302 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:33.559213    2532 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nbw5k" podStartSLOduration=3.9276945530000003 podStartE2EDuration="23.559195772s" totalImagesPullingTime="19.631501219s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-18 20:01:10 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 20:01:33.559194814 +0000 UTC m=+25.324584214" watchObservedRunningTime="2026-09-18 20:01:33.559195772 +0000 UTC m=+25.324585171"
Sep 18 20:01:33.807203 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:33.807024    2532 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-g749g"
Sep 18 20:01:33.807637 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:33.807612    2532 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-g749g"
Sep 18 20:01:34.300541 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:34.300509    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gd24n"
Sep 18 20:01:34.300739 ip-10-0-130-81 kubenswrapper[2532]: E0918 20:01:34.300637    2532 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gd24n" podUID="a966f963-b4ae-46ba-b206-2a27ad401463"
Sep 18 20:01:34.543762 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:34.543736    2532 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rtw9n_73fbf16b-3549-4a6c-a959-b633237ebd0c/ovn-acl-logging/0.log"
Sep 18 20:01:34.544200 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:34.544173    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-rtw9n" event={"ID":"73fbf16b-3549-4a6c-a959-b633237ebd0c","Type":"ContainerStarted","Data":"9c1a4afc0595c06fb258aa17a2c4da3c803c78d4e72516e9ac097a3ed32f5051"}
Sep 18 20:01:34.544437 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:34.544417    2532 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-g749g"
Sep 18 20:01:34.544980 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:34.544964    2532 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-g749g"
Sep 18 20:01:34.604010 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:34.603919    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:01:35.300031 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:35.300005    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xx7f7"
Sep 18 20:01:35.300222 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:35.300005    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gd9qz"
Sep 18 20:01:35.300222 ip-10-0-130-81 kubenswrapper[2532]: E0918 20:01:35.300143    2532 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xx7f7" podUID="96167e3f-fb44-48e0-b86e-419a4a3b117c"
Sep 18 20:01:35.300368 ip-10-0-130-81 kubenswrapper[2532]: E0918 20:01:35.300228    2532 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gd9qz" podUID="b377a9e6-dbbb-4023-8997-70178a63034f"
Sep 18 20:01:35.438181 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:35.438145    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:01:35.490308 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:35.490275    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:01:36.300470 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:36.300439    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gd24n"
Sep 18 20:01:36.301001 ip-10-0-130-81 kubenswrapper[2532]: E0918 20:01:36.300555    2532 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gd24n" podUID="a966f963-b4ae-46ba-b206-2a27ad401463"
Sep 18 20:01:36.507862 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:36.507701    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:01:36.555008 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:36.554941    2532 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rtw9n_73fbf16b-3549-4a6c-a959-b633237ebd0c/ovn-acl-logging/0.log"
Sep 18 20:01:36.555554 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:36.555520    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-rtw9n" event={"ID":"73fbf16b-3549-4a6c-a959-b633237ebd0c","Type":"ContainerStarted","Data":"80b654ec9faafd57d2bb9ea9a6ea8cff31322d2c89b4612149d3da07eb867d52"}
Sep 18 20:01:36.556012 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:36.555992    2532 scope.go:117] "RemoveContainer" containerID="0b333eb2a1c4f6c8c92644ab2c2c05a8674c4cf9b27ace122b1a584b163331b7"
Sep 18 20:01:36.590910 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:36.590858    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:01:37.300017 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:37.299990    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xx7f7"
Sep 18 20:01:37.300163 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:37.299990    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gd9qz"
Sep 18 20:01:37.300163 ip-10-0-130-81 kubenswrapper[2532]: E0918 20:01:37.300094    2532 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xx7f7" podUID="96167e3f-fb44-48e0-b86e-419a4a3b117c"
Sep 18 20:01:37.300249 ip-10-0-130-81 kubenswrapper[2532]: E0918 20:01:37.300158    2532 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gd9qz" podUID="b377a9e6-dbbb-4023-8997-70178a63034f"
Sep 18 20:01:37.559289 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:37.559220    2532 generic.go:356] "Generic (PLEG): container finished" podID="5c8c76fd-1219-40d0-b010-cd89d714a3fe" containerID="b685370c8824bbd33f6d90f210a482010625e4b247226736d35863ff154e536b" exitCode=0
Sep 18 20:01:37.559981 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:37.559310    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-pn4rk" event={"ID":"5c8c76fd-1219-40d0-b010-cd89d714a3fe","Type":"ContainerDied","Data":"b685370c8824bbd33f6d90f210a482010625e4b247226736d35863ff154e536b"}
Sep 18 20:01:37.563230 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:37.563209    2532 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rtw9n_73fbf16b-3549-4a6c-a959-b633237ebd0c/ovn-acl-logging/0.log"
Sep 18 20:01:37.563611 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:37.563587    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-rtw9n" event={"ID":"73fbf16b-3549-4a6c-a959-b633237ebd0c","Type":"ContainerStarted","Data":"e191e6cf3eb269172a101d4fdbac811035ea5bad6bae3d7b8606942d9de68531"}
Sep 18 20:01:37.563862 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:37.563844    2532 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-rtw9n"
Sep 18 20:01:37.563946 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:37.563905    2532 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-rtw9n"
Sep 18 20:01:37.564001 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:37.563952    2532 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-rtw9n"
Sep 18 20:01:37.578759 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:37.578736    2532 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-rtw9n"
Sep 18 20:01:37.578884 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:37.578871    2532 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-rtw9n"
Sep 18 20:01:37.605292 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:37.605241    2532 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-rtw9n" podStartSLOduration=10.173522628 podStartE2EDuration="27.605224614s" totalImagesPullingTime="17.431701986s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-18 20:01:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 20:01:37.604873688 +0000 UTC m=+29.370263088" watchObservedRunningTime="2026-09-18 20:01:37.605224614 +0000 UTC m=+29.370614010"
Sep 18 20:01:38.284243 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:38.284214    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-gd24n"]
Sep 18 20:01:38.284380 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:38.284309    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gd24n"
Sep 18 20:01:38.284455 ip-10-0-130-81 kubenswrapper[2532]: E0918 20:01:38.284435    2532 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gd24n" podUID="a966f963-b4ae-46ba-b206-2a27ad401463"
Sep 18 20:01:38.287562 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:38.287534    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-gd9qz"]
Sep 18 20:01:38.287694 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:38.287623    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gd9qz"
Sep 18 20:01:38.287758 ip-10-0-130-81 kubenswrapper[2532]: E0918 20:01:38.287723    2532 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gd9qz" podUID="b377a9e6-dbbb-4023-8997-70178a63034f"
Sep 18 20:01:38.287986 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:38.287966    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-xx7f7"]
Sep 18 20:01:38.288072 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:38.288047    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xx7f7"
Sep 18 20:01:38.288145 ip-10-0-130-81 kubenswrapper[2532]: E0918 20:01:38.288130    2532 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xx7f7" podUID="96167e3f-fb44-48e0-b86e-419a4a3b117c"
Sep 18 20:01:39.570244 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:39.569990    2532 generic.go:356] "Generic (PLEG): container finished" podID="5c8c76fd-1219-40d0-b010-cd89d714a3fe" containerID="4f14edd3eb7065ee25aabdcafe91d8b14cd13b31061b9c7f4aff3bc37ef4a77e" exitCode=0
Sep 18 20:01:39.570648 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:39.570062    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-pn4rk" event={"ID":"5c8c76fd-1219-40d0-b010-cd89d714a3fe","Type":"ContainerDied","Data":"4f14edd3eb7065ee25aabdcafe91d8b14cd13b31061b9c7f4aff3bc37ef4a77e"}
Sep 18 20:01:40.300746 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:40.300712    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gd24n"
Sep 18 20:01:40.300908 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:40.300793    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xx7f7"
Sep 18 20:01:40.300908 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:40.300828    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gd9qz"
Sep 18 20:01:40.300908 ip-10-0-130-81 kubenswrapper[2532]: E0918 20:01:40.300855    2532 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gd24n" podUID="a966f963-b4ae-46ba-b206-2a27ad401463"
Sep 18 20:01:40.301058 ip-10-0-130-81 kubenswrapper[2532]: E0918 20:01:40.300909    2532 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gd9qz" podUID="b377a9e6-dbbb-4023-8997-70178a63034f"
Sep 18 20:01:40.301058 ip-10-0-130-81 kubenswrapper[2532]: E0918 20:01:40.300975    2532 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xx7f7" podUID="96167e3f-fb44-48e0-b86e-419a4a3b117c"
Sep 18 20:01:41.579925 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:41.579893    2532 generic.go:356] "Generic (PLEG): container finished" podID="5c8c76fd-1219-40d0-b010-cd89d714a3fe" containerID="8615d94be1f4f2d1bcdad6b3377add490ef2bbdfd3d6d444e88d20acb2635733" exitCode=0
Sep 18 20:01:41.580612 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:41.579940    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-pn4rk" event={"ID":"5c8c76fd-1219-40d0-b010-cd89d714a3fe","Type":"ContainerDied","Data":"8615d94be1f4f2d1bcdad6b3377add490ef2bbdfd3d6d444e88d20acb2635733"}
Sep 18 20:01:42.299771 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:42.299739    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xx7f7"
Sep 18 20:01:42.299771 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:42.299772    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gd24n"
Sep 18 20:01:42.299950 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:42.299742    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gd9qz"
Sep 18 20:01:42.299950 ip-10-0-130-81 kubenswrapper[2532]: E0918 20:01:42.299858    2532 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xx7f7" podUID="96167e3f-fb44-48e0-b86e-419a4a3b117c"
Sep 18 20:01:42.299950 ip-10-0-130-81 kubenswrapper[2532]: E0918 20:01:42.299934    2532 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gd9qz" podUID="b377a9e6-dbbb-4023-8997-70178a63034f"
Sep 18 20:01:42.300065 ip-10-0-130-81 kubenswrapper[2532]: E0918 20:01:42.299998    2532 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gd24n" podUID="a966f963-b4ae-46ba-b206-2a27ad401463"
Sep 18 20:01:43.765108 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:43.765079    2532 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-81.ec2.internal" event="NodeReady"
Sep 18 20:01:43.765786 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:43.765231    2532 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Sep 18 20:01:43.807936 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:43.807863    2532 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-l8bvt"]
Sep 18 20:01:43.827474 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:43.827444    2532 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-qd64x"]
Sep 18 20:01:43.827621 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:43.827543    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-l8bvt"
Sep 18 20:01:43.830054 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:43.830029    2532 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Sep 18 20:01:43.830054 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:43.830049    2532 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-dw2ht\""
Sep 18 20:01:43.830251 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:43.830050    2532 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Sep 18 20:01:43.847436 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:43.847412    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-l8bvt"]
Sep 18 20:01:43.847577 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:43.847445    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-qd64x"]
Sep 18 20:01:43.847577 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:43.847548    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-qd64x"
Sep 18 20:01:43.849966 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:43.849913    2532 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Sep 18 20:01:43.850077 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:43.850028    2532 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-xpvnc\""
Sep 18 20:01:43.850077 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:43.850048    2532 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Sep 18 20:01:43.850191 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:43.850084    2532 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Sep 18 20:01:43.923858 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:43.923825    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:01:43.989751 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:43.989721    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/60bb026c-35d3-4f8b-bdb8-fb0b698c024c-cert\") pod \"ingress-canary-qd64x\" (UID: \"60bb026c-35d3-4f8b-bdb8-fb0b698c024c\") " pod="openshift-ingress-canary/ingress-canary-qd64x"
Sep 18 20:01:43.989925 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:43.989757    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/96167e3f-fb44-48e0-b86e-419a4a3b117c-metrics-certs\") pod \"network-metrics-daemon-xx7f7\" (UID: \"96167e3f-fb44-48e0-b86e-419a4a3b117c\") " pod="openshift-multus/network-metrics-daemon-xx7f7"
Sep 18 20:01:43.989925 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:43.989785    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/13532b96-fcd8-4f12-a4df-b4041abe4cf3-tmp-dir\") pod \"dns-default-l8bvt\" (UID: \"13532b96-fcd8-4f12-a4df-b4041abe4cf3\") " pod="openshift-dns/dns-default-l8bvt"
Sep 18 20:01:43.989925 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:43.989802    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mcxs6\" (UniqueName: \"kubernetes.io/projected/13532b96-fcd8-4f12-a4df-b4041abe4cf3-kube-api-access-mcxs6\") pod \"dns-default-l8bvt\" (UID: \"13532b96-fcd8-4f12-a4df-b4041abe4cf3\") " pod="openshift-dns/dns-default-l8bvt"
Sep 18 20:01:43.989925 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:43.989828    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/13532b96-fcd8-4f12-a4df-b4041abe4cf3-metrics-tls\") pod \"dns-default-l8bvt\" (UID: \"13532b96-fcd8-4f12-a4df-b4041abe4cf3\") " pod="openshift-dns/dns-default-l8bvt"
Sep 18 20:01:43.989925 ip-10-0-130-81 kubenswrapper[2532]: E0918 20:01:43.989910    2532 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 18 20:01:43.990163 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:43.989961    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jxc2j\" (UniqueName: \"kubernetes.io/projected/60bb026c-35d3-4f8b-bdb8-fb0b698c024c-kube-api-access-jxc2j\") pod \"ingress-canary-qd64x\" (UID: \"60bb026c-35d3-4f8b-bdb8-fb0b698c024c\") " pod="openshift-ingress-canary/ingress-canary-qd64x"
Sep 18 20:01:43.990163 ip-10-0-130-81 kubenswrapper[2532]: E0918 20:01:43.989970    2532 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/96167e3f-fb44-48e0-b86e-419a4a3b117c-metrics-certs podName:96167e3f-fb44-48e0-b86e-419a4a3b117c nodeName:}" failed. No retries permitted until 2026-09-18 20:02:15.989956224 +0000 UTC m=+67.755345601 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/96167e3f-fb44-48e0-b86e-419a4a3b117c-metrics-certs") pod "network-metrics-daemon-xx7f7" (UID: "96167e3f-fb44-48e0-b86e-419a4a3b117c") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 18 20:01:43.990163 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:43.990015    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/13532b96-fcd8-4f12-a4df-b4041abe4cf3-config-volume\") pod \"dns-default-l8bvt\" (UID: \"13532b96-fcd8-4f12-a4df-b4041abe4cf3\") " pod="openshift-dns/dns-default-l8bvt"
Sep 18 20:01:44.024908 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:44.024873    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:01:44.090783 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:44.090703    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b377a9e6-dbbb-4023-8997-70178a63034f-original-pull-secret\") pod \"global-pull-secret-syncer-gd9qz\" (UID: \"b377a9e6-dbbb-4023-8997-70178a63034f\") " pod="kube-system/global-pull-secret-syncer-gd9qz"
Sep 18 20:01:44.090783 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:44.090740    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jxc2j\" (UniqueName: \"kubernetes.io/projected/60bb026c-35d3-4f8b-bdb8-fb0b698c024c-kube-api-access-jxc2j\") pod \"ingress-canary-qd64x\" (UID: \"60bb026c-35d3-4f8b-bdb8-fb0b698c024c\") " pod="openshift-ingress-canary/ingress-canary-qd64x"
Sep 18 20:01:44.090783 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:44.090770    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/13532b96-fcd8-4f12-a4df-b4041abe4cf3-config-volume\") pod \"dns-default-l8bvt\" (UID: \"13532b96-fcd8-4f12-a4df-b4041abe4cf3\") " pod="openshift-dns/dns-default-l8bvt"
Sep 18 20:01:44.091051 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:44.090801    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/60bb026c-35d3-4f8b-bdb8-fb0b698c024c-cert\") pod \"ingress-canary-qd64x\" (UID: \"60bb026c-35d3-4f8b-bdb8-fb0b698c024c\") " pod="openshift-ingress-canary/ingress-canary-qd64x"
Sep 18 20:01:44.091051 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:44.090830    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/13532b96-fcd8-4f12-a4df-b4041abe4cf3-tmp-dir\") pod \"dns-default-l8bvt\" (UID: \"13532b96-fcd8-4f12-a4df-b4041abe4cf3\") " pod="openshift-dns/dns-default-l8bvt"
Sep 18 20:01:44.091051 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:44.090852    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mcxs6\" (UniqueName: \"kubernetes.io/projected/13532b96-fcd8-4f12-a4df-b4041abe4cf3-kube-api-access-mcxs6\") pod \"dns-default-l8bvt\" (UID: \"13532b96-fcd8-4f12-a4df-b4041abe4cf3\") " pod="openshift-dns/dns-default-l8bvt"
Sep 18 20:01:44.091051 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:44.090872    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/13532b96-fcd8-4f12-a4df-b4041abe4cf3-metrics-tls\") pod \"dns-default-l8bvt\" (UID: \"13532b96-fcd8-4f12-a4df-b4041abe4cf3\") " pod="openshift-dns/dns-default-l8bvt"
Sep 18 20:01:44.091051 ip-10-0-130-81 kubenswrapper[2532]: E0918 20:01:44.090965    2532 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 18 20:01:44.091051 ip-10-0-130-81 kubenswrapper[2532]: E0918 20:01:44.091027    2532 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/13532b96-fcd8-4f12-a4df-b4041abe4cf3-metrics-tls podName:13532b96-fcd8-4f12-a4df-b4041abe4cf3 nodeName:}" failed. No retries permitted until 2026-09-18 20:01:44.591009112 +0000 UTC m=+36.356398489 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/13532b96-fcd8-4f12-a4df-b4041abe4cf3-metrics-tls") pod "dns-default-l8bvt" (UID: "13532b96-fcd8-4f12-a4df-b4041abe4cf3") : secret "dns-default-metrics-tls" not found
Sep 18 20:01:44.091051 ip-10-0-130-81 kubenswrapper[2532]: E0918 20:01:44.091027    2532 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 18 20:01:44.091360 ip-10-0-130-81 kubenswrapper[2532]: E0918 20:01:44.091083    2532 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b377a9e6-dbbb-4023-8997-70178a63034f-original-pull-secret podName:b377a9e6-dbbb-4023-8997-70178a63034f nodeName:}" failed. No retries permitted until 2026-09-18 20:02:16.091066532 +0000 UTC m=+67.856455913 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b377a9e6-dbbb-4023-8997-70178a63034f-original-pull-secret") pod "global-pull-secret-syncer-gd9qz" (UID: "b377a9e6-dbbb-4023-8997-70178a63034f") : object "kube-system"/"original-pull-secret" not registered
Sep 18 20:01:44.091360 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:44.091206    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/13532b96-fcd8-4f12-a4df-b4041abe4cf3-tmp-dir\") pod \"dns-default-l8bvt\" (UID: \"13532b96-fcd8-4f12-a4df-b4041abe4cf3\") " pod="openshift-dns/dns-default-l8bvt"
Sep 18 20:01:44.091476 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:44.091458    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/13532b96-fcd8-4f12-a4df-b4041abe4cf3-config-volume\") pod \"dns-default-l8bvt\" (UID: \"13532b96-fcd8-4f12-a4df-b4041abe4cf3\") " pod="openshift-dns/dns-default-l8bvt"
Sep 18 20:01:44.095531 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:44.095265    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/60bb026c-35d3-4f8b-bdb8-fb0b698c024c-cert\") pod \"ingress-canary-qd64x\" (UID: \"60bb026c-35d3-4f8b-bdb8-fb0b698c024c\") " pod="openshift-ingress-canary/ingress-canary-qd64x"
Sep 18 20:01:44.099620 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:44.099583    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jxc2j\" (UniqueName: \"kubernetes.io/projected/60bb026c-35d3-4f8b-bdb8-fb0b698c024c-kube-api-access-jxc2j\") pod \"ingress-canary-qd64x\" (UID: \"60bb026c-35d3-4f8b-bdb8-fb0b698c024c\") " pod="openshift-ingress-canary/ingress-canary-qd64x"
Sep 18 20:01:44.100396 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:44.100379    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mcxs6\" (UniqueName: \"kubernetes.io/projected/13532b96-fcd8-4f12-a4df-b4041abe4cf3-kube-api-access-mcxs6\") pod \"dns-default-l8bvt\" (UID: \"13532b96-fcd8-4f12-a4df-b4041abe4cf3\") " pod="openshift-dns/dns-default-l8bvt"
Sep 18 20:01:44.157346 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:44.157311    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-qd64x"
Sep 18 20:01:44.192119 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:44.192086    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-slbwq\" (UniqueName: \"kubernetes.io/projected/a966f963-b4ae-46ba-b206-2a27ad401463-kube-api-access-slbwq\") pod \"network-check-target-gd24n\" (UID: \"a966f963-b4ae-46ba-b206-2a27ad401463\") " pod="openshift-network-diagnostics/network-check-target-gd24n"
Sep 18 20:01:44.192301 ip-10-0-130-81 kubenswrapper[2532]: E0918 20:01:44.192278    2532 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 18 20:01:44.192346 ip-10-0-130-81 kubenswrapper[2532]: E0918 20:01:44.192314    2532 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 18 20:01:44.192346 ip-10-0-130-81 kubenswrapper[2532]: E0918 20:01:44.192330    2532 projected.go:196] Error preparing data for projected volume kube-api-access-slbwq for pod openshift-network-diagnostics/network-check-target-gd24n: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 18 20:01:44.192456 ip-10-0-130-81 kubenswrapper[2532]: E0918 20:01:44.192392    2532 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a966f963-b4ae-46ba-b206-2a27ad401463-kube-api-access-slbwq podName:a966f963-b4ae-46ba-b206-2a27ad401463 nodeName:}" failed. No retries permitted until 2026-09-18 20:02:16.192373813 +0000 UTC m=+67.957763205 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-slbwq" (UniqueName: "kubernetes.io/projected/a966f963-b4ae-46ba-b206-2a27ad401463-kube-api-access-slbwq") pod "network-check-target-gd24n" (UID: "a966f963-b4ae-46ba-b206-2a27ad401463") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 18 20:01:44.300500 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:44.300291    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gd9qz"
Sep 18 20:01:44.300697 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:44.300291    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gd24n"
Sep 18 20:01:44.301054 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:44.300298    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xx7f7"
Sep 18 20:01:44.303577 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:44.303320    2532 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 18 20:01:44.303577 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:44.303472    2532 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-ms6pd\""
Sep 18 20:01:44.303577 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:44.303554    2532 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-4pmvp\""
Sep 18 20:01:44.303802 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:44.303605    2532 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 18 20:01:44.303802 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:44.303731    2532 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 18 20:01:44.304153 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:44.304119    2532 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 18 20:01:44.374769 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:44.374671    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:01:44.437266 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:44.437235    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:01:44.498789 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:44.498755    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-qd64x"]
Sep 18 20:01:44.512142 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:44.512105    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:01:44.576177 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:44.576142    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:01:44.595546 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:44.595505    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/13532b96-fcd8-4f12-a4df-b4041abe4cf3-metrics-tls\") pod \"dns-default-l8bvt\" (UID: \"13532b96-fcd8-4f12-a4df-b4041abe4cf3\") " pod="openshift-dns/dns-default-l8bvt"
Sep 18 20:01:44.595710 ip-10-0-130-81 kubenswrapper[2532]: E0918 20:01:44.595646    2532 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 18 20:01:44.595760 ip-10-0-130-81 kubenswrapper[2532]: E0918 20:01:44.595723    2532 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/13532b96-fcd8-4f12-a4df-b4041abe4cf3-metrics-tls podName:13532b96-fcd8-4f12-a4df-b4041abe4cf3 nodeName:}" failed. No retries permitted until 2026-09-18 20:01:45.595702025 +0000 UTC m=+37.361091426 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/13532b96-fcd8-4f12-a4df-b4041abe4cf3-metrics-tls") pod "dns-default-l8bvt" (UID: "13532b96-fcd8-4f12-a4df-b4041abe4cf3") : secret "dns-default-metrics-tls" not found
Sep 18 20:01:45.390756 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:45.390699    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:01:45.483174 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:45.483067    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:01:45.544594 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:45.544537    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:01:45.602677 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:45.602639    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/13532b96-fcd8-4f12-a4df-b4041abe4cf3-metrics-tls\") pod \"dns-default-l8bvt\" (UID: \"13532b96-fcd8-4f12-a4df-b4041abe4cf3\") " pod="openshift-dns/dns-default-l8bvt"
Sep 18 20:01:45.602836 ip-10-0-130-81 kubenswrapper[2532]: E0918 20:01:45.602765    2532 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 18 20:01:45.602880 ip-10-0-130-81 kubenswrapper[2532]: E0918 20:01:45.602836    2532 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/13532b96-fcd8-4f12-a4df-b4041abe4cf3-metrics-tls podName:13532b96-fcd8-4f12-a4df-b4041abe4cf3 nodeName:}" failed. No retries permitted until 2026-09-18 20:01:47.60281577 +0000 UTC m=+39.368205147 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/13532b96-fcd8-4f12-a4df-b4041abe4cf3-metrics-tls") pod "dns-default-l8bvt" (UID: "13532b96-fcd8-4f12-a4df-b4041abe4cf3") : secret "dns-default-metrics-tls" not found
Sep 18 20:01:45.944266 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:01:45.944237    2532 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod60bb026c_35d3_4f8b_bdb8_fb0b698c024c.slice/crio-8a49c79db891853489f15491871375c778f9c540fcefe4f8fed70b59ff2b8194 WatchSource:0}: Error finding container 8a49c79db891853489f15491871375c778f9c540fcefe4f8fed70b59ff2b8194: Status 404 returned error can't find the container with id 8a49c79db891853489f15491871375c778f9c540fcefe4f8fed70b59ff2b8194
Sep 18 20:01:46.263608 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:46.263578    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:01:46.364172 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:46.364145    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:01:46.593824 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:46.593792    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-qd64x" event={"ID":"60bb026c-35d3-4f8b-bdb8-fb0b698c024c","Type":"ContainerStarted","Data":"8a49c79db891853489f15491871375c778f9c540fcefe4f8fed70b59ff2b8194"}
Sep 18 20:01:46.597260 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:46.597232    2532 generic.go:356] "Generic (PLEG): container finished" podID="5c8c76fd-1219-40d0-b010-cd89d714a3fe" containerID="432c89ca659990c32addc2016553ada995206f61fce4b8e0fc7fb271427aa82c" exitCode=0
Sep 18 20:01:46.597380 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:46.597269    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-pn4rk" event={"ID":"5c8c76fd-1219-40d0-b010-cd89d714a3fe","Type":"ContainerDied","Data":"432c89ca659990c32addc2016553ada995206f61fce4b8e0fc7fb271427aa82c"}
Sep 18 20:01:47.603009 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:47.602977    2532 generic.go:356] "Generic (PLEG): container finished" podID="5c8c76fd-1219-40d0-b010-cd89d714a3fe" containerID="0543a17787392ac7c212fe19c64639bdb224fa37fe046615926d8ac33e038cad" exitCode=0
Sep 18 20:01:47.603379 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:47.603039    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-pn4rk" event={"ID":"5c8c76fd-1219-40d0-b010-cd89d714a3fe","Type":"ContainerDied","Data":"0543a17787392ac7c212fe19c64639bdb224fa37fe046615926d8ac33e038cad"}
Sep 18 20:01:47.618214 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:47.618185    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/13532b96-fcd8-4f12-a4df-b4041abe4cf3-metrics-tls\") pod \"dns-default-l8bvt\" (UID: \"13532b96-fcd8-4f12-a4df-b4041abe4cf3\") " pod="openshift-dns/dns-default-l8bvt"
Sep 18 20:01:47.618348 ip-10-0-130-81 kubenswrapper[2532]: E0918 20:01:47.618330    2532 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 18 20:01:47.618404 ip-10-0-130-81 kubenswrapper[2532]: E0918 20:01:47.618397    2532 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/13532b96-fcd8-4f12-a4df-b4041abe4cf3-metrics-tls podName:13532b96-fcd8-4f12-a4df-b4041abe4cf3 nodeName:}" failed. No retries permitted until 2026-09-18 20:01:51.61838073 +0000 UTC m=+43.383770130 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/13532b96-fcd8-4f12-a4df-b4041abe4cf3-metrics-tls") pod "dns-default-l8bvt" (UID: "13532b96-fcd8-4f12-a4df-b4041abe4cf3") : secret "dns-default-metrics-tls" not found
Sep 18 20:01:48.042807 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:48.042770    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:01:48.096994 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:48.096954    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:01:48.607300 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:48.607269    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-qd64x" event={"ID":"60bb026c-35d3-4f8b-bdb8-fb0b698c024c","Type":"ContainerStarted","Data":"6d5f85545d2b8bffee8a6522efb284839a638be847ff953485c8f600a4f5030c"}
Sep 18 20:01:48.610435 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:48.610413    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-pn4rk" event={"ID":"5c8c76fd-1219-40d0-b010-cd89d714a3fe","Type":"ContainerStarted","Data":"c95c73deec7e74883312fd9bad01d84cdefcffffb1bbdb66237894f7882a4a40"}
Sep 18 20:01:48.642263 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:48.642214    2532 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-pn4rk" podStartSLOduration=9.386318937 podStartE2EDuration="38.642199172s" totalImagesPullingTime="29.255880235s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-18 20:01:10 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 20:01:48.641404473 +0000 UTC m=+40.406793873" watchObservedRunningTime="2026-09-18 20:01:48.642199172 +0000 UTC m=+40.407588562"
Sep 18 20:01:48.642499 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:48.642460    2532 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-qd64x" podStartSLOduration=3.3875764999999998 podStartE2EDuration="5.642452658s" totalImagesPullingTime="2.254876158s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 20:01:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 20:01:48.621726696 +0000 UTC m=+40.387116094" watchObservedRunningTime="2026-09-18 20:01:48.642452658 +0000 UTC m=+40.407842059"
Sep 18 20:01:48.761814 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:48.761775    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:01:48.819568 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:48.819529    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:01:51.650137 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:51.650099    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/13532b96-fcd8-4f12-a4df-b4041abe4cf3-metrics-tls\") pod \"dns-default-l8bvt\" (UID: \"13532b96-fcd8-4f12-a4df-b4041abe4cf3\") " pod="openshift-dns/dns-default-l8bvt"
Sep 18 20:01:51.650638 ip-10-0-130-81 kubenswrapper[2532]: E0918 20:01:51.650219    2532 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 18 20:01:51.650638 ip-10-0-130-81 kubenswrapper[2532]: E0918 20:01:51.650269    2532 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/13532b96-fcd8-4f12-a4df-b4041abe4cf3-metrics-tls podName:13532b96-fcd8-4f12-a4df-b4041abe4cf3 nodeName:}" failed. No retries permitted until 2026-09-18 20:01:59.650255302 +0000 UTC m=+51.415644678 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/13532b96-fcd8-4f12-a4df-b4041abe4cf3-metrics-tls") pod "dns-default-l8bvt" (UID: "13532b96-fcd8-4f12-a4df-b4041abe4cf3") : secret "dns-default-metrics-tls" not found
Sep 18 20:01:56.608304 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:56.608276    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:01:56.659686 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:56.659647    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:01:57.427028 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:57.426989    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:01:57.488636 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:57.488601    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:01:59.703224 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:01:59.703183    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/13532b96-fcd8-4f12-a4df-b4041abe4cf3-metrics-tls\") pod \"dns-default-l8bvt\" (UID: \"13532b96-fcd8-4f12-a4df-b4041abe4cf3\") " pod="openshift-dns/dns-default-l8bvt"
Sep 18 20:01:59.703697 ip-10-0-130-81 kubenswrapper[2532]: E0918 20:01:59.703334    2532 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 18 20:01:59.703697 ip-10-0-130-81 kubenswrapper[2532]: E0918 20:01:59.703433    2532 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/13532b96-fcd8-4f12-a4df-b4041abe4cf3-metrics-tls podName:13532b96-fcd8-4f12-a4df-b4041abe4cf3 nodeName:}" failed. No retries permitted until 2026-09-18 20:02:15.703413309 +0000 UTC m=+67.468802690 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/13532b96-fcd8-4f12-a4df-b4041abe4cf3-metrics-tls") pod "dns-default-l8bvt" (UID: "13532b96-fcd8-4f12-a4df-b4041abe4cf3") : secret "dns-default-metrics-tls" not found
Sep 18 20:02:00.562139 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:02:00.562097    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:02:00.659273 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:02:00.659233    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:02:00.896992 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:02:00.896908    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:02:00.997589 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:02:00.997536    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:02:09.580478 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:02:09.580449    2532 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-rtw9n"
Sep 18 20:02:09.729181 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:02:09.729150    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:02:09.788841 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:02:09.788810    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:02:15.716032 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:02:15.715999    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/13532b96-fcd8-4f12-a4df-b4041abe4cf3-metrics-tls\") pod \"dns-default-l8bvt\" (UID: \"13532b96-fcd8-4f12-a4df-b4041abe4cf3\") " pod="openshift-dns/dns-default-l8bvt"
Sep 18 20:02:15.716479 ip-10-0-130-81 kubenswrapper[2532]: E0918 20:02:15.716136    2532 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 18 20:02:15.716479 ip-10-0-130-81 kubenswrapper[2532]: E0918 20:02:15.716194    2532 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/13532b96-fcd8-4f12-a4df-b4041abe4cf3-metrics-tls podName:13532b96-fcd8-4f12-a4df-b4041abe4cf3 nodeName:}" failed. No retries permitted until 2026-09-18 20:02:47.716176224 +0000 UTC m=+99.481565609 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/13532b96-fcd8-4f12-a4df-b4041abe4cf3-metrics-tls") pod "dns-default-l8bvt" (UID: "13532b96-fcd8-4f12-a4df-b4041abe4cf3") : secret "dns-default-metrics-tls" not found
Sep 18 20:02:16.018603 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:02:16.018482    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/96167e3f-fb44-48e0-b86e-419a4a3b117c-metrics-certs\") pod \"network-metrics-daemon-xx7f7\" (UID: \"96167e3f-fb44-48e0-b86e-419a4a3b117c\") " pod="openshift-multus/network-metrics-daemon-xx7f7"
Sep 18 20:02:16.021363 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:02:16.021344    2532 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 18 20:02:16.029403 ip-10-0-130-81 kubenswrapper[2532]: E0918 20:02:16.029386    2532 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Sep 18 20:02:16.029461 ip-10-0-130-81 kubenswrapper[2532]: E0918 20:02:16.029438    2532 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/96167e3f-fb44-48e0-b86e-419a4a3b117c-metrics-certs podName:96167e3f-fb44-48e0-b86e-419a4a3b117c nodeName:}" failed. No retries permitted until 2026-09-18 20:03:20.029424701 +0000 UTC m=+131.794814077 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/96167e3f-fb44-48e0-b86e-419a4a3b117c-metrics-certs") pod "network-metrics-daemon-xx7f7" (UID: "96167e3f-fb44-48e0-b86e-419a4a3b117c") : secret "metrics-daemon-secret" not found
Sep 18 20:02:16.118779 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:02:16.118736    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b377a9e6-dbbb-4023-8997-70178a63034f-original-pull-secret\") pod \"global-pull-secret-syncer-gd9qz\" (UID: \"b377a9e6-dbbb-4023-8997-70178a63034f\") " pod="kube-system/global-pull-secret-syncer-gd9qz"
Sep 18 20:02:16.121823 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:02:16.121806    2532 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 18 20:02:16.133159 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:02:16.133136    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b377a9e6-dbbb-4023-8997-70178a63034f-original-pull-secret\") pod \"global-pull-secret-syncer-gd9qz\" (UID: \"b377a9e6-dbbb-4023-8997-70178a63034f\") " pod="kube-system/global-pull-secret-syncer-gd9qz"
Sep 18 20:02:16.219245 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:02:16.219210    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-slbwq\" (UniqueName: \"kubernetes.io/projected/a966f963-b4ae-46ba-b206-2a27ad401463-kube-api-access-slbwq\") pod \"network-check-target-gd24n\" (UID: \"a966f963-b4ae-46ba-b206-2a27ad401463\") " pod="openshift-network-diagnostics/network-check-target-gd24n"
Sep 18 20:02:16.222267 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:02:16.222249    2532 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 18 20:02:16.232522 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:02:16.232503    2532 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 18 20:02:16.242724 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:02:16.242703    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-slbwq\" (UniqueName: \"kubernetes.io/projected/a966f963-b4ae-46ba-b206-2a27ad401463-kube-api-access-slbwq\") pod \"network-check-target-gd24n\" (UID: \"a966f963-b4ae-46ba-b206-2a27ad401463\") " pod="openshift-network-diagnostics/network-check-target-gd24n"
Sep 18 20:02:16.411691 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:02:16.411616    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gd9qz"
Sep 18 20:02:16.420900 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:02:16.420879    2532 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-4pmvp\""
Sep 18 20:02:16.429056 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:02:16.429038    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gd24n"
Sep 18 20:02:16.681466 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:02:16.681412    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-gd9qz"]
Sep 18 20:02:16.684075 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:02:16.684047    2532 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb377a9e6_dbbb_4023_8997_70178a63034f.slice/crio-b943e46dba48c85c7cf0ac0dae10e8eb5d69dbb1cb83adb1e2042cad14287a63 WatchSource:0}: Error finding container b943e46dba48c85c7cf0ac0dae10e8eb5d69dbb1cb83adb1e2042cad14287a63: Status 404 returned error can't find the container with id b943e46dba48c85c7cf0ac0dae10e8eb5d69dbb1cb83adb1e2042cad14287a63
Sep 18 20:02:16.702965 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:02:16.702940    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-gd24n"]
Sep 18 20:02:16.705764 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:02:16.705738    2532 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda966f963_b4ae_46ba_b206_2a27ad401463.slice/crio-1ea41f1fe33360c5276d814a82da9e19135e669231a3fe2a243a674c21818938 WatchSource:0}: Error finding container 1ea41f1fe33360c5276d814a82da9e19135e669231a3fe2a243a674c21818938: Status 404 returned error can't find the container with id 1ea41f1fe33360c5276d814a82da9e19135e669231a3fe2a243a674c21818938
Sep 18 20:02:17.678226 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:02:17.678186    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-gd24n" event={"ID":"a966f963-b4ae-46ba-b206-2a27ad401463","Type":"ContainerStarted","Data":"1ea41f1fe33360c5276d814a82da9e19135e669231a3fe2a243a674c21818938"}
Sep 18 20:02:17.680075 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:02:17.680041    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-gd9qz" event={"ID":"b377a9e6-dbbb-4023-8997-70178a63034f","Type":"ContainerStarted","Data":"b943e46dba48c85c7cf0ac0dae10e8eb5d69dbb1cb83adb1e2042cad14287a63"}
Sep 18 20:02:21.691914 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:02:21.691873    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-gd24n" event={"ID":"a966f963-b4ae-46ba-b206-2a27ad401463","Type":"ContainerStarted","Data":"e57ee73f729f20a6b3ac92ba6083b1a0a339fab28f96358bc8a5fdaf4b973d7f"}
Sep 18 20:02:21.692383 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:02:21.691950    2532 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-gd24n"
Sep 18 20:02:21.693405 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:02:21.693384    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-gd9qz" event={"ID":"b377a9e6-dbbb-4023-8997-70178a63034f","Type":"ContainerStarted","Data":"e5a04da3c207a15c8b295d81118133e730bb78006a2da0e84ee2c3c1d696e018"}
Sep 18 20:02:21.707323 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:02:21.707282    2532 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-gd24n" podStartSLOduration=67.281125537 podStartE2EDuration="1m11.707265838s" totalImagesPullingTime="4.426140301s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 20:01:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 20:02:21.706035263 +0000 UTC m=+73.471424661" watchObservedRunningTime="2026-09-18 20:02:21.707265838 +0000 UTC m=+73.472655237"
Sep 18 20:02:21.718924 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:02:21.718880    2532 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-gd9qz" podStartSLOduration=67.267243167 podStartE2EDuration="1m11.718865956s" totalImagesPullingTime="4.451622789s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-18 20:01:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 20:02:21.718805318 +0000 UTC m=+73.484194719" watchObservedRunningTime="2026-09-18 20:02:21.718865956 +0000 UTC m=+73.484255355"
Sep 18 20:02:22.480548 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:02:22.480413    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:02:22.580524 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:02:22.580474    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:02:42.994127 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:02:42.994090    2532 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-78d794d58d-nzvv5"]
Sep 18 20:02:42.996836 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:02:42.996818    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-78d794d58d-nzvv5"
Sep 18 20:02:42.999342 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:02:42.999321    2532 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\""
Sep 18 20:02:43.000497 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:02:43.000466    2532 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Sep 18 20:02:43.000576 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:02:43.000551    2532 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Sep 18 20:02:43.000576 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:02:43.000551    2532 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Sep 18 20:02:43.006221 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:02:43.006201    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-78d794d58d-nzvv5"]
Sep 18 20:02:43.097562 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:02:43.097527    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/56c8406d-fcd1-465a-841f-48318b6d12d2-klusterlet-config\") pod \"klusterlet-addon-workmgr-78d794d58d-nzvv5\" (UID: \"56c8406d-fcd1-465a-841f-48318b6d12d2\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-78d794d58d-nzvv5"
Sep 18 20:02:43.097705 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:02:43.097572    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6slgf\" (UniqueName: \"kubernetes.io/projected/56c8406d-fcd1-465a-841f-48318b6d12d2-kube-api-access-6slgf\") pod \"klusterlet-addon-workmgr-78d794d58d-nzvv5\" (UID: \"56c8406d-fcd1-465a-841f-48318b6d12d2\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-78d794d58d-nzvv5"
Sep 18 20:02:43.097705 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:02:43.097595    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/56c8406d-fcd1-465a-841f-48318b6d12d2-tmp\") pod \"klusterlet-addon-workmgr-78d794d58d-nzvv5\" (UID: \"56c8406d-fcd1-465a-841f-48318b6d12d2\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-78d794d58d-nzvv5"
Sep 18 20:02:43.198099 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:02:43.198054    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/56c8406d-fcd1-465a-841f-48318b6d12d2-klusterlet-config\") pod \"klusterlet-addon-workmgr-78d794d58d-nzvv5\" (UID: \"56c8406d-fcd1-465a-841f-48318b6d12d2\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-78d794d58d-nzvv5"
Sep 18 20:02:43.198099 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:02:43.198106    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6slgf\" (UniqueName: \"kubernetes.io/projected/56c8406d-fcd1-465a-841f-48318b6d12d2-kube-api-access-6slgf\") pod \"klusterlet-addon-workmgr-78d794d58d-nzvv5\" (UID: \"56c8406d-fcd1-465a-841f-48318b6d12d2\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-78d794d58d-nzvv5"
Sep 18 20:02:43.198263 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:02:43.198128    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/56c8406d-fcd1-465a-841f-48318b6d12d2-tmp\") pod \"klusterlet-addon-workmgr-78d794d58d-nzvv5\" (UID: \"56c8406d-fcd1-465a-841f-48318b6d12d2\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-78d794d58d-nzvv5"
Sep 18 20:02:43.198525 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:02:43.198510    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/56c8406d-fcd1-465a-841f-48318b6d12d2-tmp\") pod \"klusterlet-addon-workmgr-78d794d58d-nzvv5\" (UID: \"56c8406d-fcd1-465a-841f-48318b6d12d2\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-78d794d58d-nzvv5"
Sep 18 20:02:43.200538 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:02:43.200523    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/56c8406d-fcd1-465a-841f-48318b6d12d2-klusterlet-config\") pod \"klusterlet-addon-workmgr-78d794d58d-nzvv5\" (UID: \"56c8406d-fcd1-465a-841f-48318b6d12d2\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-78d794d58d-nzvv5"
Sep 18 20:02:43.205754 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:02:43.205730    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6slgf\" (UniqueName: \"kubernetes.io/projected/56c8406d-fcd1-465a-841f-48318b6d12d2-kube-api-access-6slgf\") pod \"klusterlet-addon-workmgr-78d794d58d-nzvv5\" (UID: \"56c8406d-fcd1-465a-841f-48318b6d12d2\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-78d794d58d-nzvv5"
Sep 18 20:02:43.305508 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:02:43.305403    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-78d794d58d-nzvv5"
Sep 18 20:02:43.570314 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:02:43.570236    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-78d794d58d-nzvv5"]
Sep 18 20:02:43.573554 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:02:43.573526    2532 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod56c8406d_fcd1_465a_841f_48318b6d12d2.slice/crio-0576d2901164f47a7a99b2c175128ca6b00806dd3f990398c0ecdfffe09b59c7 WatchSource:0}: Error finding container 0576d2901164f47a7a99b2c175128ca6b00806dd3f990398c0ecdfffe09b59c7: Status 404 returned error can't find the container with id 0576d2901164f47a7a99b2c175128ca6b00806dd3f990398c0ecdfffe09b59c7
Sep 18 20:02:43.744544 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:02:43.744510    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-78d794d58d-nzvv5" event={"ID":"56c8406d-fcd1-465a-841f-48318b6d12d2","Type":"ContainerStarted","Data":"0576d2901164f47a7a99b2c175128ca6b00806dd3f990398c0ecdfffe09b59c7"}
Sep 18 20:02:47.734502 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:02:47.734436    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/13532b96-fcd8-4f12-a4df-b4041abe4cf3-metrics-tls\") pod \"dns-default-l8bvt\" (UID: \"13532b96-fcd8-4f12-a4df-b4041abe4cf3\") " pod="openshift-dns/dns-default-l8bvt"
Sep 18 20:02:47.734882 ip-10-0-130-81 kubenswrapper[2532]: E0918 20:02:47.734587    2532 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 18 20:02:47.734882 ip-10-0-130-81 kubenswrapper[2532]: E0918 20:02:47.734654    2532 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/13532b96-fcd8-4f12-a4df-b4041abe4cf3-metrics-tls podName:13532b96-fcd8-4f12-a4df-b4041abe4cf3 nodeName:}" failed. No retries permitted until 2026-09-18 20:03:51.734637537 +0000 UTC m=+163.500026914 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/13532b96-fcd8-4f12-a4df-b4041abe4cf3-metrics-tls") pod "dns-default-l8bvt" (UID: "13532b96-fcd8-4f12-a4df-b4041abe4cf3") : secret "dns-default-metrics-tls" not found
Sep 18 20:02:47.758657 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:02:47.758625    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-78d794d58d-nzvv5" event={"ID":"56c8406d-fcd1-465a-841f-48318b6d12d2","Type":"ContainerStarted","Data":"3ddba6f5317bda3884f19c64eed0fc470eadfe4f9852971837a445f4e12afceb"}
Sep 18 20:02:47.758888 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:02:47.758870    2532 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-78d794d58d-nzvv5"
Sep 18 20:02:47.760429 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:02:47.760410    2532 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-78d794d58d-nzvv5"
Sep 18 20:02:47.775874 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:02:47.775731    2532 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-78d794d58d-nzvv5" podStartSLOduration=1.965276726 podStartE2EDuration="5.775713909s" totalImagesPullingTime="3.810437183s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 20:02:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 20:02:47.775519776 +0000 UTC m=+99.540909173" watchObservedRunningTime="2026-09-18 20:02:47.775713909 +0000 UTC m=+99.541103309"
Sep 18 20:02:52.698261 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:02:52.698231    2532 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-gd24n"
Sep 18 20:03:06.864938 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:03:06.864906    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:03:20.052145 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:03:20.052111    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/96167e3f-fb44-48e0-b86e-419a4a3b117c-metrics-certs\") pod \"network-metrics-daemon-xx7f7\" (UID: \"96167e3f-fb44-48e0-b86e-419a4a3b117c\") " pod="openshift-multus/network-metrics-daemon-xx7f7"
Sep 18 20:03:20.052565 ip-10-0-130-81 kubenswrapper[2532]: E0918 20:03:20.052229    2532 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Sep 18 20:03:20.052565 ip-10-0-130-81 kubenswrapper[2532]: E0918 20:03:20.052293    2532 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/96167e3f-fb44-48e0-b86e-419a4a3b117c-metrics-certs podName:96167e3f-fb44-48e0-b86e-419a4a3b117c nodeName:}" failed. No retries permitted until 2026-09-18 20:05:22.052278867 +0000 UTC m=+253.817668243 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/96167e3f-fb44-48e0-b86e-419a4a3b117c-metrics-certs") pod "network-metrics-daemon-xx7f7" (UID: "96167e3f-fb44-48e0-b86e-419a4a3b117c") : secret "metrics-daemon-secret" not found
Sep 18 20:03:46.837822 ip-10-0-130-81 kubenswrapper[2532]: E0918 20:03:46.837776    2532 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-l8bvt" podUID="13532b96-fcd8-4f12-a4df-b4041abe4cf3"
Sep 18 20:03:46.898702 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:03:46.898673    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-l8bvt"
Sep 18 20:03:47.324820 ip-10-0-130-81 kubenswrapper[2532]: E0918 20:03:47.324767    2532 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-xx7f7" podUID="96167e3f-fb44-48e0-b86e-419a4a3b117c"
Sep 18 20:03:51.767264 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:03:51.767222    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/13532b96-fcd8-4f12-a4df-b4041abe4cf3-metrics-tls\") pod \"dns-default-l8bvt\" (UID: \"13532b96-fcd8-4f12-a4df-b4041abe4cf3\") " pod="openshift-dns/dns-default-l8bvt"
Sep 18 20:03:51.767696 ip-10-0-130-81 kubenswrapper[2532]: E0918 20:03:51.767359    2532 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 18 20:03:51.767696 ip-10-0-130-81 kubenswrapper[2532]: E0918 20:03:51.767422    2532 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/13532b96-fcd8-4f12-a4df-b4041abe4cf3-metrics-tls podName:13532b96-fcd8-4f12-a4df-b4041abe4cf3 nodeName:}" failed. No retries permitted until 2026-09-18 20:05:53.767406985 +0000 UTC m=+285.532796366 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/13532b96-fcd8-4f12-a4df-b4041abe4cf3-metrics-tls") pod "dns-default-l8bvt" (UID: "13532b96-fcd8-4f12-a4df-b4041abe4cf3") : secret "dns-default-metrics-tls" not found
Sep 18 20:03:59.299894 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:03:59.299805    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xx7f7"
Sep 18 20:04:39.841392 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:04:39.841362    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:04:47.759938 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:04:47.759826    2532 prober.go:120] "Probe failed" probeType="Readiness" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-78d794d58d-nzvv5" podUID="56c8406d-fcd1-465a-841f-48318b6d12d2" containerName="acm-agent" probeResult="failure" output="Get \"http://10.133.0.8:8000/readyz\": dial tcp 10.133.0.8:8000: connect: connection refused"
Sep 18 20:04:48.047210 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:04:48.047132    2532 generic.go:356] "Generic (PLEG): container finished" podID="56c8406d-fcd1-465a-841f-48318b6d12d2" containerID="3ddba6f5317bda3884f19c64eed0fc470eadfe4f9852971837a445f4e12afceb" exitCode=1
Sep 18 20:04:48.047210 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:04:48.047191    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-78d794d58d-nzvv5" event={"ID":"56c8406d-fcd1-465a-841f-48318b6d12d2","Type":"ContainerDied","Data":"3ddba6f5317bda3884f19c64eed0fc470eadfe4f9852971837a445f4e12afceb"}
Sep 18 20:04:48.047529 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:04:48.047505    2532 scope.go:117] "RemoveContainer" containerID="3ddba6f5317bda3884f19c64eed0fc470eadfe4f9852971837a445f4e12afceb"
Sep 18 20:04:49.054087 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:04:49.054054    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-78d794d58d-nzvv5" event={"ID":"56c8406d-fcd1-465a-841f-48318b6d12d2","Type":"ContainerStarted","Data":"bf01a6cb47c0bde38d18c22ebf3427d8e4cd594c8e216bff7f29891d12d2b03c"}
Sep 18 20:04:49.054475 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:04:49.054288    2532 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-78d794d58d-nzvv5"
Sep 18 20:04:49.054924 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:04:49.054909    2532 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-78d794d58d-nzvv5"
Sep 18 20:05:02.815419 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:02.815384    2532 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-6k4dr"]
Sep 18 20:05:02.818134 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:02.818117    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-6k4dr"
Sep 18 20:05:02.824303 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:02.824259    2532 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-9q76m\""
Sep 18 20:05:02.824303 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:02.824260    2532 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Sep 18 20:05:02.824303 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:02.824283    2532 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\""
Sep 18 20:05:02.824565 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:02.824293    2532 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\""
Sep 18 20:05:02.824565 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:02.824260    2532 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Sep 18 20:05:02.831738 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:02.831705    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-6k4dr"]
Sep 18 20:05:02.910509 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:02.910461    2532 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-qk5t7"]
Sep 18 20:05:02.913114 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:02.913090    2532 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-57cdc4b768-gr8p9"]
Sep 18 20:05:02.913268 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:02.913233    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-qk5t7"
Sep 18 20:05:02.915602 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:02.915581    2532 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-fd4bb46c-m7wdd"]
Sep 18 20:05:02.915695 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:02.915685    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-57cdc4b768-gr8p9"
Sep 18 20:05:02.916733 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:02.916716    2532 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\""
Sep 18 20:05:02.917020 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:02.916999    2532 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-hn6xx\""
Sep 18 20:05:02.917105 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:02.917045    2532 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\""
Sep 18 20:05:02.917460 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:02.917447    2532 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\""
Sep 18 20:05:02.918001 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:02.917987    2532 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-64795c474b-cmxpf"]
Sep 18 20:05:02.918141 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:02.918126    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-fd4bb46c-m7wdd"
Sep 18 20:05:02.918799 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:02.918784    2532 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\""
Sep 18 20:05:02.918891 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:02.918875    2532 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-56pnx\""
Sep 18 20:05:02.918992 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:02.918975    2532 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\""
Sep 18 20:05:02.919278 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:02.919264    2532 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\""
Sep 18 20:05:02.919448 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:02.919436    2532 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\""
Sep 18 20:05:02.920644 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:02.920624    2532 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\""
Sep 18 20:05:02.920741 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:02.920664    2532 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-przm6\""
Sep 18 20:05:02.920741 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:02.920726    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-64795c474b-cmxpf"
Sep 18 20:05:02.920860 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:02.920761    2532 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\""
Sep 18 20:05:02.920860 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:02.920772    2532 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\""
Sep 18 20:05:02.921309 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:02.921294    2532 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\""
Sep 18 20:05:02.923438 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:02.923423    2532 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Sep 18 20:05:02.923438 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:02.923435    2532 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-t5jb8\""
Sep 18 20:05:02.923608 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:02.923462    2532 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Sep 18 20:05:02.924019 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:02.924001    2532 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Sep 18 20:05:02.928388 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:02.928345    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-57cdc4b768-gr8p9"]
Sep 18 20:05:02.928388 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:02.928385    2532 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\""
Sep 18 20:05:02.929097 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:02.929079    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-qk5t7"]
Sep 18 20:05:02.929646 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:02.929622    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qd798\" (UniqueName: \"kubernetes.io/projected/dba6d982-36df-45cd-bb26-4deb21429b12-kube-api-access-qd798\") pod \"cluster-monitoring-operator-847b8bd8d7-6k4dr\" (UID: \"dba6d982-36df-45cd-bb26-4deb21429b12\") " pod="openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-6k4dr"
Sep 18 20:05:02.929741 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:02.929710    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/dba6d982-36df-45cd-bb26-4deb21429b12-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-847b8bd8d7-6k4dr\" (UID: \"dba6d982-36df-45cd-bb26-4deb21429b12\") " pod="openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-6k4dr"
Sep 18 20:05:02.929799 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:02.929753    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/dba6d982-36df-45cd-bb26-4deb21429b12-telemetry-config\") pod \"cluster-monitoring-operator-847b8bd8d7-6k4dr\" (UID: \"dba6d982-36df-45cd-bb26-4deb21429b12\") " pod="openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-6k4dr"
Sep 18 20:05:02.929975 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:02.929961    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-fd4bb46c-m7wdd"]
Sep 18 20:05:02.930707 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:02.930687    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-64795c474b-cmxpf"]
Sep 18 20:05:02.930793 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:02.930779    2532 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Sep 18 20:05:03.030897 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:03.030862    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/dba6d982-36df-45cd-bb26-4deb21429b12-telemetry-config\") pod \"cluster-monitoring-operator-847b8bd8d7-6k4dr\" (UID: \"dba6d982-36df-45cd-bb26-4deb21429b12\") " pod="openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-6k4dr"
Sep 18 20:05:03.031089 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:03.030903    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/b09e90e3-0343-40d8-af09-03c0cdcfaf36-ca-trust-extracted\") pod \"image-registry-64795c474b-cmxpf\" (UID: \"b09e90e3-0343-40d8-af09-03c0cdcfaf36\") " pod="openshift-image-registry/image-registry-64795c474b-cmxpf"
Sep 18 20:05:03.031089 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:03.030928    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4zzw4\" (UniqueName: \"kubernetes.io/projected/32e33d39-7826-499d-bc01-0487b7580e7a-kube-api-access-4zzw4\") pod \"console-operator-fd4bb46c-m7wdd\" (UID: \"32e33d39-7826-499d-bc01-0487b7580e7a\") " pod="openshift-console-operator/console-operator-fd4bb46c-m7wdd"
Sep 18 20:05:03.031089 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:03.030944    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/b09e90e3-0343-40d8-af09-03c0cdcfaf36-registry-tls\") pod \"image-registry-64795c474b-cmxpf\" (UID: \"b09e90e3-0343-40d8-af09-03c0cdcfaf36\") " pod="openshift-image-registry/image-registry-64795c474b-cmxpf"
Sep 18 20:05:03.031089 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:03.031009    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/32e33d39-7826-499d-bc01-0487b7580e7a-config\") pod \"console-operator-fd4bb46c-m7wdd\" (UID: \"32e33d39-7826-499d-bc01-0487b7580e7a\") " pod="openshift-console-operator/console-operator-fd4bb46c-m7wdd"
Sep 18 20:05:03.031089 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:03.031048    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/8e4262e9-3e90-4ed5-a3ac-4ea829f4c702-serving-cert\") pod \"service-ca-operator-57cdc4b768-gr8p9\" (UID: \"8e4262e9-3e90-4ed5-a3ac-4ea829f4c702\") " pod="openshift-service-ca-operator/service-ca-operator-57cdc4b768-gr8p9"
Sep 18 20:05:03.031089 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:03.031062    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/8e4262e9-3e90-4ed5-a3ac-4ea829f4c702-config\") pod \"service-ca-operator-57cdc4b768-gr8p9\" (UID: \"8e4262e9-3e90-4ed5-a3ac-4ea829f4c702\") " pod="openshift-service-ca-operator/service-ca-operator-57cdc4b768-gr8p9"
Sep 18 20:05:03.031089 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:03.031077    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/b09e90e3-0343-40d8-af09-03c0cdcfaf36-image-registry-private-configuration\") pod \"image-registry-64795c474b-cmxpf\" (UID: \"b09e90e3-0343-40d8-af09-03c0cdcfaf36\") " pod="openshift-image-registry/image-registry-64795c474b-cmxpf"
Sep 18 20:05:03.031089 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:03.031093    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/32e33d39-7826-499d-bc01-0487b7580e7a-trusted-ca\") pod \"console-operator-fd4bb46c-m7wdd\" (UID: \"32e33d39-7826-499d-bc01-0487b7580e7a\") " pod="openshift-console-operator/console-operator-fd4bb46c-m7wdd"
Sep 18 20:05:03.031479 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:03.031117    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/dba6d982-36df-45cd-bb26-4deb21429b12-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-847b8bd8d7-6k4dr\" (UID: \"dba6d982-36df-45cd-bb26-4deb21429b12\") " pod="openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-6k4dr"
Sep 18 20:05:03.031479 ip-10-0-130-81 kubenswrapper[2532]: E0918 20:05:03.031215    2532 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 18 20:05:03.031479 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:03.031252    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/32e33d39-7826-499d-bc01-0487b7580e7a-serving-cert\") pod \"console-operator-fd4bb46c-m7wdd\" (UID: \"32e33d39-7826-499d-bc01-0487b7580e7a\") " pod="openshift-console-operator/console-operator-fd4bb46c-m7wdd"
Sep 18 20:05:03.031479 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:03.031274    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q2db5\" (UniqueName: \"kubernetes.io/projected/71a5442b-4ebc-4e32-9cea-5a5ada982958-kube-api-access-q2db5\") pod \"cluster-samples-operator-685d7d68b5-qk5t7\" (UID: \"71a5442b-4ebc-4e32-9cea-5a5ada982958\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-qk5t7"
Sep 18 20:05:03.031479 ip-10-0-130-81 kubenswrapper[2532]: E0918 20:05:03.031296    2532 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dba6d982-36df-45cd-bb26-4deb21429b12-cluster-monitoring-operator-tls podName:dba6d982-36df-45cd-bb26-4deb21429b12 nodeName:}" failed. No retries permitted until 2026-09-18 20:05:03.531273358 +0000 UTC m=+235.296662760 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/dba6d982-36df-45cd-bb26-4deb21429b12-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-847b8bd8d7-6k4dr" (UID: "dba6d982-36df-45cd-bb26-4deb21429b12") : secret "cluster-monitoring-operator-tls" not found
Sep 18 20:05:03.031479 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:03.031332    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/b09e90e3-0343-40d8-af09-03c0cdcfaf36-trusted-ca\") pod \"image-registry-64795c474b-cmxpf\" (UID: \"b09e90e3-0343-40d8-af09-03c0cdcfaf36\") " pod="openshift-image-registry/image-registry-64795c474b-cmxpf"
Sep 18 20:05:03.031479 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:03.031357    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qd798\" (UniqueName: \"kubernetes.io/projected/dba6d982-36df-45cd-bb26-4deb21429b12-kube-api-access-qd798\") pod \"cluster-monitoring-operator-847b8bd8d7-6k4dr\" (UID: \"dba6d982-36df-45cd-bb26-4deb21429b12\") " pod="openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-6k4dr"
Sep 18 20:05:03.031479 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:03.031385    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/b09e90e3-0343-40d8-af09-03c0cdcfaf36-registry-certificates\") pod \"image-registry-64795c474b-cmxpf\" (UID: \"b09e90e3-0343-40d8-af09-03c0cdcfaf36\") " pod="openshift-image-registry/image-registry-64795c474b-cmxpf"
Sep 18 20:05:03.031479 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:03.031400    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sns7j\" (UniqueName: \"kubernetes.io/projected/8e4262e9-3e90-4ed5-a3ac-4ea829f4c702-kube-api-access-sns7j\") pod \"service-ca-operator-57cdc4b768-gr8p9\" (UID: \"8e4262e9-3e90-4ed5-a3ac-4ea829f4c702\") " pod="openshift-service-ca-operator/service-ca-operator-57cdc4b768-gr8p9"
Sep 18 20:05:03.031479 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:03.031441    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/b09e90e3-0343-40d8-af09-03c0cdcfaf36-installation-pull-secrets\") pod \"image-registry-64795c474b-cmxpf\" (UID: \"b09e90e3-0343-40d8-af09-03c0cdcfaf36\") " pod="openshift-image-registry/image-registry-64795c474b-cmxpf"
Sep 18 20:05:03.031479 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:03.031466    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nqb9w\" (UniqueName: \"kubernetes.io/projected/b09e90e3-0343-40d8-af09-03c0cdcfaf36-kube-api-access-nqb9w\") pod \"image-registry-64795c474b-cmxpf\" (UID: \"b09e90e3-0343-40d8-af09-03c0cdcfaf36\") " pod="openshift-image-registry/image-registry-64795c474b-cmxpf"
Sep 18 20:05:03.031889 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:03.031526    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/71a5442b-4ebc-4e32-9cea-5a5ada982958-samples-operator-tls\") pod \"cluster-samples-operator-685d7d68b5-qk5t7\" (UID: \"71a5442b-4ebc-4e32-9cea-5a5ada982958\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-qk5t7"
Sep 18 20:05:03.031889 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:03.031557    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/b09e90e3-0343-40d8-af09-03c0cdcfaf36-bound-sa-token\") pod \"image-registry-64795c474b-cmxpf\" (UID: \"b09e90e3-0343-40d8-af09-03c0cdcfaf36\") " pod="openshift-image-registry/image-registry-64795c474b-cmxpf"
Sep 18 20:05:03.031889 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:03.031606    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/dba6d982-36df-45cd-bb26-4deb21429b12-telemetry-config\") pod \"cluster-monitoring-operator-847b8bd8d7-6k4dr\" (UID: \"dba6d982-36df-45cd-bb26-4deb21429b12\") " pod="openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-6k4dr"
Sep 18 20:05:03.039481 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:03.039464    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qd798\" (UniqueName: \"kubernetes.io/projected/dba6d982-36df-45cd-bb26-4deb21429b12-kube-api-access-qd798\") pod \"cluster-monitoring-operator-847b8bd8d7-6k4dr\" (UID: \"dba6d982-36df-45cd-bb26-4deb21429b12\") " pod="openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-6k4dr"
Sep 18 20:05:03.048290 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:03.048266    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:05:03.118190 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:03.118110    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:05:03.132709 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:03.132682    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/b09e90e3-0343-40d8-af09-03c0cdcfaf36-bound-sa-token\") pod \"image-registry-64795c474b-cmxpf\" (UID: \"b09e90e3-0343-40d8-af09-03c0cdcfaf36\") " pod="openshift-image-registry/image-registry-64795c474b-cmxpf"
Sep 18 20:05:03.132818 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:03.132716    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/b09e90e3-0343-40d8-af09-03c0cdcfaf36-ca-trust-extracted\") pod \"image-registry-64795c474b-cmxpf\" (UID: \"b09e90e3-0343-40d8-af09-03c0cdcfaf36\") " pod="openshift-image-registry/image-registry-64795c474b-cmxpf"
Sep 18 20:05:03.132818 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:03.132738    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4zzw4\" (UniqueName: \"kubernetes.io/projected/32e33d39-7826-499d-bc01-0487b7580e7a-kube-api-access-4zzw4\") pod \"console-operator-fd4bb46c-m7wdd\" (UID: \"32e33d39-7826-499d-bc01-0487b7580e7a\") " pod="openshift-console-operator/console-operator-fd4bb46c-m7wdd"
Sep 18 20:05:03.132818 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:03.132757    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/b09e90e3-0343-40d8-af09-03c0cdcfaf36-registry-tls\") pod \"image-registry-64795c474b-cmxpf\" (UID: \"b09e90e3-0343-40d8-af09-03c0cdcfaf36\") " pod="openshift-image-registry/image-registry-64795c474b-cmxpf"
Sep 18 20:05:03.132818 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:03.132781    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/32e33d39-7826-499d-bc01-0487b7580e7a-config\") pod \"console-operator-fd4bb46c-m7wdd\" (UID: \"32e33d39-7826-499d-bc01-0487b7580e7a\") " pod="openshift-console-operator/console-operator-fd4bb46c-m7wdd"
Sep 18 20:05:03.132818 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:03.132799    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/8e4262e9-3e90-4ed5-a3ac-4ea829f4c702-serving-cert\") pod \"service-ca-operator-57cdc4b768-gr8p9\" (UID: \"8e4262e9-3e90-4ed5-a3ac-4ea829f4c702\") " pod="openshift-service-ca-operator/service-ca-operator-57cdc4b768-gr8p9"
Sep 18 20:05:03.132818 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:03.132814    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/8e4262e9-3e90-4ed5-a3ac-4ea829f4c702-config\") pod \"service-ca-operator-57cdc4b768-gr8p9\" (UID: \"8e4262e9-3e90-4ed5-a3ac-4ea829f4c702\") " pod="openshift-service-ca-operator/service-ca-operator-57cdc4b768-gr8p9"
Sep 18 20:05:03.133121 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:03.132836    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/b09e90e3-0343-40d8-af09-03c0cdcfaf36-image-registry-private-configuration\") pod \"image-registry-64795c474b-cmxpf\" (UID: \"b09e90e3-0343-40d8-af09-03c0cdcfaf36\") " pod="openshift-image-registry/image-registry-64795c474b-cmxpf"
Sep 18 20:05:03.133121 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:03.132854    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/32e33d39-7826-499d-bc01-0487b7580e7a-trusted-ca\") pod \"console-operator-fd4bb46c-m7wdd\" (UID: \"32e33d39-7826-499d-bc01-0487b7580e7a\") " pod="openshift-console-operator/console-operator-fd4bb46c-m7wdd"
Sep 18 20:05:03.133121 ip-10-0-130-81 kubenswrapper[2532]: E0918 20:05:03.132857    2532 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 18 20:05:03.133121 ip-10-0-130-81 kubenswrapper[2532]: E0918 20:05:03.132874    2532 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-64795c474b-cmxpf: secret "image-registry-tls" not found
Sep 18 20:05:03.133121 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:03.132894    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/32e33d39-7826-499d-bc01-0487b7580e7a-serving-cert\") pod \"console-operator-fd4bb46c-m7wdd\" (UID: \"32e33d39-7826-499d-bc01-0487b7580e7a\") " pod="openshift-console-operator/console-operator-fd4bb46c-m7wdd"
Sep 18 20:05:03.133121 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:03.132925    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q2db5\" (UniqueName: \"kubernetes.io/projected/71a5442b-4ebc-4e32-9cea-5a5ada982958-kube-api-access-q2db5\") pod \"cluster-samples-operator-685d7d68b5-qk5t7\" (UID: \"71a5442b-4ebc-4e32-9cea-5a5ada982958\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-qk5t7"
Sep 18 20:05:03.133121 ip-10-0-130-81 kubenswrapper[2532]: E0918 20:05:03.132945    2532 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b09e90e3-0343-40d8-af09-03c0cdcfaf36-registry-tls podName:b09e90e3-0343-40d8-af09-03c0cdcfaf36 nodeName:}" failed. No retries permitted until 2026-09-18 20:05:03.632924542 +0000 UTC m=+235.398313943 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/b09e90e3-0343-40d8-af09-03c0cdcfaf36-registry-tls") pod "image-registry-64795c474b-cmxpf" (UID: "b09e90e3-0343-40d8-af09-03c0cdcfaf36") : secret "image-registry-tls" not found
Sep 18 20:05:03.133121 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:03.132979    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/b09e90e3-0343-40d8-af09-03c0cdcfaf36-trusted-ca\") pod \"image-registry-64795c474b-cmxpf\" (UID: \"b09e90e3-0343-40d8-af09-03c0cdcfaf36\") " pod="openshift-image-registry/image-registry-64795c474b-cmxpf"
Sep 18 20:05:03.133121 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:03.133024    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/b09e90e3-0343-40d8-af09-03c0cdcfaf36-registry-certificates\") pod \"image-registry-64795c474b-cmxpf\" (UID: \"b09e90e3-0343-40d8-af09-03c0cdcfaf36\") " pod="openshift-image-registry/image-registry-64795c474b-cmxpf"
Sep 18 20:05:03.133121 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:03.133052    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sns7j\" (UniqueName: \"kubernetes.io/projected/8e4262e9-3e90-4ed5-a3ac-4ea829f4c702-kube-api-access-sns7j\") pod \"service-ca-operator-57cdc4b768-gr8p9\" (UID: \"8e4262e9-3e90-4ed5-a3ac-4ea829f4c702\") " pod="openshift-service-ca-operator/service-ca-operator-57cdc4b768-gr8p9"
Sep 18 20:05:03.133121 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:03.133103    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/b09e90e3-0343-40d8-af09-03c0cdcfaf36-installation-pull-secrets\") pod \"image-registry-64795c474b-cmxpf\" (UID: \"b09e90e3-0343-40d8-af09-03c0cdcfaf36\") " pod="openshift-image-registry/image-registry-64795c474b-cmxpf"
Sep 18 20:05:03.133687 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:03.133145    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nqb9w\" (UniqueName: \"kubernetes.io/projected/b09e90e3-0343-40d8-af09-03c0cdcfaf36-kube-api-access-nqb9w\") pod \"image-registry-64795c474b-cmxpf\" (UID: \"b09e90e3-0343-40d8-af09-03c0cdcfaf36\") " pod="openshift-image-registry/image-registry-64795c474b-cmxpf"
Sep 18 20:05:03.133687 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:03.133171    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/71a5442b-4ebc-4e32-9cea-5a5ada982958-samples-operator-tls\") pod \"cluster-samples-operator-685d7d68b5-qk5t7\" (UID: \"71a5442b-4ebc-4e32-9cea-5a5ada982958\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-qk5t7"
Sep 18 20:05:03.133687 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:03.133547    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/b09e90e3-0343-40d8-af09-03c0cdcfaf36-ca-trust-extracted\") pod \"image-registry-64795c474b-cmxpf\" (UID: \"b09e90e3-0343-40d8-af09-03c0cdcfaf36\") " pod="openshift-image-registry/image-registry-64795c474b-cmxpf"
Sep 18 20:05:03.133687 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:03.133564    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/8e4262e9-3e90-4ed5-a3ac-4ea829f4c702-config\") pod \"service-ca-operator-57cdc4b768-gr8p9\" (UID: \"8e4262e9-3e90-4ed5-a3ac-4ea829f4c702\") " pod="openshift-service-ca-operator/service-ca-operator-57cdc4b768-gr8p9"
Sep 18 20:05:03.133891 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:03.133686    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/32e33d39-7826-499d-bc01-0487b7580e7a-config\") pod \"console-operator-fd4bb46c-m7wdd\" (UID: \"32e33d39-7826-499d-bc01-0487b7580e7a\") " pod="openshift-console-operator/console-operator-fd4bb46c-m7wdd"
Sep 18 20:05:03.133891 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:03.133698    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/b09e90e3-0343-40d8-af09-03c0cdcfaf36-registry-certificates\") pod \"image-registry-64795c474b-cmxpf\" (UID: \"b09e90e3-0343-40d8-af09-03c0cdcfaf36\") " pod="openshift-image-registry/image-registry-64795c474b-cmxpf"
Sep 18 20:05:03.133891 ip-10-0-130-81 kubenswrapper[2532]: E0918 20:05:03.133778    2532 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 18 20:05:03.133891 ip-10-0-130-81 kubenswrapper[2532]: E0918 20:05:03.133838    2532 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/71a5442b-4ebc-4e32-9cea-5a5ada982958-samples-operator-tls podName:71a5442b-4ebc-4e32-9cea-5a5ada982958 nodeName:}" failed. No retries permitted until 2026-09-18 20:05:03.633815452 +0000 UTC m=+235.399204828 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/71a5442b-4ebc-4e32-9cea-5a5ada982958-samples-operator-tls") pod "cluster-samples-operator-685d7d68b5-qk5t7" (UID: "71a5442b-4ebc-4e32-9cea-5a5ada982958") : secret "samples-operator-tls" not found
Sep 18 20:05:03.134107 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:03.133983    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/32e33d39-7826-499d-bc01-0487b7580e7a-trusted-ca\") pod \"console-operator-fd4bb46c-m7wdd\" (UID: \"32e33d39-7826-499d-bc01-0487b7580e7a\") " pod="openshift-console-operator/console-operator-fd4bb46c-m7wdd"
Sep 18 20:05:03.134607 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:03.134587    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/b09e90e3-0343-40d8-af09-03c0cdcfaf36-trusted-ca\") pod \"image-registry-64795c474b-cmxpf\" (UID: \"b09e90e3-0343-40d8-af09-03c0cdcfaf36\") " pod="openshift-image-registry/image-registry-64795c474b-cmxpf"
Sep 18 20:05:03.135332 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:03.135314    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/8e4262e9-3e90-4ed5-a3ac-4ea829f4c702-serving-cert\") pod \"service-ca-operator-57cdc4b768-gr8p9\" (UID: \"8e4262e9-3e90-4ed5-a3ac-4ea829f4c702\") " pod="openshift-service-ca-operator/service-ca-operator-57cdc4b768-gr8p9"
Sep 18 20:05:03.135902 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:03.135887    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/b09e90e3-0343-40d8-af09-03c0cdcfaf36-image-registry-private-configuration\") pod \"image-registry-64795c474b-cmxpf\" (UID: \"b09e90e3-0343-40d8-af09-03c0cdcfaf36\") " pod="openshift-image-registry/image-registry-64795c474b-cmxpf"
Sep 18 20:05:03.136056 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:03.136038    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/b09e90e3-0343-40d8-af09-03c0cdcfaf36-installation-pull-secrets\") pod \"image-registry-64795c474b-cmxpf\" (UID: \"b09e90e3-0343-40d8-af09-03c0cdcfaf36\") " pod="openshift-image-registry/image-registry-64795c474b-cmxpf"
Sep 18 20:05:03.136099 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:03.136057    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/32e33d39-7826-499d-bc01-0487b7580e7a-serving-cert\") pod \"console-operator-fd4bb46c-m7wdd\" (UID: \"32e33d39-7826-499d-bc01-0487b7580e7a\") " pod="openshift-console-operator/console-operator-fd4bb46c-m7wdd"
Sep 18 20:05:03.143244 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:03.143217    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4zzw4\" (UniqueName: \"kubernetes.io/projected/32e33d39-7826-499d-bc01-0487b7580e7a-kube-api-access-4zzw4\") pod \"console-operator-fd4bb46c-m7wdd\" (UID: \"32e33d39-7826-499d-bc01-0487b7580e7a\") " pod="openshift-console-operator/console-operator-fd4bb46c-m7wdd"
Sep 18 20:05:03.143764 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:03.143738    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sns7j\" (UniqueName: \"kubernetes.io/projected/8e4262e9-3e90-4ed5-a3ac-4ea829f4c702-kube-api-access-sns7j\") pod \"service-ca-operator-57cdc4b768-gr8p9\" (UID: \"8e4262e9-3e90-4ed5-a3ac-4ea829f4c702\") " pod="openshift-service-ca-operator/service-ca-operator-57cdc4b768-gr8p9"
Sep 18 20:05:03.144254 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:03.144232    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q2db5\" (UniqueName: \"kubernetes.io/projected/71a5442b-4ebc-4e32-9cea-5a5ada982958-kube-api-access-q2db5\") pod \"cluster-samples-operator-685d7d68b5-qk5t7\" (UID: \"71a5442b-4ebc-4e32-9cea-5a5ada982958\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-qk5t7"
Sep 18 20:05:03.144321 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:03.144283    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/b09e90e3-0343-40d8-af09-03c0cdcfaf36-bound-sa-token\") pod \"image-registry-64795c474b-cmxpf\" (UID: \"b09e90e3-0343-40d8-af09-03c0cdcfaf36\") " pod="openshift-image-registry/image-registry-64795c474b-cmxpf"
Sep 18 20:05:03.144606 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:03.144590    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nqb9w\" (UniqueName: \"kubernetes.io/projected/b09e90e3-0343-40d8-af09-03c0cdcfaf36-kube-api-access-nqb9w\") pod \"image-registry-64795c474b-cmxpf\" (UID: \"b09e90e3-0343-40d8-af09-03c0cdcfaf36\") " pod="openshift-image-registry/image-registry-64795c474b-cmxpf"
Sep 18 20:05:03.232026 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:03.231981    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-57cdc4b768-gr8p9"
Sep 18 20:05:03.238468 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:03.238447    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-fd4bb46c-m7wdd"
Sep 18 20:05:03.501071 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:03.501040    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-57cdc4b768-gr8p9"]
Sep 18 20:05:03.504097 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:05:03.504060    2532 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8e4262e9_3e90_4ed5_a3ac_4ea829f4c702.slice/crio-285cebbe5a2bdcbad2115f8e24665fe5039e244f3a365e6b25e260a4cee68ff7 WatchSource:0}: Error finding container 285cebbe5a2bdcbad2115f8e24665fe5039e244f3a365e6b25e260a4cee68ff7: Status 404 returned error can't find the container with id 285cebbe5a2bdcbad2115f8e24665fe5039e244f3a365e6b25e260a4cee68ff7
Sep 18 20:05:03.505156 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:03.505134    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-fd4bb46c-m7wdd"]
Sep 18 20:05:03.509440 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:05:03.509415    2532 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod32e33d39_7826_499d_bc01_0487b7580e7a.slice/crio-d08c7d6bee067bdab9f357f0c303a913710132bb306bd507d2fff45ecdfb06be WatchSource:0}: Error finding container d08c7d6bee067bdab9f357f0c303a913710132bb306bd507d2fff45ecdfb06be: Status 404 returned error can't find the container with id d08c7d6bee067bdab9f357f0c303a913710132bb306bd507d2fff45ecdfb06be
Sep 18 20:05:03.536583 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:03.536553    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/dba6d982-36df-45cd-bb26-4deb21429b12-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-847b8bd8d7-6k4dr\" (UID: \"dba6d982-36df-45cd-bb26-4deb21429b12\") " pod="openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-6k4dr"
Sep 18 20:05:03.536724 ip-10-0-130-81 kubenswrapper[2532]: E0918 20:05:03.536702    2532 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 18 20:05:03.536776 ip-10-0-130-81 kubenswrapper[2532]: E0918 20:05:03.536766    2532 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dba6d982-36df-45cd-bb26-4deb21429b12-cluster-monitoring-operator-tls podName:dba6d982-36df-45cd-bb26-4deb21429b12 nodeName:}" failed. No retries permitted until 2026-09-18 20:05:04.536747343 +0000 UTC m=+236.302136734 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/dba6d982-36df-45cd-bb26-4deb21429b12-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-847b8bd8d7-6k4dr" (UID: "dba6d982-36df-45cd-bb26-4deb21429b12") : secret "cluster-monitoring-operator-tls" not found
Sep 18 20:05:03.637859 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:03.637823    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/71a5442b-4ebc-4e32-9cea-5a5ada982958-samples-operator-tls\") pod \"cluster-samples-operator-685d7d68b5-qk5t7\" (UID: \"71a5442b-4ebc-4e32-9cea-5a5ada982958\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-qk5t7"
Sep 18 20:05:03.637859 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:03.637866    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/b09e90e3-0343-40d8-af09-03c0cdcfaf36-registry-tls\") pod \"image-registry-64795c474b-cmxpf\" (UID: \"b09e90e3-0343-40d8-af09-03c0cdcfaf36\") " pod="openshift-image-registry/image-registry-64795c474b-cmxpf"
Sep 18 20:05:03.638027 ip-10-0-130-81 kubenswrapper[2532]: E0918 20:05:03.637961    2532 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 18 20:05:03.638027 ip-10-0-130-81 kubenswrapper[2532]: E0918 20:05:03.637965    2532 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 18 20:05:03.638087 ip-10-0-130-81 kubenswrapper[2532]: E0918 20:05:03.638028    2532 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/71a5442b-4ebc-4e32-9cea-5a5ada982958-samples-operator-tls podName:71a5442b-4ebc-4e32-9cea-5a5ada982958 nodeName:}" failed. No retries permitted until 2026-09-18 20:05:04.638013346 +0000 UTC m=+236.403402723 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/71a5442b-4ebc-4e32-9cea-5a5ada982958-samples-operator-tls") pod "cluster-samples-operator-685d7d68b5-qk5t7" (UID: "71a5442b-4ebc-4e32-9cea-5a5ada982958") : secret "samples-operator-tls" not found
Sep 18 20:05:03.638087 ip-10-0-130-81 kubenswrapper[2532]: E0918 20:05:03.637972    2532 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-64795c474b-cmxpf: secret "image-registry-tls" not found
Sep 18 20:05:03.638161 ip-10-0-130-81 kubenswrapper[2532]: E0918 20:05:03.638108    2532 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b09e90e3-0343-40d8-af09-03c0cdcfaf36-registry-tls podName:b09e90e3-0343-40d8-af09-03c0cdcfaf36 nodeName:}" failed. No retries permitted until 2026-09-18 20:05:04.638091579 +0000 UTC m=+236.403480969 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/b09e90e3-0343-40d8-af09-03c0cdcfaf36-registry-tls") pod "image-registry-64795c474b-cmxpf" (UID: "b09e90e3-0343-40d8-af09-03c0cdcfaf36") : secret "image-registry-tls" not found
Sep 18 20:05:04.095311 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:04.095272    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-fd4bb46c-m7wdd" event={"ID":"32e33d39-7826-499d-bc01-0487b7580e7a","Type":"ContainerStarted","Data":"d08c7d6bee067bdab9f357f0c303a913710132bb306bd507d2fff45ecdfb06be"}
Sep 18 20:05:04.096291 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:04.096268    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-57cdc4b768-gr8p9" event={"ID":"8e4262e9-3e90-4ed5-a3ac-4ea829f4c702","Type":"ContainerStarted","Data":"285cebbe5a2bdcbad2115f8e24665fe5039e244f3a365e6b25e260a4cee68ff7"}
Sep 18 20:05:04.545389 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:04.545346    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/dba6d982-36df-45cd-bb26-4deb21429b12-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-847b8bd8d7-6k4dr\" (UID: \"dba6d982-36df-45cd-bb26-4deb21429b12\") " pod="openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-6k4dr"
Sep 18 20:05:04.545610 ip-10-0-130-81 kubenswrapper[2532]: E0918 20:05:04.545547    2532 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 18 20:05:04.545675 ip-10-0-130-81 kubenswrapper[2532]: E0918 20:05:04.545645    2532 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dba6d982-36df-45cd-bb26-4deb21429b12-cluster-monitoring-operator-tls podName:dba6d982-36df-45cd-bb26-4deb21429b12 nodeName:}" failed. No retries permitted until 2026-09-18 20:05:06.545622918 +0000 UTC m=+238.311012295 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/dba6d982-36df-45cd-bb26-4deb21429b12-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-847b8bd8d7-6k4dr" (UID: "dba6d982-36df-45cd-bb26-4deb21429b12") : secret "cluster-monitoring-operator-tls" not found
Sep 18 20:05:04.646237 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:04.646195    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/71a5442b-4ebc-4e32-9cea-5a5ada982958-samples-operator-tls\") pod \"cluster-samples-operator-685d7d68b5-qk5t7\" (UID: \"71a5442b-4ebc-4e32-9cea-5a5ada982958\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-qk5t7"
Sep 18 20:05:04.646470 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:04.646260    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/b09e90e3-0343-40d8-af09-03c0cdcfaf36-registry-tls\") pod \"image-registry-64795c474b-cmxpf\" (UID: \"b09e90e3-0343-40d8-af09-03c0cdcfaf36\") " pod="openshift-image-registry/image-registry-64795c474b-cmxpf"
Sep 18 20:05:04.646470 ip-10-0-130-81 kubenswrapper[2532]: E0918 20:05:04.646353    2532 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 18 20:05:04.646470 ip-10-0-130-81 kubenswrapper[2532]: E0918 20:05:04.646388    2532 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 18 20:05:04.646470 ip-10-0-130-81 kubenswrapper[2532]: E0918 20:05:04.646403    2532 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-64795c474b-cmxpf: secret "image-registry-tls" not found
Sep 18 20:05:04.646470 ip-10-0-130-81 kubenswrapper[2532]: E0918 20:05:04.646444    2532 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/71a5442b-4ebc-4e32-9cea-5a5ada982958-samples-operator-tls podName:71a5442b-4ebc-4e32-9cea-5a5ada982958 nodeName:}" failed. No retries permitted until 2026-09-18 20:05:06.64642337 +0000 UTC m=+238.411812762 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/71a5442b-4ebc-4e32-9cea-5a5ada982958-samples-operator-tls") pod "cluster-samples-operator-685d7d68b5-qk5t7" (UID: "71a5442b-4ebc-4e32-9cea-5a5ada982958") : secret "samples-operator-tls" not found
Sep 18 20:05:04.646470 ip-10-0-130-81 kubenswrapper[2532]: E0918 20:05:04.646459    2532 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b09e90e3-0343-40d8-af09-03c0cdcfaf36-registry-tls podName:b09e90e3-0343-40d8-af09-03c0cdcfaf36 nodeName:}" failed. No retries permitted until 2026-09-18 20:05:06.646452787 +0000 UTC m=+238.411842164 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/b09e90e3-0343-40d8-af09-03c0cdcfaf36-registry-tls") pod "image-registry-64795c474b-cmxpf" (UID: "b09e90e3-0343-40d8-af09-03c0cdcfaf36") : secret "image-registry-tls" not found
Sep 18 20:05:06.104142 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:06.104114    2532 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-m7wdd_32e33d39-7826-499d-bc01-0487b7580e7a/console-operator/0.log"
Sep 18 20:05:06.104579 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:06.104155    2532 generic.go:356] "Generic (PLEG): container finished" podID="32e33d39-7826-499d-bc01-0487b7580e7a" containerID="318fde1e34883657951984f626d56b2e3646fb0a512898c6af7a5fe85ecf658d" exitCode=255
Sep 18 20:05:06.104579 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:06.104232    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-fd4bb46c-m7wdd" event={"ID":"32e33d39-7826-499d-bc01-0487b7580e7a","Type":"ContainerDied","Data":"318fde1e34883657951984f626d56b2e3646fb0a512898c6af7a5fe85ecf658d"}
Sep 18 20:05:06.104579 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:06.104454    2532 scope.go:117] "RemoveContainer" containerID="318fde1e34883657951984f626d56b2e3646fb0a512898c6af7a5fe85ecf658d"
Sep 18 20:05:06.105905 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:06.105883    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-57cdc4b768-gr8p9" event={"ID":"8e4262e9-3e90-4ed5-a3ac-4ea829f4c702","Type":"ContainerStarted","Data":"0a3c364aab1f04804f39e1c53e6458aa42a110f68cc6bde97be32fdc55ff3113"}
Sep 18 20:05:06.133141 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:06.133094    2532 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-57cdc4b768-gr8p9" podStartSLOduration=1.700761012 podStartE2EDuration="4.133078027s" totalImagesPullingTime="2.432317015s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 20:05:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 20:05:06.132518065 +0000 UTC m=+237.897907464" watchObservedRunningTime="2026-09-18 20:05:06.133078027 +0000 UTC m=+237.898467442"
Sep 18 20:05:06.563953 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:06.563851    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/dba6d982-36df-45cd-bb26-4deb21429b12-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-847b8bd8d7-6k4dr\" (UID: \"dba6d982-36df-45cd-bb26-4deb21429b12\") " pod="openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-6k4dr"
Sep 18 20:05:06.564101 ip-10-0-130-81 kubenswrapper[2532]: E0918 20:05:06.563999    2532 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 18 20:05:06.564101 ip-10-0-130-81 kubenswrapper[2532]: E0918 20:05:06.564065    2532 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dba6d982-36df-45cd-bb26-4deb21429b12-cluster-monitoring-operator-tls podName:dba6d982-36df-45cd-bb26-4deb21429b12 nodeName:}" failed. No retries permitted until 2026-09-18 20:05:10.564051153 +0000 UTC m=+242.329440530 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/dba6d982-36df-45cd-bb26-4deb21429b12-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-847b8bd8d7-6k4dr" (UID: "dba6d982-36df-45cd-bb26-4deb21429b12") : secret "cluster-monitoring-operator-tls" not found
Sep 18 20:05:06.665258 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:06.665222    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/71a5442b-4ebc-4e32-9cea-5a5ada982958-samples-operator-tls\") pod \"cluster-samples-operator-685d7d68b5-qk5t7\" (UID: \"71a5442b-4ebc-4e32-9cea-5a5ada982958\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-qk5t7"
Sep 18 20:05:06.665436 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:06.665278    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/b09e90e3-0343-40d8-af09-03c0cdcfaf36-registry-tls\") pod \"image-registry-64795c474b-cmxpf\" (UID: \"b09e90e3-0343-40d8-af09-03c0cdcfaf36\") " pod="openshift-image-registry/image-registry-64795c474b-cmxpf"
Sep 18 20:05:06.665436 ip-10-0-130-81 kubenswrapper[2532]: E0918 20:05:06.665407    2532 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 18 20:05:06.665532 ip-10-0-130-81 kubenswrapper[2532]: E0918 20:05:06.665445    2532 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 18 20:05:06.665532 ip-10-0-130-81 kubenswrapper[2532]: E0918 20:05:06.665460    2532 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-64795c474b-cmxpf: secret "image-registry-tls" not found
Sep 18 20:05:06.665532 ip-10-0-130-81 kubenswrapper[2532]: E0918 20:05:06.665470    2532 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/71a5442b-4ebc-4e32-9cea-5a5ada982958-samples-operator-tls podName:71a5442b-4ebc-4e32-9cea-5a5ada982958 nodeName:}" failed. No retries permitted until 2026-09-18 20:05:10.66545578 +0000 UTC m=+242.430845162 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/71a5442b-4ebc-4e32-9cea-5a5ada982958-samples-operator-tls") pod "cluster-samples-operator-685d7d68b5-qk5t7" (UID: "71a5442b-4ebc-4e32-9cea-5a5ada982958") : secret "samples-operator-tls" not found
Sep 18 20:05:06.665532 ip-10-0-130-81 kubenswrapper[2532]: E0918 20:05:06.665518    2532 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b09e90e3-0343-40d8-af09-03c0cdcfaf36-registry-tls podName:b09e90e3-0343-40d8-af09-03c0cdcfaf36 nodeName:}" failed. No retries permitted until 2026-09-18 20:05:10.665506108 +0000 UTC m=+242.430895485 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/b09e90e3-0343-40d8-af09-03c0cdcfaf36-registry-tls") pod "image-registry-64795c474b-cmxpf" (UID: "b09e90e3-0343-40d8-af09-03c0cdcfaf36") : secret "image-registry-tls" not found
Sep 18 20:05:06.906941 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:06.906861    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:05:07.038617 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:07.038584    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:05:07.110760 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:07.110735    2532 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-m7wdd_32e33d39-7826-499d-bc01-0487b7580e7a/console-operator/1.log"
Sep 18 20:05:07.111198 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:07.111167    2532 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-m7wdd_32e33d39-7826-499d-bc01-0487b7580e7a/console-operator/0.log"
Sep 18 20:05:07.111267 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:07.111207    2532 generic.go:356] "Generic (PLEG): container finished" podID="32e33d39-7826-499d-bc01-0487b7580e7a" containerID="b0783927736013c9ba1430dbebeebbbb53b795397a898b9c7ad4ee1798722948" exitCode=255
Sep 18 20:05:07.111267 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:07.111238    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-fd4bb46c-m7wdd" event={"ID":"32e33d39-7826-499d-bc01-0487b7580e7a","Type":"ContainerDied","Data":"b0783927736013c9ba1430dbebeebbbb53b795397a898b9c7ad4ee1798722948"}
Sep 18 20:05:07.111341 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:07.111295    2532 scope.go:117] "RemoveContainer" containerID="318fde1e34883657951984f626d56b2e3646fb0a512898c6af7a5fe85ecf658d"
Sep 18 20:05:07.111561 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:07.111535    2532 scope.go:117] "RemoveContainer" containerID="b0783927736013c9ba1430dbebeebbbb53b795397a898b9c7ad4ee1798722948"
Sep 18 20:05:07.111743 ip-10-0-130-81 kubenswrapper[2532]: E0918 20:05:07.111726    2532 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-fd4bb46c-m7wdd_openshift-console-operator(32e33d39-7826-499d-bc01-0487b7580e7a)\"" pod="openshift-console-operator/console-operator-fd4bb46c-m7wdd" podUID="32e33d39-7826-499d-bc01-0487b7580e7a"
Sep 18 20:05:07.122779 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:07.122754    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:05:07.224109 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:07.224074    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:05:07.247194 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:07.247162    2532 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-vhbcf"]
Sep 18 20:05:07.251775 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:07.251744    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-vhbcf"
Sep 18 20:05:07.254250 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:07.254225    2532 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Sep 18 20:05:07.259614 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:07.259583    2532 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Sep 18 20:05:07.259768 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:07.259623    2532 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Sep 18 20:05:07.259768 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:07.259590    2532 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Sep 18 20:05:07.259768 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:07.259592    2532 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-z7xss\""
Sep 18 20:05:07.262873 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:07.262850    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-vhbcf"]
Sep 18 20:05:07.292174 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:07.292142    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:05:07.371630 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:07.371596    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c0ef2e11-9230-478a-982a-bae678c749b4-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-vhbcf\" (UID: \"c0ef2e11-9230-478a-982a-bae678c749b4\") " pod="openshift-insights/insights-runtime-extractor-vhbcf"
Sep 18 20:05:07.371791 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:07.371642    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/c0ef2e11-9230-478a-982a-bae678c749b4-crio-socket\") pod \"insights-runtime-extractor-vhbcf\" (UID: \"c0ef2e11-9230-478a-982a-bae678c749b4\") " pod="openshift-insights/insights-runtime-extractor-vhbcf"
Sep 18 20:05:07.371791 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:07.371703    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/c0ef2e11-9230-478a-982a-bae678c749b4-data-volume\") pod \"insights-runtime-extractor-vhbcf\" (UID: \"c0ef2e11-9230-478a-982a-bae678c749b4\") " pod="openshift-insights/insights-runtime-extractor-vhbcf"
Sep 18 20:05:07.371791 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:07.371754    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c0ef2e11-9230-478a-982a-bae678c749b4-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-vhbcf\" (UID: \"c0ef2e11-9230-478a-982a-bae678c749b4\") " pod="openshift-insights/insights-runtime-extractor-vhbcf"
Sep 18 20:05:07.371791 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:07.371788    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qv2k5\" (UniqueName: \"kubernetes.io/projected/c0ef2e11-9230-478a-982a-bae678c749b4-kube-api-access-qv2k5\") pod \"insights-runtime-extractor-vhbcf\" (UID: \"c0ef2e11-9230-478a-982a-bae678c749b4\") " pod="openshift-insights/insights-runtime-extractor-vhbcf"
Sep 18 20:05:07.382927 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:07.382887    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:05:07.440136 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:07.440105    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:05:07.472968 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:07.472942    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c0ef2e11-9230-478a-982a-bae678c749b4-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-vhbcf\" (UID: \"c0ef2e11-9230-478a-982a-bae678c749b4\") " pod="openshift-insights/insights-runtime-extractor-vhbcf"
Sep 18 20:05:07.473113 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:07.472985    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/c0ef2e11-9230-478a-982a-bae678c749b4-crio-socket\") pod \"insights-runtime-extractor-vhbcf\" (UID: \"c0ef2e11-9230-478a-982a-bae678c749b4\") " pod="openshift-insights/insights-runtime-extractor-vhbcf"
Sep 18 20:05:07.473113 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:07.473016    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/c0ef2e11-9230-478a-982a-bae678c749b4-data-volume\") pod \"insights-runtime-extractor-vhbcf\" (UID: \"c0ef2e11-9230-478a-982a-bae678c749b4\") " pod="openshift-insights/insights-runtime-extractor-vhbcf"
Sep 18 20:05:07.473113 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:07.473040    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c0ef2e11-9230-478a-982a-bae678c749b4-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-vhbcf\" (UID: \"c0ef2e11-9230-478a-982a-bae678c749b4\") " pod="openshift-insights/insights-runtime-extractor-vhbcf"
Sep 18 20:05:07.473113 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:07.473079    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qv2k5\" (UniqueName: \"kubernetes.io/projected/c0ef2e11-9230-478a-982a-bae678c749b4-kube-api-access-qv2k5\") pod \"insights-runtime-extractor-vhbcf\" (UID: \"c0ef2e11-9230-478a-982a-bae678c749b4\") " pod="openshift-insights/insights-runtime-extractor-vhbcf"
Sep 18 20:05:07.473113 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:07.473090    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/c0ef2e11-9230-478a-982a-bae678c749b4-crio-socket\") pod \"insights-runtime-extractor-vhbcf\" (UID: \"c0ef2e11-9230-478a-982a-bae678c749b4\") " pod="openshift-insights/insights-runtime-extractor-vhbcf"
Sep 18 20:05:07.473113 ip-10-0-130-81 kubenswrapper[2532]: E0918 20:05:07.473109    2532 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 18 20:05:07.473395 ip-10-0-130-81 kubenswrapper[2532]: E0918 20:05:07.473141    2532 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 18 20:05:07.473395 ip-10-0-130-81 kubenswrapper[2532]: E0918 20:05:07.473177    2532 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c0ef2e11-9230-478a-982a-bae678c749b4-insights-runtime-extractor-tls podName:c0ef2e11-9230-478a-982a-bae678c749b4 nodeName:}" failed. No retries permitted until 2026-09-18 20:05:07.973155906 +0000 UTC m=+239.738545294 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/c0ef2e11-9230-478a-982a-bae678c749b4-insights-runtime-extractor-tls") pod "insights-runtime-extractor-vhbcf" (UID: "c0ef2e11-9230-478a-982a-bae678c749b4") : secret "insights-runtime-extractor-tls" not found
Sep 18 20:05:07.473395 ip-10-0-130-81 kubenswrapper[2532]: E0918 20:05:07.473197    2532 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c0ef2e11-9230-478a-982a-bae678c749b4-kube-rbac-proxy-cm podName:c0ef2e11-9230-478a-982a-bae678c749b4 nodeName:}" failed. No retries permitted until 2026-09-18 20:05:07.973185002 +0000 UTC m=+239.738574378 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/c0ef2e11-9230-478a-982a-bae678c749b4-kube-rbac-proxy-cm") pod "insights-runtime-extractor-vhbcf" (UID: "c0ef2e11-9230-478a-982a-bae678c749b4") : configmap "kube-rbac-proxy" not found
Sep 18 20:05:07.473395 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:07.473341    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/c0ef2e11-9230-478a-982a-bae678c749b4-data-volume\") pod \"insights-runtime-extractor-vhbcf\" (UID: \"c0ef2e11-9230-478a-982a-bae678c749b4\") " pod="openshift-insights/insights-runtime-extractor-vhbcf"
Sep 18 20:05:07.483812 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:07.483767    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qv2k5\" (UniqueName: \"kubernetes.io/projected/c0ef2e11-9230-478a-982a-bae678c749b4-kube-api-access-qv2k5\") pod \"insights-runtime-extractor-vhbcf\" (UID: \"c0ef2e11-9230-478a-982a-bae678c749b4\") " pod="openshift-insights/insights-runtime-extractor-vhbcf"
Sep 18 20:05:07.876661 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:07.876580    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:05:07.927460 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:07.927424    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:05:07.976215 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:07.976183    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c0ef2e11-9230-478a-982a-bae678c749b4-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-vhbcf\" (UID: \"c0ef2e11-9230-478a-982a-bae678c749b4\") " pod="openshift-insights/insights-runtime-extractor-vhbcf"
Sep 18 20:05:07.976356 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:07.976235    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c0ef2e11-9230-478a-982a-bae678c749b4-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-vhbcf\" (UID: \"c0ef2e11-9230-478a-982a-bae678c749b4\") " pod="openshift-insights/insights-runtime-extractor-vhbcf"
Sep 18 20:05:07.976416 ip-10-0-130-81 kubenswrapper[2532]: E0918 20:05:07.976358    2532 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 18 20:05:07.976416 ip-10-0-130-81 kubenswrapper[2532]: E0918 20:05:07.976411    2532 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 18 20:05:07.976549 ip-10-0-130-81 kubenswrapper[2532]: E0918 20:05:07.976444    2532 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c0ef2e11-9230-478a-982a-bae678c749b4-insights-runtime-extractor-tls podName:c0ef2e11-9230-478a-982a-bae678c749b4 nodeName:}" failed. No retries permitted until 2026-09-18 20:05:08.976421129 +0000 UTC m=+240.741810522 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/c0ef2e11-9230-478a-982a-bae678c749b4-insights-runtime-extractor-tls") pod "insights-runtime-extractor-vhbcf" (UID: "c0ef2e11-9230-478a-982a-bae678c749b4") : secret "insights-runtime-extractor-tls" not found
Sep 18 20:05:07.976549 ip-10-0-130-81 kubenswrapper[2532]: E0918 20:05:07.976464    2532 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c0ef2e11-9230-478a-982a-bae678c749b4-kube-rbac-proxy-cm podName:c0ef2e11-9230-478a-982a-bae678c749b4 nodeName:}" failed. No retries permitted until 2026-09-18 20:05:08.976453008 +0000 UTC m=+240.741842385 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/c0ef2e11-9230-478a-982a-bae678c749b4-kube-rbac-proxy-cm") pod "insights-runtime-extractor-vhbcf" (UID: "c0ef2e11-9230-478a-982a-bae678c749b4") : configmap "kube-rbac-proxy" not found
Sep 18 20:05:08.090062 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:08.090026    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:05:08.116297 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:08.116272    2532 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-m7wdd_32e33d39-7826-499d-bc01-0487b7580e7a/console-operator/1.log"
Sep 18 20:05:08.116678 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:08.116590    2532 scope.go:117] "RemoveContainer" containerID="b0783927736013c9ba1430dbebeebbbb53b795397a898b9c7ad4ee1798722948"
Sep 18 20:05:08.116756 ip-10-0-130-81 kubenswrapper[2532]: E0918 20:05:08.116738    2532 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-fd4bb46c-m7wdd_openshift-console-operator(32e33d39-7826-499d-bc01-0487b7580e7a)\"" pod="openshift-console-operator/console-operator-fd4bb46c-m7wdd" podUID="32e33d39-7826-499d-bc01-0487b7580e7a"
Sep 18 20:05:08.145032 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:08.144957    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:05:08.985108 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:08.985061    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c0ef2e11-9230-478a-982a-bae678c749b4-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-vhbcf\" (UID: \"c0ef2e11-9230-478a-982a-bae678c749b4\") " pod="openshift-insights/insights-runtime-extractor-vhbcf"
Sep 18 20:05:08.985108 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:08.985123    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c0ef2e11-9230-478a-982a-bae678c749b4-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-vhbcf\" (UID: \"c0ef2e11-9230-478a-982a-bae678c749b4\") " pod="openshift-insights/insights-runtime-extractor-vhbcf"
Sep 18 20:05:08.985368 ip-10-0-130-81 kubenswrapper[2532]: E0918 20:05:08.985205    2532 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 18 20:05:08.985368 ip-10-0-130-81 kubenswrapper[2532]: E0918 20:05:08.985257    2532 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 18 20:05:08.985368 ip-10-0-130-81 kubenswrapper[2532]: E0918 20:05:08.985268    2532 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c0ef2e11-9230-478a-982a-bae678c749b4-insights-runtime-extractor-tls podName:c0ef2e11-9230-478a-982a-bae678c749b4 nodeName:}" failed. No retries permitted until 2026-09-18 20:05:10.985253248 +0000 UTC m=+242.750642625 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/c0ef2e11-9230-478a-982a-bae678c749b4-insights-runtime-extractor-tls") pod "insights-runtime-extractor-vhbcf" (UID: "c0ef2e11-9230-478a-982a-bae678c749b4") : secret "insights-runtime-extractor-tls" not found
Sep 18 20:05:08.985368 ip-10-0-130-81 kubenswrapper[2532]: E0918 20:05:08.985315    2532 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c0ef2e11-9230-478a-982a-bae678c749b4-kube-rbac-proxy-cm podName:c0ef2e11-9230-478a-982a-bae678c749b4 nodeName:}" failed. No retries permitted until 2026-09-18 20:05:10.985297615 +0000 UTC m=+242.750687010 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/c0ef2e11-9230-478a-982a-bae678c749b4-kube-rbac-proxy-cm") pod "insights-runtime-extractor-vhbcf" (UID: "c0ef2e11-9230-478a-982a-bae678c749b4") : configmap "kube-rbac-proxy" not found
Sep 18 20:05:09.688307 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:09.688273    2532 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-55f6fc7b86-vql6p"]
Sep 18 20:05:09.691367 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:09.691351    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-55f6fc7b86-vql6p"
Sep 18 20:05:09.694268 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:09.694247    2532 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\""
Sep 18 20:05:09.694407 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:09.694299    2532 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\""
Sep 18 20:05:09.694461 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:09.694452    2532 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\""
Sep 18 20:05:09.695241 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:09.695224    2532 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-927hh\""
Sep 18 20:05:09.695311 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:09.695254    2532 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\""
Sep 18 20:05:09.700863 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:09.700842    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-55f6fc7b86-vql6p"]
Sep 18 20:05:09.726208 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:09.726174    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:05:09.792638 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:09.792607    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xzqr9\" (UniqueName: \"kubernetes.io/projected/5f5b3f82-0a24-42b3-b75c-739041cc6c34-kube-api-access-xzqr9\") pod \"service-ca-55f6fc7b86-vql6p\" (UID: \"5f5b3f82-0a24-42b3-b75c-739041cc6c34\") " pod="openshift-service-ca/service-ca-55f6fc7b86-vql6p"
Sep 18 20:05:09.792767 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:09.792645    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/5f5b3f82-0a24-42b3-b75c-739041cc6c34-signing-cabundle\") pod \"service-ca-55f6fc7b86-vql6p\" (UID: \"5f5b3f82-0a24-42b3-b75c-739041cc6c34\") " pod="openshift-service-ca/service-ca-55f6fc7b86-vql6p"
Sep 18 20:05:09.792767 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:09.792677    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/5f5b3f82-0a24-42b3-b75c-739041cc6c34-signing-key\") pod \"service-ca-55f6fc7b86-vql6p\" (UID: \"5f5b3f82-0a24-42b3-b75c-739041cc6c34\") " pod="openshift-service-ca/service-ca-55f6fc7b86-vql6p"
Sep 18 20:05:09.893798 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:09.893758    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xzqr9\" (UniqueName: \"kubernetes.io/projected/5f5b3f82-0a24-42b3-b75c-739041cc6c34-kube-api-access-xzqr9\") pod \"service-ca-55f6fc7b86-vql6p\" (UID: \"5f5b3f82-0a24-42b3-b75c-739041cc6c34\") " pod="openshift-service-ca/service-ca-55f6fc7b86-vql6p"
Sep 18 20:05:09.893798 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:09.893800    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/5f5b3f82-0a24-42b3-b75c-739041cc6c34-signing-cabundle\") pod \"service-ca-55f6fc7b86-vql6p\" (UID: \"5f5b3f82-0a24-42b3-b75c-739041cc6c34\") " pod="openshift-service-ca/service-ca-55f6fc7b86-vql6p"
Sep 18 20:05:09.894070 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:09.893930    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/5f5b3f82-0a24-42b3-b75c-739041cc6c34-signing-key\") pod \"service-ca-55f6fc7b86-vql6p\" (UID: \"5f5b3f82-0a24-42b3-b75c-739041cc6c34\") " pod="openshift-service-ca/service-ca-55f6fc7b86-vql6p"
Sep 18 20:05:09.894451 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:09.894427    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/5f5b3f82-0a24-42b3-b75c-739041cc6c34-signing-cabundle\") pod \"service-ca-55f6fc7b86-vql6p\" (UID: \"5f5b3f82-0a24-42b3-b75c-739041cc6c34\") " pod="openshift-service-ca/service-ca-55f6fc7b86-vql6p"
Sep 18 20:05:09.896210 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:09.896191    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/5f5b3f82-0a24-42b3-b75c-739041cc6c34-signing-key\") pod \"service-ca-55f6fc7b86-vql6p\" (UID: \"5f5b3f82-0a24-42b3-b75c-739041cc6c34\") " pod="openshift-service-ca/service-ca-55f6fc7b86-vql6p"
Sep 18 20:05:09.901571 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:09.901548    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xzqr9\" (UniqueName: \"kubernetes.io/projected/5f5b3f82-0a24-42b3-b75c-739041cc6c34-kube-api-access-xzqr9\") pod \"service-ca-55f6fc7b86-vql6p\" (UID: \"5f5b3f82-0a24-42b3-b75c-739041cc6c34\") " pod="openshift-service-ca/service-ca-55f6fc7b86-vql6p"
Sep 18 20:05:09.911330 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:09.911308    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:05:10.000312 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:10.000274    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-55f6fc7b86-vql6p"
Sep 18 20:05:10.011977 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:10.011944    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:05:10.015793 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:10.015776    2532 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-b8jbz_87c4bfcc-4ad2-4166-b5c9-1051bdd2717d/dns-node-resolver/0.log"
Sep 18 20:05:10.273962 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:10.273939    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-55f6fc7b86-vql6p"]
Sep 18 20:05:10.275924 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:05:10.275900    2532 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5f5b3f82_0a24_42b3_b75c_739041cc6c34.slice/crio-a00c34e88453fd243358b632ff68288fba4640fd138e10e1c956556c01119d91 WatchSource:0}: Error finding container a00c34e88453fd243358b632ff68288fba4640fd138e10e1c956556c01119d91: Status 404 returned error can't find the container with id a00c34e88453fd243358b632ff68288fba4640fd138e10e1c956556c01119d91
Sep 18 20:05:10.591334 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:10.591235    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:05:10.600418 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:10.600392    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/dba6d982-36df-45cd-bb26-4deb21429b12-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-847b8bd8d7-6k4dr\" (UID: \"dba6d982-36df-45cd-bb26-4deb21429b12\") " pod="openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-6k4dr"
Sep 18 20:05:10.600566 ip-10-0-130-81 kubenswrapper[2532]: E0918 20:05:10.600549    2532 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 18 20:05:10.600617 ip-10-0-130-81 kubenswrapper[2532]: E0918 20:05:10.600606    2532 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dba6d982-36df-45cd-bb26-4deb21429b12-cluster-monitoring-operator-tls podName:dba6d982-36df-45cd-bb26-4deb21429b12 nodeName:}" failed. No retries permitted until 2026-09-18 20:05:18.600591187 +0000 UTC m=+250.365980564 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/dba6d982-36df-45cd-bb26-4deb21429b12-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-847b8bd8d7-6k4dr" (UID: "dba6d982-36df-45cd-bb26-4deb21429b12") : secret "cluster-monitoring-operator-tls" not found
Sep 18 20:05:10.700989 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:10.700956    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/71a5442b-4ebc-4e32-9cea-5a5ada982958-samples-operator-tls\") pod \"cluster-samples-operator-685d7d68b5-qk5t7\" (UID: \"71a5442b-4ebc-4e32-9cea-5a5ada982958\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-qk5t7"
Sep 18 20:05:10.701372 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:10.701013    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/b09e90e3-0343-40d8-af09-03c0cdcfaf36-registry-tls\") pod \"image-registry-64795c474b-cmxpf\" (UID: \"b09e90e3-0343-40d8-af09-03c0cdcfaf36\") " pod="openshift-image-registry/image-registry-64795c474b-cmxpf"
Sep 18 20:05:10.701372 ip-10-0-130-81 kubenswrapper[2532]: E0918 20:05:10.701103    2532 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 18 20:05:10.701372 ip-10-0-130-81 kubenswrapper[2532]: E0918 20:05:10.701177    2532 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/71a5442b-4ebc-4e32-9cea-5a5ada982958-samples-operator-tls podName:71a5442b-4ebc-4e32-9cea-5a5ada982958 nodeName:}" failed. No retries permitted until 2026-09-18 20:05:18.701159555 +0000 UTC m=+250.466548934 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/71a5442b-4ebc-4e32-9cea-5a5ada982958-samples-operator-tls") pod "cluster-samples-operator-685d7d68b5-qk5t7" (UID: "71a5442b-4ebc-4e32-9cea-5a5ada982958") : secret "samples-operator-tls" not found
Sep 18 20:05:10.701372 ip-10-0-130-81 kubenswrapper[2532]: E0918 20:05:10.701180    2532 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 18 20:05:10.701372 ip-10-0-130-81 kubenswrapper[2532]: E0918 20:05:10.701199    2532 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-64795c474b-cmxpf: secret "image-registry-tls" not found
Sep 18 20:05:10.701372 ip-10-0-130-81 kubenswrapper[2532]: E0918 20:05:10.701250    2532 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b09e90e3-0343-40d8-af09-03c0cdcfaf36-registry-tls podName:b09e90e3-0343-40d8-af09-03c0cdcfaf36 nodeName:}" failed. No retries permitted until 2026-09-18 20:05:18.701233673 +0000 UTC m=+250.466623057 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/b09e90e3-0343-40d8-af09-03c0cdcfaf36-registry-tls") pod "image-registry-64795c474b-cmxpf" (UID: "b09e90e3-0343-40d8-af09-03c0cdcfaf36") : secret "image-registry-tls" not found
Sep 18 20:05:11.003176 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:11.003143    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c0ef2e11-9230-478a-982a-bae678c749b4-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-vhbcf\" (UID: \"c0ef2e11-9230-478a-982a-bae678c749b4\") " pod="openshift-insights/insights-runtime-extractor-vhbcf"
Sep 18 20:05:11.003451 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:11.003198    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c0ef2e11-9230-478a-982a-bae678c749b4-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-vhbcf\" (UID: \"c0ef2e11-9230-478a-982a-bae678c749b4\") " pod="openshift-insights/insights-runtime-extractor-vhbcf"
Sep 18 20:05:11.003451 ip-10-0-130-81 kubenswrapper[2532]: E0918 20:05:11.003290    2532 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 18 20:05:11.003451 ip-10-0-130-81 kubenswrapper[2532]: E0918 20:05:11.003300    2532 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 18 20:05:11.003451 ip-10-0-130-81 kubenswrapper[2532]: E0918 20:05:11.003348    2532 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c0ef2e11-9230-478a-982a-bae678c749b4-kube-rbac-proxy-cm podName:c0ef2e11-9230-478a-982a-bae678c749b4 nodeName:}" failed. No retries permitted until 2026-09-18 20:05:15.003332225 +0000 UTC m=+246.768721603 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/c0ef2e11-9230-478a-982a-bae678c749b4-kube-rbac-proxy-cm") pod "insights-runtime-extractor-vhbcf" (UID: "c0ef2e11-9230-478a-982a-bae678c749b4") : configmap "kube-rbac-proxy" not found
Sep 18 20:05:11.003451 ip-10-0-130-81 kubenswrapper[2532]: E0918 20:05:11.003361    2532 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c0ef2e11-9230-478a-982a-bae678c749b4-insights-runtime-extractor-tls podName:c0ef2e11-9230-478a-982a-bae678c749b4 nodeName:}" failed. No retries permitted until 2026-09-18 20:05:15.003355442 +0000 UTC m=+246.768744819 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/c0ef2e11-9230-478a-982a-bae678c749b4-insights-runtime-extractor-tls") pod "insights-runtime-extractor-vhbcf" (UID: "c0ef2e11-9230-478a-982a-bae678c749b4") : secret "insights-runtime-extractor-tls" not found
Sep 18 20:05:11.127401 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:11.127360    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-55f6fc7b86-vql6p" event={"ID":"5f5b3f82-0a24-42b3-b75c-739041cc6c34","Type":"ContainerStarted","Data":"aac411c900350358d4253dbc8c08a2b143e6c7b3437ff07e49868f2f07cd5a34"}
Sep 18 20:05:11.127592 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:11.127418    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-55f6fc7b86-vql6p" event={"ID":"5f5b3f82-0a24-42b3-b75c-739041cc6c34","Type":"ContainerStarted","Data":"a00c34e88453fd243358b632ff68288fba4640fd138e10e1c956556c01119d91"}
Sep 18 20:05:11.144500 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:11.144439    2532 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-55f6fc7b86-vql6p" podStartSLOduration=2.144425157 podStartE2EDuration="2.144425157s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 20:05:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 20:05:11.143707157 +0000 UTC m=+242.909096557" watchObservedRunningTime="2026-09-18 20:05:11.144425157 +0000 UTC m=+242.909814556"
Sep 18 20:05:11.216768 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:11.216740    2532 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-wkwnc_b771ff0e-edf5-4d81-adf7-6350fe1ea781/node-ca/0.log"
Sep 18 20:05:11.652578 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:11.652542    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:05:11.797475 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:11.797426    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:05:11.817413 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:11.817371    2532 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-qd64x_60bb026c-35d3-4f8b-bdb8-fb0b698c024c/serve-healthcheck-canary/0.log"
Sep 18 20:05:11.923575 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:11.923467    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:05:12.005153 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:12.005117    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:05:12.749284 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:12.749249    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:05:13.084305 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:13.084215    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:05:13.238949 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:13.238914    2532 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-fd4bb46c-m7wdd"
Sep 18 20:05:13.239147 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:13.238968    2532 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-fd4bb46c-m7wdd"
Sep 18 20:05:13.239290 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:13.239278    2532 scope.go:117] "RemoveContainer" containerID="b0783927736013c9ba1430dbebeebbbb53b795397a898b9c7ad4ee1798722948"
Sep 18 20:05:13.239425 ip-10-0-130-81 kubenswrapper[2532]: E0918 20:05:13.239409    2532 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-fd4bb46c-m7wdd_openshift-console-operator(32e33d39-7826-499d-bc01-0487b7580e7a)\"" pod="openshift-console-operator/console-operator-fd4bb46c-m7wdd" podUID="32e33d39-7826-499d-bc01-0487b7580e7a"
Sep 18 20:05:13.362524 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:13.362420    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:05:13.557532 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:13.557504    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:05:13.756978 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:13.756927    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:05:13.975554 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:13.975528    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:05:15.037573 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:15.037505    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c0ef2e11-9230-478a-982a-bae678c749b4-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-vhbcf\" (UID: \"c0ef2e11-9230-478a-982a-bae678c749b4\") " pod="openshift-insights/insights-runtime-extractor-vhbcf"
Sep 18 20:05:15.037954 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:15.037618    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c0ef2e11-9230-478a-982a-bae678c749b4-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-vhbcf\" (UID: \"c0ef2e11-9230-478a-982a-bae678c749b4\") " pod="openshift-insights/insights-runtime-extractor-vhbcf"
Sep 18 20:05:15.037954 ip-10-0-130-81 kubenswrapper[2532]: E0918 20:05:15.037643    2532 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 18 20:05:15.037954 ip-10-0-130-81 kubenswrapper[2532]: E0918 20:05:15.037720    2532 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c0ef2e11-9230-478a-982a-bae678c749b4-insights-runtime-extractor-tls podName:c0ef2e11-9230-478a-982a-bae678c749b4 nodeName:}" failed. No retries permitted until 2026-09-18 20:05:23.037706293 +0000 UTC m=+254.803095674 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/c0ef2e11-9230-478a-982a-bae678c749b4-insights-runtime-extractor-tls") pod "insights-runtime-extractor-vhbcf" (UID: "c0ef2e11-9230-478a-982a-bae678c749b4") : secret "insights-runtime-extractor-tls" not found
Sep 18 20:05:15.037954 ip-10-0-130-81 kubenswrapper[2532]: E0918 20:05:15.037762    2532 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 18 20:05:15.037954 ip-10-0-130-81 kubenswrapper[2532]: E0918 20:05:15.037837    2532 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c0ef2e11-9230-478a-982a-bae678c749b4-kube-rbac-proxy-cm podName:c0ef2e11-9230-478a-982a-bae678c749b4 nodeName:}" failed. No retries permitted until 2026-09-18 20:05:23.037822387 +0000 UTC m=+254.803211764 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/c0ef2e11-9230-478a-982a-bae678c749b4-kube-rbac-proxy-cm") pod "insights-runtime-extractor-vhbcf" (UID: "c0ef2e11-9230-478a-982a-bae678c749b4") : configmap "kube-rbac-proxy" not found
Sep 18 20:05:15.127815 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:15.127779    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:05:15.199286 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:15.199252    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:05:15.349714 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:15.349646    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:05:15.540283 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:15.540250    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:05:15.589739 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:15.589705    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:05:15.746789 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:15.746757    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:05:15.951928 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:15.951887    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:05:16.944694 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:16.944663    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:05:17.314651 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:17.314621    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:05:17.416333 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:17.416304    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:05:17.551525 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:17.551498    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:05:17.755921 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:17.755889    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:05:17.946104 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:17.946072    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:05:18.113641 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:18.113559    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:05:18.666645 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:18.666614    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/dba6d982-36df-45cd-bb26-4deb21429b12-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-847b8bd8d7-6k4dr\" (UID: \"dba6d982-36df-45cd-bb26-4deb21429b12\") " pod="openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-6k4dr"
Sep 18 20:05:18.666806 ip-10-0-130-81 kubenswrapper[2532]: E0918 20:05:18.666718    2532 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 18 20:05:18.666806 ip-10-0-130-81 kubenswrapper[2532]: E0918 20:05:18.666776    2532 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dba6d982-36df-45cd-bb26-4deb21429b12-cluster-monitoring-operator-tls podName:dba6d982-36df-45cd-bb26-4deb21429b12 nodeName:}" failed. No retries permitted until 2026-09-18 20:05:34.6667624 +0000 UTC m=+266.432151776 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/dba6d982-36df-45cd-bb26-4deb21429b12-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-847b8bd8d7-6k4dr" (UID: "dba6d982-36df-45cd-bb26-4deb21429b12") : secret "cluster-monitoring-operator-tls" not found
Sep 18 20:05:18.767760 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:18.767724    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/b09e90e3-0343-40d8-af09-03c0cdcfaf36-registry-tls\") pod \"image-registry-64795c474b-cmxpf\" (UID: \"b09e90e3-0343-40d8-af09-03c0cdcfaf36\") " pod="openshift-image-registry/image-registry-64795c474b-cmxpf"
Sep 18 20:05:18.767947 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:18.767864    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/71a5442b-4ebc-4e32-9cea-5a5ada982958-samples-operator-tls\") pod \"cluster-samples-operator-685d7d68b5-qk5t7\" (UID: \"71a5442b-4ebc-4e32-9cea-5a5ada982958\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-qk5t7"
Sep 18 20:05:18.770086 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:18.770062    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/71a5442b-4ebc-4e32-9cea-5a5ada982958-samples-operator-tls\") pod \"cluster-samples-operator-685d7d68b5-qk5t7\" (UID: \"71a5442b-4ebc-4e32-9cea-5a5ada982958\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-qk5t7"
Sep 18 20:05:18.770086 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:18.770074    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/b09e90e3-0343-40d8-af09-03c0cdcfaf36-registry-tls\") pod \"image-registry-64795c474b-cmxpf\" (UID: \"b09e90e3-0343-40d8-af09-03c0cdcfaf36\") " pod="openshift-image-registry/image-registry-64795c474b-cmxpf"
Sep 18 20:05:18.782826 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:18.782797    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:05:18.826858 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:18.826833    2532 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-hn6xx\""
Sep 18 20:05:18.834840 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:18.834818    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-qk5t7"
Sep 18 20:05:18.846151 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:18.846132    2532 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-t5jb8\""
Sep 18 20:05:18.854334 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:18.854305    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-64795c474b-cmxpf"
Sep 18 20:05:19.102623 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:19.102593    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-qk5t7"]
Sep 18 20:05:19.129361 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:19.129338    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-64795c474b-cmxpf"]
Sep 18 20:05:19.131930 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:05:19.131903    2532 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb09e90e3_0343_40d8_af09_03c0cdcfaf36.slice/crio-98743b4ed8fc615d18c37bcdc4b80738f561b807af4b10d578b4545d6c2f3d4d WatchSource:0}: Error finding container 98743b4ed8fc615d18c37bcdc4b80738f561b807af4b10d578b4545d6c2f3d4d: Status 404 returned error can't find the container with id 98743b4ed8fc615d18c37bcdc4b80738f561b807af4b10d578b4545d6c2f3d4d
Sep 18 20:05:19.147511 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:19.147477    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:05:19.154701 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:19.154671    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-64795c474b-cmxpf" event={"ID":"b09e90e3-0343-40d8-af09-03c0cdcfaf36","Type":"ContainerStarted","Data":"98743b4ed8fc615d18c37bcdc4b80738f561b807af4b10d578b4545d6c2f3d4d"}
Sep 18 20:05:19.155662 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:19.155641    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-qk5t7" event={"ID":"71a5442b-4ebc-4e32-9cea-5a5ada982958","Type":"ContainerStarted","Data":"5bd6abf62452b878a0b1c2247259e00d1f85a6a2f9d5b5df0b974a5f08d7d7f3"}
Sep 18 20:05:19.343650 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:19.343614    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:05:19.547164 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:19.547131    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:05:19.750851 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:19.750790    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:05:19.810285 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:19.810208    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:05:19.946968 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:19.946933    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:05:20.161512 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:20.161356    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-64795c474b-cmxpf" event={"ID":"b09e90e3-0343-40d8-af09-03c0cdcfaf36","Type":"ContainerStarted","Data":"2c8253355e6fff30052637655e4509c52a9417482433a6be0db8bfc71e7fe503"}
Sep 18 20:05:20.161512 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:20.161501    2532 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-64795c474b-cmxpf"
Sep 18 20:05:20.180911 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:20.180862    2532 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-64795c474b-cmxpf" podStartSLOduration=18.180819153 podStartE2EDuration="18.180819153s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 20:05:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 20:05:20.180011791 +0000 UTC m=+251.945401190" watchObservedRunningTime="2026-09-18 20:05:20.180819153 +0000 UTC m=+251.946208548"
Sep 18 20:05:20.748944 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:20.748907    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:05:21.165973 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:21.165930    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-qk5t7" event={"ID":"71a5442b-4ebc-4e32-9cea-5a5ada982958","Type":"ContainerStarted","Data":"de0814e7840605b67ae85a79b3d832fa44eb7110f3e7336b8dc22bec8df571e2"}
Sep 18 20:05:21.166439 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:21.165989    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-qk5t7" event={"ID":"71a5442b-4ebc-4e32-9cea-5a5ada982958","Type":"ContainerStarted","Data":"edb5340fd6e2682324bb6e0929694980ed6e351f0d150f4afda929b8e7e74a07"}
Sep 18 20:05:21.182103 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:21.182061    2532 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-qk5t7" podStartSLOduration=17.388245967 podStartE2EDuration="19.182047312s" totalImagesPullingTime="1.793801345s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 20:05:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 20:05:21.180848831 +0000 UTC m=+252.946238235" watchObservedRunningTime="2026-09-18 20:05:21.182047312 +0000 UTC m=+252.947436711"
Sep 18 20:05:21.185505 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:21.185463    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:05:21.542636 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:21.542603    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:05:21.748662 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:21.748634    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:05:22.095857 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:22.095827    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/96167e3f-fb44-48e0-b86e-419a4a3b117c-metrics-certs\") pod \"network-metrics-daemon-xx7f7\" (UID: \"96167e3f-fb44-48e0-b86e-419a4a3b117c\") " pod="openshift-multus/network-metrics-daemon-xx7f7"
Sep 18 20:05:22.098123 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:22.098105    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/96167e3f-fb44-48e0-b86e-419a4a3b117c-metrics-certs\") pod \"network-metrics-daemon-xx7f7\" (UID: \"96167e3f-fb44-48e0-b86e-419a4a3b117c\") " pod="openshift-multus/network-metrics-daemon-xx7f7"
Sep 18 20:05:22.103433 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:22.103414    2532 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-ms6pd\""
Sep 18 20:05:22.111972 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:22.111951    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xx7f7"
Sep 18 20:05:22.149333 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:22.149298    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:05:22.347570 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:22.347444    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:05:22.377641 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:22.377616    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-xx7f7"]
Sep 18 20:05:22.380326 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:05:22.380303    2532 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod96167e3f_fb44_48e0_b86e_419a4a3b117c.slice/crio-d7dd8fd0dca088d4e889fe5a5b0df713a56de5a86667eceb003682a0664a6a6d WatchSource:0}: Error finding container d7dd8fd0dca088d4e889fe5a5b0df713a56de5a86667eceb003682a0664a6a6d: Status 404 returned error can't find the container with id d7dd8fd0dca088d4e889fe5a5b0df713a56de5a86667eceb003682a0664a6a6d
Sep 18 20:05:23.104115 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:23.104078    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c0ef2e11-9230-478a-982a-bae678c749b4-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-vhbcf\" (UID: \"c0ef2e11-9230-478a-982a-bae678c749b4\") " pod="openshift-insights/insights-runtime-extractor-vhbcf"
Sep 18 20:05:23.104307 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:23.104148    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c0ef2e11-9230-478a-982a-bae678c749b4-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-vhbcf\" (UID: \"c0ef2e11-9230-478a-982a-bae678c749b4\") " pod="openshift-insights/insights-runtime-extractor-vhbcf"
Sep 18 20:05:23.104761 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:23.104736    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c0ef2e11-9230-478a-982a-bae678c749b4-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-vhbcf\" (UID: \"c0ef2e11-9230-478a-982a-bae678c749b4\") " pod="openshift-insights/insights-runtime-extractor-vhbcf"
Sep 18 20:05:23.106830 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:23.106805    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c0ef2e11-9230-478a-982a-bae678c749b4-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-vhbcf\" (UID: \"c0ef2e11-9230-478a-982a-bae678c749b4\") " pod="openshift-insights/insights-runtime-extractor-vhbcf"
Sep 18 20:05:23.168943 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:23.168910    2532 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-z7xss\""
Sep 18 20:05:23.174218 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:23.174192    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-xx7f7" event={"ID":"96167e3f-fb44-48e0-b86e-419a4a3b117c","Type":"ContainerStarted","Data":"d7dd8fd0dca088d4e889fe5a5b0df713a56de5a86667eceb003682a0664a6a6d"}
Sep 18 20:05:23.177350 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:23.177321    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-vhbcf"
Sep 18 20:05:23.451608 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:23.451541    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-vhbcf"]
Sep 18 20:05:23.454384 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:05:23.454360    2532 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc0ef2e11_9230_478a_982a_bae678c749b4.slice/crio-b534b1b61ceb37a846af8da53b7155e9f20a45e6f22bc592e30fb3b51f15ca39 WatchSource:0}: Error finding container b534b1b61ceb37a846af8da53b7155e9f20a45e6f22bc592e30fb3b51f15ca39: Status 404 returned error can't find the container with id b534b1b61ceb37a846af8da53b7155e9f20a45e6f22bc592e30fb3b51f15ca39
Sep 18 20:05:24.180566 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:24.180525    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-xx7f7" event={"ID":"96167e3f-fb44-48e0-b86e-419a4a3b117c","Type":"ContainerStarted","Data":"fa4731dd0b419b6f2d90f80bddee5c42ea7ba3f1c2421ab63a05c89f9ce61a08"}
Sep 18 20:05:24.180751 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:24.180592    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-xx7f7" event={"ID":"96167e3f-fb44-48e0-b86e-419a4a3b117c","Type":"ContainerStarted","Data":"4bff89519d261895d70da662dc013b8d8a86d65648efda1deaed21d43c193e2f"}
Sep 18 20:05:24.182415 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:24.182387    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-vhbcf" event={"ID":"c0ef2e11-9230-478a-982a-bae678c749b4","Type":"ContainerStarted","Data":"55014fedcdec2281c276f56d0831d192c564af4e50946ba659fc95fb3aa460d1"}
Sep 18 20:05:24.182568 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:24.182442    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-vhbcf" event={"ID":"c0ef2e11-9230-478a-982a-bae678c749b4","Type":"ContainerStarted","Data":"b534b1b61ceb37a846af8da53b7155e9f20a45e6f22bc592e30fb3b51f15ca39"}
Sep 18 20:05:24.196644 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:24.196597    2532 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-xx7f7" podStartSLOduration=253.1228531 podStartE2EDuration="4m14.196582556s" totalImagesPullingTime="1.073729456s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 20:01:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 20:05:24.195647938 +0000 UTC m=+255.961037336" watchObservedRunningTime="2026-09-18 20:05:24.196582556 +0000 UTC m=+255.961971979"
Sep 18 20:05:25.187630 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:25.187596    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-vhbcf" event={"ID":"c0ef2e11-9230-478a-982a-bae678c749b4","Type":"ContainerStarted","Data":"571c12d60d0be319215a765eace29e2935602663ac2ac875757a3aa9a9c94a48"}
Sep 18 20:05:25.300960 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:25.300928    2532 scope.go:117] "RemoveContainer" containerID="b0783927736013c9ba1430dbebeebbbb53b795397a898b9c7ad4ee1798722948"
Sep 18 20:05:26.192608 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:26.192573    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-vhbcf" event={"ID":"c0ef2e11-9230-478a-982a-bae678c749b4","Type":"ContainerStarted","Data":"a22f4a5ba3d4f0f72370a5f0337aac3bac7ef15fb6e9b8b6d17ab28f98dd04be"}
Sep 18 20:05:26.194512 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:26.194479    2532 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-m7wdd_32e33d39-7826-499d-bc01-0487b7580e7a/console-operator/1.log"
Sep 18 20:05:26.194609 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:26.194552    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-fd4bb46c-m7wdd" event={"ID":"32e33d39-7826-499d-bc01-0487b7580e7a","Type":"ContainerStarted","Data":"06b746564ae1aabadde572af3d9660edcdbb6d9458d11b4c9562484072feb746"}
Sep 18 20:05:26.194803 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:26.194787    2532 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-fd4bb46c-m7wdd"
Sep 18 20:05:26.214338 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:26.214294    2532 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-vhbcf" podStartSLOduration=17.162254498 podStartE2EDuration="19.214279958s" totalImagesPullingTime="2.05202546s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-18 20:05:07 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 20:05:26.213559436 +0000 UTC m=+257.978948864" watchObservedRunningTime="2026-09-18 20:05:26.214279958 +0000 UTC m=+257.979669356"
Sep 18 20:05:26.228301 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:26.228239    2532 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-fd4bb46c-m7wdd" podStartSLOduration=21.800649715 podStartE2EDuration="24.228220185s" totalImagesPullingTime="2.42757047s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 20:05:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 20:05:26.227931316 +0000 UTC m=+257.993320715" watchObservedRunningTime="2026-09-18 20:05:26.228220185 +0000 UTC m=+257.993609585"
Sep 18 20:05:26.356148 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:26.356118    2532 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-fd4bb46c-m7wdd"
Sep 18 20:05:26.552900 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:26.552869    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:05:26.654077 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:26.654037    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:05:26.705666 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:26.705634    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:05:26.839003 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:26.838912    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:05:27.208818 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:27.208783    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:05:27.270544 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:27.270513    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:05:27.414636 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:27.414605    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:05:27.471215 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:27.471126    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:05:27.623412 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:27.623381    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:05:27.809961 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:27.809877    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:05:27.864291 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:27.864266    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:05:28.230514 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:28.230468    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:05:28.424351 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:28.424319    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:05:28.484786 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:28.484712    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:05:29.045600 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:29.045569    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:05:29.104660 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:29.104633    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:05:29.470349 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:29.470316    2532 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-6594f4c75-5vxpb"]
Sep 18 20:05:29.473548 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:29.473530    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-6594f4c75-5vxpb"
Sep 18 20:05:29.476060 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:29.476040    2532 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Sep 18 20:05:29.476172 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:29.476042    2532 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Sep 18 20:05:29.476172 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:29.476042    2532 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-96zv2\""
Sep 18 20:05:29.485176 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:29.485155    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-6594f4c75-5vxpb"]
Sep 18 20:05:29.486547 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:29.486520    2532 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-64795c474b-cmxpf"]
Sep 18 20:05:29.495048 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:29.495020    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:05:29.520846 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:29.520811    2532 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-5546d96886-g8mqv"]
Sep 18 20:05:29.523848 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:29.523829    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5546d96886-g8mqv"
Sep 18 20:05:29.537885 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:29.537864    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-5546d96886-g8mqv"]
Sep 18 20:05:29.584338 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:29.584309    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:05:29.657177 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:29.657149    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/20b90eb2-bbca-49d7-b2f3-c8be70bc619e-registry-tls\") pod \"image-registry-5546d96886-g8mqv\" (UID: \"20b90eb2-bbca-49d7-b2f3-c8be70bc619e\") " pod="openshift-image-registry/image-registry-5546d96886-g8mqv"
Sep 18 20:05:29.657177 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:29.657181    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/20b90eb2-bbca-49d7-b2f3-c8be70bc619e-bound-sa-token\") pod \"image-registry-5546d96886-g8mqv\" (UID: \"20b90eb2-bbca-49d7-b2f3-c8be70bc619e\") " pod="openshift-image-registry/image-registry-5546d96886-g8mqv"
Sep 18 20:05:29.657405 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:29.657200    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j85cp\" (UniqueName: \"kubernetes.io/projected/3992f04b-9fe3-4dcd-bcbc-8baf2889081c-kube-api-access-j85cp\") pod \"downloads-6594f4c75-5vxpb\" (UID: \"3992f04b-9fe3-4dcd-bcbc-8baf2889081c\") " pod="openshift-console/downloads-6594f4c75-5vxpb"
Sep 18 20:05:29.657405 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:29.657221    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/20b90eb2-bbca-49d7-b2f3-c8be70bc619e-ca-trust-extracted\") pod \"image-registry-5546d96886-g8mqv\" (UID: \"20b90eb2-bbca-49d7-b2f3-c8be70bc619e\") " pod="openshift-image-registry/image-registry-5546d96886-g8mqv"
Sep 18 20:05:29.657405 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:29.657295    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wtpfr\" (UniqueName: \"kubernetes.io/projected/20b90eb2-bbca-49d7-b2f3-c8be70bc619e-kube-api-access-wtpfr\") pod \"image-registry-5546d96886-g8mqv\" (UID: \"20b90eb2-bbca-49d7-b2f3-c8be70bc619e\") " pod="openshift-image-registry/image-registry-5546d96886-g8mqv"
Sep 18 20:05:29.657405 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:29.657333    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/20b90eb2-bbca-49d7-b2f3-c8be70bc619e-image-registry-private-configuration\") pod \"image-registry-5546d96886-g8mqv\" (UID: \"20b90eb2-bbca-49d7-b2f3-c8be70bc619e\") " pod="openshift-image-registry/image-registry-5546d96886-g8mqv"
Sep 18 20:05:29.657405 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:29.657352    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/20b90eb2-bbca-49d7-b2f3-c8be70bc619e-installation-pull-secrets\") pod \"image-registry-5546d96886-g8mqv\" (UID: \"20b90eb2-bbca-49d7-b2f3-c8be70bc619e\") " pod="openshift-image-registry/image-registry-5546d96886-g8mqv"
Sep 18 20:05:29.657405 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:29.657374    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/20b90eb2-bbca-49d7-b2f3-c8be70bc619e-trusted-ca\") pod \"image-registry-5546d96886-g8mqv\" (UID: \"20b90eb2-bbca-49d7-b2f3-c8be70bc619e\") " pod="openshift-image-registry/image-registry-5546d96886-g8mqv"
Sep 18 20:05:29.657659 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:29.657452    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/20b90eb2-bbca-49d7-b2f3-c8be70bc619e-registry-certificates\") pod \"image-registry-5546d96886-g8mqv\" (UID: \"20b90eb2-bbca-49d7-b2f3-c8be70bc619e\") " pod="openshift-image-registry/image-registry-5546d96886-g8mqv"
Sep 18 20:05:29.684662 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:29.684633    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:05:29.758342 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:29.758260    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/20b90eb2-bbca-49d7-b2f3-c8be70bc619e-ca-trust-extracted\") pod \"image-registry-5546d96886-g8mqv\" (UID: \"20b90eb2-bbca-49d7-b2f3-c8be70bc619e\") " pod="openshift-image-registry/image-registry-5546d96886-g8mqv"
Sep 18 20:05:29.758342 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:29.758300    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wtpfr\" (UniqueName: \"kubernetes.io/projected/20b90eb2-bbca-49d7-b2f3-c8be70bc619e-kube-api-access-wtpfr\") pod \"image-registry-5546d96886-g8mqv\" (UID: \"20b90eb2-bbca-49d7-b2f3-c8be70bc619e\") " pod="openshift-image-registry/image-registry-5546d96886-g8mqv"
Sep 18 20:05:29.758342 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:29.758337    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/20b90eb2-bbca-49d7-b2f3-c8be70bc619e-image-registry-private-configuration\") pod \"image-registry-5546d96886-g8mqv\" (UID: \"20b90eb2-bbca-49d7-b2f3-c8be70bc619e\") " pod="openshift-image-registry/image-registry-5546d96886-g8mqv"
Sep 18 20:05:29.758649 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:29.758357    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/20b90eb2-bbca-49d7-b2f3-c8be70bc619e-installation-pull-secrets\") pod \"image-registry-5546d96886-g8mqv\" (UID: \"20b90eb2-bbca-49d7-b2f3-c8be70bc619e\") " pod="openshift-image-registry/image-registry-5546d96886-g8mqv"
Sep 18 20:05:29.758649 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:29.758379    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/20b90eb2-bbca-49d7-b2f3-c8be70bc619e-trusted-ca\") pod \"image-registry-5546d96886-g8mqv\" (UID: \"20b90eb2-bbca-49d7-b2f3-c8be70bc619e\") " pod="openshift-image-registry/image-registry-5546d96886-g8mqv"
Sep 18 20:05:29.758649 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:29.758552    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/20b90eb2-bbca-49d7-b2f3-c8be70bc619e-registry-certificates\") pod \"image-registry-5546d96886-g8mqv\" (UID: \"20b90eb2-bbca-49d7-b2f3-c8be70bc619e\") " pod="openshift-image-registry/image-registry-5546d96886-g8mqv"
Sep 18 20:05:29.758649 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:29.758607    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/20b90eb2-bbca-49d7-b2f3-c8be70bc619e-registry-tls\") pod \"image-registry-5546d96886-g8mqv\" (UID: \"20b90eb2-bbca-49d7-b2f3-c8be70bc619e\") " pod="openshift-image-registry/image-registry-5546d96886-g8mqv"
Sep 18 20:05:29.758649 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:29.758632    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/20b90eb2-bbca-49d7-b2f3-c8be70bc619e-bound-sa-token\") pod \"image-registry-5546d96886-g8mqv\" (UID: \"20b90eb2-bbca-49d7-b2f3-c8be70bc619e\") " pod="openshift-image-registry/image-registry-5546d96886-g8mqv"
Sep 18 20:05:29.758891 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:29.758653    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j85cp\" (UniqueName: \"kubernetes.io/projected/3992f04b-9fe3-4dcd-bcbc-8baf2889081c-kube-api-access-j85cp\") pod \"downloads-6594f4c75-5vxpb\" (UID: \"3992f04b-9fe3-4dcd-bcbc-8baf2889081c\") " pod="openshift-console/downloads-6594f4c75-5vxpb"
Sep 18 20:05:29.759088 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:29.759067    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/20b90eb2-bbca-49d7-b2f3-c8be70bc619e-ca-trust-extracted\") pod \"image-registry-5546d96886-g8mqv\" (UID: \"20b90eb2-bbca-49d7-b2f3-c8be70bc619e\") " pod="openshift-image-registry/image-registry-5546d96886-g8mqv"
Sep 18 20:05:29.759348 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:29.759328    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/20b90eb2-bbca-49d7-b2f3-c8be70bc619e-trusted-ca\") pod \"image-registry-5546d96886-g8mqv\" (UID: \"20b90eb2-bbca-49d7-b2f3-c8be70bc619e\") " pod="openshift-image-registry/image-registry-5546d96886-g8mqv"
Sep 18 20:05:29.759657 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:29.759632    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/20b90eb2-bbca-49d7-b2f3-c8be70bc619e-registry-certificates\") pod \"image-registry-5546d96886-g8mqv\" (UID: \"20b90eb2-bbca-49d7-b2f3-c8be70bc619e\") " pod="openshift-image-registry/image-registry-5546d96886-g8mqv"
Sep 18 20:05:29.760946 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:29.760927    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/20b90eb2-bbca-49d7-b2f3-c8be70bc619e-installation-pull-secrets\") pod \"image-registry-5546d96886-g8mqv\" (UID: \"20b90eb2-bbca-49d7-b2f3-c8be70bc619e\") " pod="openshift-image-registry/image-registry-5546d96886-g8mqv"
Sep 18 20:05:29.761039 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:29.760965    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/20b90eb2-bbca-49d7-b2f3-c8be70bc619e-image-registry-private-configuration\") pod \"image-registry-5546d96886-g8mqv\" (UID: \"20b90eb2-bbca-49d7-b2f3-c8be70bc619e\") " pod="openshift-image-registry/image-registry-5546d96886-g8mqv"
Sep 18 20:05:29.761187 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:29.761168    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/20b90eb2-bbca-49d7-b2f3-c8be70bc619e-registry-tls\") pod \"image-registry-5546d96886-g8mqv\" (UID: \"20b90eb2-bbca-49d7-b2f3-c8be70bc619e\") " pod="openshift-image-registry/image-registry-5546d96886-g8mqv"
Sep 18 20:05:29.767012 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:29.766970    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wtpfr\" (UniqueName: \"kubernetes.io/projected/20b90eb2-bbca-49d7-b2f3-c8be70bc619e-kube-api-access-wtpfr\") pod \"image-registry-5546d96886-g8mqv\" (UID: \"20b90eb2-bbca-49d7-b2f3-c8be70bc619e\") " pod="openshift-image-registry/image-registry-5546d96886-g8mqv"
Sep 18 20:05:29.767012 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:29.766984    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j85cp\" (UniqueName: \"kubernetes.io/projected/3992f04b-9fe3-4dcd-bcbc-8baf2889081c-kube-api-access-j85cp\") pod \"downloads-6594f4c75-5vxpb\" (UID: \"3992f04b-9fe3-4dcd-bcbc-8baf2889081c\") " pod="openshift-console/downloads-6594f4c75-5vxpb"
Sep 18 20:05:29.767373 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:29.767350    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/20b90eb2-bbca-49d7-b2f3-c8be70bc619e-bound-sa-token\") pod \"image-registry-5546d96886-g8mqv\" (UID: \"20b90eb2-bbca-49d7-b2f3-c8be70bc619e\") " pod="openshift-image-registry/image-registry-5546d96886-g8mqv"
Sep 18 20:05:29.782297 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:29.782271    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-6594f4c75-5vxpb"
Sep 18 20:05:29.832834 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:29.832804    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5546d96886-g8mqv"
Sep 18 20:05:30.050947 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:30.050734    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-6594f4c75-5vxpb"]
Sep 18 20:05:30.053267 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:05:30.053242    2532 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3992f04b_9fe3_4dcd_bcbc_8baf2889081c.slice/crio-2a0a1057cd18f2abe3962e09e6642e11bb8cf664cbc107b700245d2fe37908aa WatchSource:0}: Error finding container 2a0a1057cd18f2abe3962e09e6642e11bb8cf664cbc107b700245d2fe37908aa: Status 404 returned error can't find the container with id 2a0a1057cd18f2abe3962e09e6642e11bb8cf664cbc107b700245d2fe37908aa
Sep 18 20:05:30.107187 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:30.107132    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-5546d96886-g8mqv"]
Sep 18 20:05:30.109660 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:05:30.109625    2532 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod20b90eb2_bbca_49d7_b2f3_c8be70bc619e.slice/crio-626ef532c8d3f5d3822dc037978cd7699cbe45acfc2aff993f4f908d35c6271e WatchSource:0}: Error finding container 626ef532c8d3f5d3822dc037978cd7699cbe45acfc2aff993f4f908d35c6271e: Status 404 returned error can't find the container with id 626ef532c8d3f5d3822dc037978cd7699cbe45acfc2aff993f4f908d35c6271e
Sep 18 20:05:30.209796 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:30.209763    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-6594f4c75-5vxpb" event={"ID":"3992f04b-9fe3-4dcd-bcbc-8baf2889081c","Type":"ContainerStarted","Data":"2a0a1057cd18f2abe3962e09e6642e11bb8cf664cbc107b700245d2fe37908aa"}
Sep 18 20:05:30.211249 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:30.211218    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5546d96886-g8mqv" event={"ID":"20b90eb2-bbca-49d7-b2f3-c8be70bc619e","Type":"ContainerStarted","Data":"90bbfe6c1520b51427d9a67a4266144af8ef6fc9ca96ef5b87c33e070703ab1d"}
Sep 18 20:05:30.211367 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:30.211268    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5546d96886-g8mqv" event={"ID":"20b90eb2-bbca-49d7-b2f3-c8be70bc619e","Type":"ContainerStarted","Data":"626ef532c8d3f5d3822dc037978cd7699cbe45acfc2aff993f4f908d35c6271e"}
Sep 18 20:05:30.211367 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:30.211309    2532 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-5546d96886-g8mqv"
Sep 18 20:05:30.230398 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:30.230347    2532 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-5546d96886-g8mqv" podStartSLOduration=1.230329907 podStartE2EDuration="1.230329907s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 20:05:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 20:05:30.228778771 +0000 UTC m=+261.994168171" watchObservedRunningTime="2026-09-18 20:05:30.230329907 +0000 UTC m=+261.995719306"
Sep 18 20:05:31.265841 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:31.265807    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:05:31.320348 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:31.320318    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:05:34.704084 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:34.704048    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/dba6d982-36df-45cd-bb26-4deb21429b12-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-847b8bd8d7-6k4dr\" (UID: \"dba6d982-36df-45cd-bb26-4deb21429b12\") " pod="openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-6k4dr"
Sep 18 20:05:34.706864 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:34.706834    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/dba6d982-36df-45cd-bb26-4deb21429b12-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-847b8bd8d7-6k4dr\" (UID: \"dba6d982-36df-45cd-bb26-4deb21429b12\") " pod="openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-6k4dr"
Sep 18 20:05:34.932417 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:34.932382    2532 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-9q76m\""
Sep 18 20:05:34.940430 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:34.940406    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-6k4dr"
Sep 18 20:05:35.218915 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:35.218825    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-6k4dr"]
Sep 18 20:05:35.222049 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:05:35.222018    2532 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddba6d982_36df_45cd_bb26_4deb21429b12.slice/crio-2bcf3a78022e6423a0311600a6c9140a1fe87798a4f04260147782d68e785233 WatchSource:0}: Error finding container 2bcf3a78022e6423a0311600a6c9140a1fe87798a4f04260147782d68e785233: Status 404 returned error can't find the container with id 2bcf3a78022e6423a0311600a6c9140a1fe87798a4f04260147782d68e785233
Sep 18 20:05:35.231143 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:35.231116    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-6k4dr" event={"ID":"dba6d982-36df-45cd-bb26-4deb21429b12","Type":"ContainerStarted","Data":"2bcf3a78022e6423a0311600a6c9140a1fe87798a4f04260147782d68e785233"}
Sep 18 20:05:37.137633 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:37.137595    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:05:37.191336 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:37.191303    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:05:37.535994 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:37.535955    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:05:37.638455 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:37.638422    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:05:37.720734 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:37.720687    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:05:38.036471 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:38.036437    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:05:38.246152 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:38.246109    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-6k4dr" event={"ID":"dba6d982-36df-45cd-bb26-4deb21429b12","Type":"ContainerStarted","Data":"29032e2806e8537ec166dfbf5246e2cb7d509358912d838321190c93ef40a9b5"}
Sep 18 20:05:38.264607 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:38.264537    2532 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-6k4dr" podStartSLOduration=34.294384167 podStartE2EDuration="36.264521543s" totalImagesPullingTime="1.970137376s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 20:05:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 20:05:38.263669289 +0000 UTC m=+270.029058692" watchObservedRunningTime="2026-09-18 20:05:38.264521543 +0000 UTC m=+270.029910938"
Sep 18 20:05:39.494256 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:39.494220    2532 patch_prober.go:28] interesting pod/image-registry-64795c474b-cmxpf container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Sep 18 20:05:39.494701 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:39.494290    2532 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-64795c474b-cmxpf" podUID="b09e90e3-0343-40d8-af09-03c0cdcfaf36" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 18 20:05:40.647998 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:40.647962    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:05:40.696970 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:40.696931    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:05:41.728993 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:41.728960    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:05:41.794274 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:41.794236    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:05:42.255798 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:42.255765    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:05:43.848350 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:43.848317    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:05:43.948408 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:43.948373    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:05:44.086935 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:44.086891    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:05:44.188173 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:44.188095    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:05:44.800694 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:44.800666    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:05:46.277809 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:46.277758    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-6594f4c75-5vxpb" event={"ID":"3992f04b-9fe3-4dcd-bcbc-8baf2889081c","Type":"ContainerStarted","Data":"dc69ea0b521cfb7117c0454ad265485b61034d7ad5cfa3df3930f2a8abaaa450"}
Sep 18 20:05:46.278243 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:46.278056    2532 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-6594f4c75-5vxpb"
Sep 18 20:05:46.287863 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:46.287834    2532 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-6594f4c75-5vxpb"
Sep 18 20:05:46.295382 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:46.295308    2532 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-6594f4c75-5vxpb" podStartSLOduration=1.287302644 podStartE2EDuration="17.295293689s" totalImagesPullingTime="16.007991045s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 20:05:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 20:05:46.293524059 +0000 UTC m=+278.058913455" watchObservedRunningTime="2026-09-18 20:05:46.295293689 +0000 UTC m=+278.060683130"
Sep 18 20:05:46.877667 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:46.877630    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:05:47.109660 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:47.109612    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:05:47.165686 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:47.165608    2532 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-trx7g"]
Sep 18 20:05:47.168657 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:47.168633    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-trx7g"
Sep 18 20:05:47.171893 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:47.171868    2532 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-bvmbl\""
Sep 18 20:05:47.173165 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:47.173147    2532 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Sep 18 20:05:47.173524 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:47.173147    2532 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Sep 18 20:05:47.173628 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:47.173199    2532 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Sep 18 20:05:47.173628 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:47.173259    2532 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Sep 18 20:05:47.203112 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:47.203081    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:05:47.213406 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:47.213357    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sbrcp\" (UniqueName: \"kubernetes.io/projected/73d422ef-a46d-46ec-baba-15a648a415fe-kube-api-access-sbrcp\") pod \"node-exporter-trx7g\" (UID: \"73d422ef-a46d-46ec-baba-15a648a415fe\") " pod="openshift-monitoring/node-exporter-trx7g"
Sep 18 20:05:47.213634 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:47.213420    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/73d422ef-a46d-46ec-baba-15a648a415fe-root\") pod \"node-exporter-trx7g\" (UID: \"73d422ef-a46d-46ec-baba-15a648a415fe\") " pod="openshift-monitoring/node-exporter-trx7g"
Sep 18 20:05:47.213634 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:47.213463    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/73d422ef-a46d-46ec-baba-15a648a415fe-sys\") pod \"node-exporter-trx7g\" (UID: \"73d422ef-a46d-46ec-baba-15a648a415fe\") " pod="openshift-monitoring/node-exporter-trx7g"
Sep 18 20:05:47.213634 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:47.213529    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/73d422ef-a46d-46ec-baba-15a648a415fe-metrics-client-ca\") pod \"node-exporter-trx7g\" (UID: \"73d422ef-a46d-46ec-baba-15a648a415fe\") " pod="openshift-monitoring/node-exporter-trx7g"
Sep 18 20:05:47.213634 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:47.213581    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/73d422ef-a46d-46ec-baba-15a648a415fe-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-trx7g\" (UID: \"73d422ef-a46d-46ec-baba-15a648a415fe\") " pod="openshift-monitoring/node-exporter-trx7g"
Sep 18 20:05:47.213634 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:47.213610    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/73d422ef-a46d-46ec-baba-15a648a415fe-node-exporter-textfile\") pod \"node-exporter-trx7g\" (UID: \"73d422ef-a46d-46ec-baba-15a648a415fe\") " pod="openshift-monitoring/node-exporter-trx7g"
Sep 18 20:05:47.213910 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:47.213663    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/73d422ef-a46d-46ec-baba-15a648a415fe-node-exporter-tls\") pod \"node-exporter-trx7g\" (UID: \"73d422ef-a46d-46ec-baba-15a648a415fe\") " pod="openshift-monitoring/node-exporter-trx7g"
Sep 18 20:05:47.213910 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:47.213690    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/73d422ef-a46d-46ec-baba-15a648a415fe-node-exporter-wtmp\") pod \"node-exporter-trx7g\" (UID: \"73d422ef-a46d-46ec-baba-15a648a415fe\") " pod="openshift-monitoring/node-exporter-trx7g"
Sep 18 20:05:47.213910 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:47.213762    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/73d422ef-a46d-46ec-baba-15a648a415fe-node-exporter-accelerators-collector-config\") pod \"node-exporter-trx7g\" (UID: \"73d422ef-a46d-46ec-baba-15a648a415fe\") " pod="openshift-monitoring/node-exporter-trx7g"
Sep 18 20:05:47.279139 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:47.278825    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:05:47.314802 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:47.314764    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/73d422ef-a46d-46ec-baba-15a648a415fe-node-exporter-tls\") pod \"node-exporter-trx7g\" (UID: \"73d422ef-a46d-46ec-baba-15a648a415fe\") " pod="openshift-monitoring/node-exporter-trx7g"
Sep 18 20:05:47.315060 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:47.315039    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/73d422ef-a46d-46ec-baba-15a648a415fe-node-exporter-wtmp\") pod \"node-exporter-trx7g\" (UID: \"73d422ef-a46d-46ec-baba-15a648a415fe\") " pod="openshift-monitoring/node-exporter-trx7g"
Sep 18 20:05:47.315371 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:47.315350    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/73d422ef-a46d-46ec-baba-15a648a415fe-node-exporter-accelerators-collector-config\") pod \"node-exporter-trx7g\" (UID: \"73d422ef-a46d-46ec-baba-15a648a415fe\") " pod="openshift-monitoring/node-exporter-trx7g"
Sep 18 20:05:47.315511 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:47.315292    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/73d422ef-a46d-46ec-baba-15a648a415fe-node-exporter-wtmp\") pod \"node-exporter-trx7g\" (UID: \"73d422ef-a46d-46ec-baba-15a648a415fe\") " pod="openshift-monitoring/node-exporter-trx7g"
Sep 18 20:05:47.315666 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:47.315649    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sbrcp\" (UniqueName: \"kubernetes.io/projected/73d422ef-a46d-46ec-baba-15a648a415fe-kube-api-access-sbrcp\") pod \"node-exporter-trx7g\" (UID: \"73d422ef-a46d-46ec-baba-15a648a415fe\") " pod="openshift-monitoring/node-exporter-trx7g"
Sep 18 20:05:47.316431 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:47.316411    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/73d422ef-a46d-46ec-baba-15a648a415fe-root\") pod \"node-exporter-trx7g\" (UID: \"73d422ef-a46d-46ec-baba-15a648a415fe\") " pod="openshift-monitoring/node-exporter-trx7g"
Sep 18 20:05:47.316652 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:47.316635    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/73d422ef-a46d-46ec-baba-15a648a415fe-sys\") pod \"node-exporter-trx7g\" (UID: \"73d422ef-a46d-46ec-baba-15a648a415fe\") " pod="openshift-monitoring/node-exporter-trx7g"
Sep 18 20:05:47.316836 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:47.316589    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/73d422ef-a46d-46ec-baba-15a648a415fe-root\") pod \"node-exporter-trx7g\" (UID: \"73d422ef-a46d-46ec-baba-15a648a415fe\") " pod="openshift-monitoring/node-exporter-trx7g"
Sep 18 20:05:47.316836 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:47.316810    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/73d422ef-a46d-46ec-baba-15a648a415fe-sys\") pod \"node-exporter-trx7g\" (UID: \"73d422ef-a46d-46ec-baba-15a648a415fe\") " pod="openshift-monitoring/node-exporter-trx7g"
Sep 18 20:05:47.316970 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:47.316032    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/73d422ef-a46d-46ec-baba-15a648a415fe-node-exporter-accelerators-collector-config\") pod \"node-exporter-trx7g\" (UID: \"73d422ef-a46d-46ec-baba-15a648a415fe\") " pod="openshift-monitoring/node-exporter-trx7g"
Sep 18 20:05:47.317112 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:47.317094    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/73d422ef-a46d-46ec-baba-15a648a415fe-metrics-client-ca\") pod \"node-exporter-trx7g\" (UID: \"73d422ef-a46d-46ec-baba-15a648a415fe\") " pod="openshift-monitoring/node-exporter-trx7g"
Sep 18 20:05:47.317243 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:47.317223    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/73d422ef-a46d-46ec-baba-15a648a415fe-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-trx7g\" (UID: \"73d422ef-a46d-46ec-baba-15a648a415fe\") " pod="openshift-monitoring/node-exporter-trx7g"
Sep 18 20:05:47.317330 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:47.317261    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/73d422ef-a46d-46ec-baba-15a648a415fe-node-exporter-textfile\") pod \"node-exporter-trx7g\" (UID: \"73d422ef-a46d-46ec-baba-15a648a415fe\") " pod="openshift-monitoring/node-exporter-trx7g"
Sep 18 20:05:47.317849 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:47.317827    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/73d422ef-a46d-46ec-baba-15a648a415fe-node-exporter-tls\") pod \"node-exporter-trx7g\" (UID: \"73d422ef-a46d-46ec-baba-15a648a415fe\") " pod="openshift-monitoring/node-exporter-trx7g"
Sep 18 20:05:47.317998 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:47.317833    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/73d422ef-a46d-46ec-baba-15a648a415fe-node-exporter-textfile\") pod \"node-exporter-trx7g\" (UID: \"73d422ef-a46d-46ec-baba-15a648a415fe\") " pod="openshift-monitoring/node-exporter-trx7g"
Sep 18 20:05:47.318119 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:47.317944    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/73d422ef-a46d-46ec-baba-15a648a415fe-metrics-client-ca\") pod \"node-exporter-trx7g\" (UID: \"73d422ef-a46d-46ec-baba-15a648a415fe\") " pod="openshift-monitoring/node-exporter-trx7g"
Sep 18 20:05:47.320239 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:47.320205    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/73d422ef-a46d-46ec-baba-15a648a415fe-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-trx7g\" (UID: \"73d422ef-a46d-46ec-baba-15a648a415fe\") " pod="openshift-monitoring/node-exporter-trx7g"
Sep 18 20:05:47.327400 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:47.327370    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sbrcp\" (UniqueName: \"kubernetes.io/projected/73d422ef-a46d-46ec-baba-15a648a415fe-kube-api-access-sbrcp\") pod \"node-exporter-trx7g\" (UID: \"73d422ef-a46d-46ec-baba-15a648a415fe\") " pod="openshift-monitoring/node-exporter-trx7g"
Sep 18 20:05:47.380048 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:47.380012    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:05:47.480101 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:47.480057    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:05:47.482393 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:47.482366    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-trx7g"
Sep 18 20:05:47.494593 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:05:47.494565    2532 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod73d422ef_a46d_46ec_baba_15a648a415fe.slice/crio-44152644c865876110075d66ad6c1995dacc5226c2a513373d3e2d2f13ea4806 WatchSource:0}: Error finding container 44152644c865876110075d66ad6c1995dacc5226c2a513373d3e2d2f13ea4806: Status 404 returned error can't find the container with id 44152644c865876110075d66ad6c1995dacc5226c2a513373d3e2d2f13ea4806
Sep 18 20:05:47.579992 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:47.579959    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:05:47.680643 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:47.680613    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:05:48.104750 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:48.104669    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:05:48.193803 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:48.193774    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:05:48.291762 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:48.291692    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-trx7g" event={"ID":"73d422ef-a46d-46ec-baba-15a648a415fe","Type":"ContainerStarted","Data":"44152644c865876110075d66ad6c1995dacc5226c2a513373d3e2d2f13ea4806"}
Sep 18 20:05:48.304445 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:48.304359    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:05:49.492959 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:49.492873    2532 patch_prober.go:28] interesting pod/image-registry-64795c474b-cmxpf container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Sep 18 20:05:49.492959 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:49.492929    2532 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-64795c474b-cmxpf" podUID="b09e90e3-0343-40d8-af09-03c0cdcfaf36" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 18 20:05:49.838613 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:49.838581    2532 patch_prober.go:28] interesting pod/image-registry-5546d96886-g8mqv container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Sep 18 20:05:49.838790 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:49.838641    2532 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-5546d96886-g8mqv" podUID="20b90eb2-bbca-49d7-b2f3-c8be70bc619e" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 18 20:05:49.899342 ip-10-0-130-81 kubenswrapper[2532]: E0918 20:05:49.899304    2532 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-l8bvt" podUID="13532b96-fcd8-4f12-a4df-b4041abe4cf3"
Sep 18 20:05:50.075828 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:50.075781    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:05:50.171348 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:50.171266    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:05:50.237600 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:50.237556    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:05:50.309820 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:50.309777    2532 generic.go:356] "Generic (PLEG): container finished" podID="73d422ef-a46d-46ec-baba-15a648a415fe" containerID="18ceb2dc6e76401af35c96e85160f4a577462f15f97188435559a350f7080080" exitCode=0
Sep 18 20:05:50.310006 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:50.309919    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-l8bvt"
Sep 18 20:05:50.312182 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:50.312151    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-trx7g" event={"ID":"73d422ef-a46d-46ec-baba-15a648a415fe","Type":"ContainerDied","Data":"18ceb2dc6e76401af35c96e85160f4a577462f15f97188435559a350f7080080"}
Sep 18 20:05:50.523118 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:50.523087    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:05:50.622756 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:50.622670    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:05:50.720585 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:50.720550    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:05:51.219674 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:51.219632    2532 patch_prober.go:28] interesting pod/image-registry-5546d96886-g8mqv container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Sep 18 20:05:51.219935 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:51.219692    2532 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-5546d96886-g8mqv" podUID="20b90eb2-bbca-49d7-b2f3-c8be70bc619e" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 18 20:05:51.316899 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:51.316859    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-trx7g" event={"ID":"73d422ef-a46d-46ec-baba-15a648a415fe","Type":"ContainerStarted","Data":"b0ed5d6135e94278e793b448e83fcf4d84bbcc12c6debc1a0f166729f15483db"}
Sep 18 20:05:51.317060 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:51.316923    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-trx7g" event={"ID":"73d422ef-a46d-46ec-baba-15a648a415fe","Type":"ContainerStarted","Data":"24213712bc40e16664d5762638a107f612302a51f492681d384485293da37828"}
Sep 18 20:05:51.340115 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:51.340058    2532 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-trx7g" podStartSLOduration=2.530320809 podStartE2EDuration="4.340036823s" totalImagesPullingTime="1.724888321s" totalInitContainerRuntime="84.827693ms" isStatefulPod=true podCreationTimestamp="2026-09-18 20:05:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 20:05:51.338530535 +0000 UTC m=+283.103919934" watchObservedRunningTime="2026-09-18 20:05:51.340036823 +0000 UTC m=+283.105426223"
Sep 18 20:05:51.435108 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:51.435066    2532 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-647c4f6b67-8bh9b"]
Sep 18 20:05:51.456305 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:51.456271    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-647c4f6b67-8bh9b"]
Sep 18 20:05:51.456680 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:51.456616    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-647c4f6b67-8bh9b"
Sep 18 20:05:51.459883 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:51.459854    2532 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-epv0ned821rbf\""
Sep 18 20:05:51.460426 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:51.460406    2532 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\""
Sep 18 20:05:51.460906 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:51.460892    2532 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\""
Sep 18 20:05:51.461265 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:51.461238    2532 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Sep 18 20:05:51.461625 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:51.461610    2532 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\""
Sep 18 20:05:51.461976 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:51.461949    2532 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-hxg9l\""
Sep 18 20:05:51.554759 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:51.554670    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/83993929-c843-41ee-825b-7362813e6a1d-metrics-server-audit-profiles\") pod \"metrics-server-647c4f6b67-8bh9b\" (UID: \"83993929-c843-41ee-825b-7362813e6a1d\") " pod="openshift-monitoring/metrics-server-647c4f6b67-8bh9b"
Sep 18 20:05:51.554759 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:51.554732    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/83993929-c843-41ee-825b-7362813e6a1d-audit-log\") pod \"metrics-server-647c4f6b67-8bh9b\" (UID: \"83993929-c843-41ee-825b-7362813e6a1d\") " pod="openshift-monitoring/metrics-server-647c4f6b67-8bh9b"
Sep 18 20:05:51.555248 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:51.554789    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/83993929-c843-41ee-825b-7362813e6a1d-client-ca-bundle\") pod \"metrics-server-647c4f6b67-8bh9b\" (UID: \"83993929-c843-41ee-825b-7362813e6a1d\") " pod="openshift-monitoring/metrics-server-647c4f6b67-8bh9b"
Sep 18 20:05:51.555248 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:51.554822    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/83993929-c843-41ee-825b-7362813e6a1d-secret-metrics-server-client-certs\") pod \"metrics-server-647c4f6b67-8bh9b\" (UID: \"83993929-c843-41ee-825b-7362813e6a1d\") " pod="openshift-monitoring/metrics-server-647c4f6b67-8bh9b"
Sep 18 20:05:51.555248 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:51.554880    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/83993929-c843-41ee-825b-7362813e6a1d-secret-metrics-server-tls\") pod \"metrics-server-647c4f6b67-8bh9b\" (UID: \"83993929-c843-41ee-825b-7362813e6a1d\") " pod="openshift-monitoring/metrics-server-647c4f6b67-8bh9b"
Sep 18 20:05:51.555248 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:51.554903    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/83993929-c843-41ee-825b-7362813e6a1d-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-647c4f6b67-8bh9b\" (UID: \"83993929-c843-41ee-825b-7362813e6a1d\") " pod="openshift-monitoring/metrics-server-647c4f6b67-8bh9b"
Sep 18 20:05:51.555248 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:51.554936    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fvxll\" (UniqueName: \"kubernetes.io/projected/83993929-c843-41ee-825b-7362813e6a1d-kube-api-access-fvxll\") pod \"metrics-server-647c4f6b67-8bh9b\" (UID: \"83993929-c843-41ee-825b-7362813e6a1d\") " pod="openshift-monitoring/metrics-server-647c4f6b67-8bh9b"
Sep 18 20:05:51.655733 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:51.655689    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/83993929-c843-41ee-825b-7362813e6a1d-secret-metrics-server-tls\") pod \"metrics-server-647c4f6b67-8bh9b\" (UID: \"83993929-c843-41ee-825b-7362813e6a1d\") " pod="openshift-monitoring/metrics-server-647c4f6b67-8bh9b"
Sep 18 20:05:51.655733 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:51.655735    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/83993929-c843-41ee-825b-7362813e6a1d-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-647c4f6b67-8bh9b\" (UID: \"83993929-c843-41ee-825b-7362813e6a1d\") " pod="openshift-monitoring/metrics-server-647c4f6b67-8bh9b"
Sep 18 20:05:51.656035 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:51.656009    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fvxll\" (UniqueName: \"kubernetes.io/projected/83993929-c843-41ee-825b-7362813e6a1d-kube-api-access-fvxll\") pod \"metrics-server-647c4f6b67-8bh9b\" (UID: \"83993929-c843-41ee-825b-7362813e6a1d\") " pod="openshift-monitoring/metrics-server-647c4f6b67-8bh9b"
Sep 18 20:05:51.656118 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:51.656080    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/83993929-c843-41ee-825b-7362813e6a1d-metrics-server-audit-profiles\") pod \"metrics-server-647c4f6b67-8bh9b\" (UID: \"83993929-c843-41ee-825b-7362813e6a1d\") " pod="openshift-monitoring/metrics-server-647c4f6b67-8bh9b"
Sep 18 20:05:51.656177 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:51.656129    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/83993929-c843-41ee-825b-7362813e6a1d-audit-log\") pod \"metrics-server-647c4f6b67-8bh9b\" (UID: \"83993929-c843-41ee-825b-7362813e6a1d\") " pod="openshift-monitoring/metrics-server-647c4f6b67-8bh9b"
Sep 18 20:05:51.656236 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:51.656180    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/83993929-c843-41ee-825b-7362813e6a1d-client-ca-bundle\") pod \"metrics-server-647c4f6b67-8bh9b\" (UID: \"83993929-c843-41ee-825b-7362813e6a1d\") " pod="openshift-monitoring/metrics-server-647c4f6b67-8bh9b"
Sep 18 20:05:51.656236 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:51.656220    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/83993929-c843-41ee-825b-7362813e6a1d-secret-metrics-server-client-certs\") pod \"metrics-server-647c4f6b67-8bh9b\" (UID: \"83993929-c843-41ee-825b-7362813e6a1d\") " pod="openshift-monitoring/metrics-server-647c4f6b67-8bh9b"
Sep 18 20:05:51.656650 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:51.656623    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/83993929-c843-41ee-825b-7362813e6a1d-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-647c4f6b67-8bh9b\" (UID: \"83993929-c843-41ee-825b-7362813e6a1d\") " pod="openshift-monitoring/metrics-server-647c4f6b67-8bh9b"
Sep 18 20:05:51.656942 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:51.656911    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/83993929-c843-41ee-825b-7362813e6a1d-audit-log\") pod \"metrics-server-647c4f6b67-8bh9b\" (UID: \"83993929-c843-41ee-825b-7362813e6a1d\") " pod="openshift-monitoring/metrics-server-647c4f6b67-8bh9b"
Sep 18 20:05:51.657144 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:51.657119    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/83993929-c843-41ee-825b-7362813e6a1d-metrics-server-audit-profiles\") pod \"metrics-server-647c4f6b67-8bh9b\" (UID: \"83993929-c843-41ee-825b-7362813e6a1d\") " pod="openshift-monitoring/metrics-server-647c4f6b67-8bh9b"
Sep 18 20:05:51.658915 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:51.658892    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/83993929-c843-41ee-825b-7362813e6a1d-secret-metrics-server-tls\") pod \"metrics-server-647c4f6b67-8bh9b\" (UID: \"83993929-c843-41ee-825b-7362813e6a1d\") " pod="openshift-monitoring/metrics-server-647c4f6b67-8bh9b"
Sep 18 20:05:51.659128 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:51.659108    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/83993929-c843-41ee-825b-7362813e6a1d-secret-metrics-server-client-certs\") pod \"metrics-server-647c4f6b67-8bh9b\" (UID: \"83993929-c843-41ee-825b-7362813e6a1d\") " pod="openshift-monitoring/metrics-server-647c4f6b67-8bh9b"
Sep 18 20:05:51.659334 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:51.659308    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/83993929-c843-41ee-825b-7362813e6a1d-client-ca-bundle\") pod \"metrics-server-647c4f6b67-8bh9b\" (UID: \"83993929-c843-41ee-825b-7362813e6a1d\") " pod="openshift-monitoring/metrics-server-647c4f6b67-8bh9b"
Sep 18 20:05:51.664584 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:51.664564    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fvxll\" (UniqueName: \"kubernetes.io/projected/83993929-c843-41ee-825b-7362813e6a1d-kube-api-access-fvxll\") pod \"metrics-server-647c4f6b67-8bh9b\" (UID: \"83993929-c843-41ee-825b-7362813e6a1d\") " pod="openshift-monitoring/metrics-server-647c4f6b67-8bh9b"
Sep 18 20:05:51.773077 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:51.773036    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-647c4f6b67-8bh9b"
Sep 18 20:05:51.915658 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:51.915625    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:05:52.015053 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:52.015007    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:05:52.076234 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:52.076068    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-647c4f6b67-8bh9b"]
Sep 18 20:05:52.079068 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:05:52.079036    2532 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod83993929_c843_41ee_825b_7362813e6a1d.slice/crio-669a01185ee66b1a74a5c6c740502be4ed14a00ecb9bfa320e1031123b865d4c WatchSource:0}: Error finding container 669a01185ee66b1a74a5c6c740502be4ed14a00ecb9bfa320e1031123b865d4c: Status 404 returned error can't find the container with id 669a01185ee66b1a74a5c6c740502be4ed14a00ecb9bfa320e1031123b865d4c
Sep 18 20:05:52.115851 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:52.115765    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:05:52.322707 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:52.322672    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-647c4f6b67-8bh9b" event={"ID":"83993929-c843-41ee-825b-7362813e6a1d","Type":"ContainerStarted","Data":"669a01185ee66b1a74a5c6c740502be4ed14a00ecb9bfa320e1031123b865d4c"}
Sep 18 20:05:52.373419 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:52.373335    2532 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-fff57f85f-8qw8c"]
Sep 18 20:05:52.402255 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:52.402219    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-fff57f85f-8qw8c"]
Sep 18 20:05:52.402435 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:52.402359    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-fff57f85f-8qw8c"
Sep 18 20:05:52.405135 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:52.405108    2532 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\""
Sep 18 20:05:52.405287 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:52.405263    2532 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-8rqdz\""
Sep 18 20:05:52.405372 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:52.405352    2532 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\""
Sep 18 20:05:52.405372 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:52.405372    2532 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\""
Sep 18 20:05:52.405562 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:52.405413    2532 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\""
Sep 18 20:05:52.405562 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:52.405470    2532 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\""
Sep 18 20:05:52.410184 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:52.410153    2532 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\""
Sep 18 20:05:52.464201 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:52.464163    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/fd54629b-72af-4c18-b7e1-1fcda1e4e6d4-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-fff57f85f-8qw8c\" (UID: \"fd54629b-72af-4c18-b7e1-1fcda1e4e6d4\") " pod="openshift-monitoring/telemeter-client-fff57f85f-8qw8c"
Sep 18 20:05:52.464452 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:52.464431    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fd54629b-72af-4c18-b7e1-1fcda1e4e6d4-serving-certs-ca-bundle\") pod \"telemeter-client-fff57f85f-8qw8c\" (UID: \"fd54629b-72af-4c18-b7e1-1fcda1e4e6d4\") " pod="openshift-monitoring/telemeter-client-fff57f85f-8qw8c"
Sep 18 20:05:52.464695 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:52.464679    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/fd54629b-72af-4c18-b7e1-1fcda1e4e6d4-secret-telemeter-client\") pod \"telemeter-client-fff57f85f-8qw8c\" (UID: \"fd54629b-72af-4c18-b7e1-1fcda1e4e6d4\") " pod="openshift-monitoring/telemeter-client-fff57f85f-8qw8c"
Sep 18 20:05:52.464893 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:52.464878    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5mbvc\" (UniqueName: \"kubernetes.io/projected/fd54629b-72af-4c18-b7e1-1fcda1e4e6d4-kube-api-access-5mbvc\") pod \"telemeter-client-fff57f85f-8qw8c\" (UID: \"fd54629b-72af-4c18-b7e1-1fcda1e4e6d4\") " pod="openshift-monitoring/telemeter-client-fff57f85f-8qw8c"
Sep 18 20:05:52.465000 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:52.464987    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/fd54629b-72af-4c18-b7e1-1fcda1e4e6d4-metrics-client-ca\") pod \"telemeter-client-fff57f85f-8qw8c\" (UID: \"fd54629b-72af-4c18-b7e1-1fcda1e4e6d4\") " pod="openshift-monitoring/telemeter-client-fff57f85f-8qw8c"
Sep 18 20:05:52.465104 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:52.465091    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/fd54629b-72af-4c18-b7e1-1fcda1e4e6d4-telemeter-client-tls\") pod \"telemeter-client-fff57f85f-8qw8c\" (UID: \"fd54629b-72af-4c18-b7e1-1fcda1e4e6d4\") " pod="openshift-monitoring/telemeter-client-fff57f85f-8qw8c"
Sep 18 20:05:52.465210 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:52.465198    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/fd54629b-72af-4c18-b7e1-1fcda1e4e6d4-federate-client-tls\") pod \"telemeter-client-fff57f85f-8qw8c\" (UID: \"fd54629b-72af-4c18-b7e1-1fcda1e4e6d4\") " pod="openshift-monitoring/telemeter-client-fff57f85f-8qw8c"
Sep 18 20:05:52.465350 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:52.465335    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fd54629b-72af-4c18-b7e1-1fcda1e4e6d4-telemeter-trusted-ca-bundle\") pod \"telemeter-client-fff57f85f-8qw8c\" (UID: \"fd54629b-72af-4c18-b7e1-1fcda1e4e6d4\") " pod="openshift-monitoring/telemeter-client-fff57f85f-8qw8c"
Sep 18 20:05:52.566729 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:52.566657    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/fd54629b-72af-4c18-b7e1-1fcda1e4e6d4-telemeter-client-tls\") pod \"telemeter-client-fff57f85f-8qw8c\" (UID: \"fd54629b-72af-4c18-b7e1-1fcda1e4e6d4\") " pod="openshift-monitoring/telemeter-client-fff57f85f-8qw8c"
Sep 18 20:05:52.567201 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:52.566744    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/fd54629b-72af-4c18-b7e1-1fcda1e4e6d4-federate-client-tls\") pod \"telemeter-client-fff57f85f-8qw8c\" (UID: \"fd54629b-72af-4c18-b7e1-1fcda1e4e6d4\") " pod="openshift-monitoring/telemeter-client-fff57f85f-8qw8c"
Sep 18 20:05:52.567201 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:52.566795    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fd54629b-72af-4c18-b7e1-1fcda1e4e6d4-telemeter-trusted-ca-bundle\") pod \"telemeter-client-fff57f85f-8qw8c\" (UID: \"fd54629b-72af-4c18-b7e1-1fcda1e4e6d4\") " pod="openshift-monitoring/telemeter-client-fff57f85f-8qw8c"
Sep 18 20:05:52.567201 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:52.566835    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/fd54629b-72af-4c18-b7e1-1fcda1e4e6d4-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-fff57f85f-8qw8c\" (UID: \"fd54629b-72af-4c18-b7e1-1fcda1e4e6d4\") " pod="openshift-monitoring/telemeter-client-fff57f85f-8qw8c"
Sep 18 20:05:52.567201 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:52.566862    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fd54629b-72af-4c18-b7e1-1fcda1e4e6d4-serving-certs-ca-bundle\") pod \"telemeter-client-fff57f85f-8qw8c\" (UID: \"fd54629b-72af-4c18-b7e1-1fcda1e4e6d4\") " pod="openshift-monitoring/telemeter-client-fff57f85f-8qw8c"
Sep 18 20:05:52.567201 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:52.566931    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/fd54629b-72af-4c18-b7e1-1fcda1e4e6d4-secret-telemeter-client\") pod \"telemeter-client-fff57f85f-8qw8c\" (UID: \"fd54629b-72af-4c18-b7e1-1fcda1e4e6d4\") " pod="openshift-monitoring/telemeter-client-fff57f85f-8qw8c"
Sep 18 20:05:52.567201 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:52.567004    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5mbvc\" (UniqueName: \"kubernetes.io/projected/fd54629b-72af-4c18-b7e1-1fcda1e4e6d4-kube-api-access-5mbvc\") pod \"telemeter-client-fff57f85f-8qw8c\" (UID: \"fd54629b-72af-4c18-b7e1-1fcda1e4e6d4\") " pod="openshift-monitoring/telemeter-client-fff57f85f-8qw8c"
Sep 18 20:05:52.567201 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:52.567041    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/fd54629b-72af-4c18-b7e1-1fcda1e4e6d4-metrics-client-ca\") pod \"telemeter-client-fff57f85f-8qw8c\" (UID: \"fd54629b-72af-4c18-b7e1-1fcda1e4e6d4\") " pod="openshift-monitoring/telemeter-client-fff57f85f-8qw8c"
Sep 18 20:05:52.567879 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:52.567848    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/fd54629b-72af-4c18-b7e1-1fcda1e4e6d4-metrics-client-ca\") pod \"telemeter-client-fff57f85f-8qw8c\" (UID: \"fd54629b-72af-4c18-b7e1-1fcda1e4e6d4\") " pod="openshift-monitoring/telemeter-client-fff57f85f-8qw8c"
Sep 18 20:05:52.567989 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:52.567939    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fd54629b-72af-4c18-b7e1-1fcda1e4e6d4-serving-certs-ca-bundle\") pod \"telemeter-client-fff57f85f-8qw8c\" (UID: \"fd54629b-72af-4c18-b7e1-1fcda1e4e6d4\") " pod="openshift-monitoring/telemeter-client-fff57f85f-8qw8c"
Sep 18 20:05:52.568228 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:52.568199    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fd54629b-72af-4c18-b7e1-1fcda1e4e6d4-telemeter-trusted-ca-bundle\") pod \"telemeter-client-fff57f85f-8qw8c\" (UID: \"fd54629b-72af-4c18-b7e1-1fcda1e4e6d4\") " pod="openshift-monitoring/telemeter-client-fff57f85f-8qw8c"
Sep 18 20:05:52.569835 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:52.569807    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/fd54629b-72af-4c18-b7e1-1fcda1e4e6d4-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-fff57f85f-8qw8c\" (UID: \"fd54629b-72af-4c18-b7e1-1fcda1e4e6d4\") " pod="openshift-monitoring/telemeter-client-fff57f85f-8qw8c"
Sep 18 20:05:52.570006 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:52.569985    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/fd54629b-72af-4c18-b7e1-1fcda1e4e6d4-federate-client-tls\") pod \"telemeter-client-fff57f85f-8qw8c\" (UID: \"fd54629b-72af-4c18-b7e1-1fcda1e4e6d4\") " pod="openshift-monitoring/telemeter-client-fff57f85f-8qw8c"
Sep 18 20:05:52.570102 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:52.570078    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/fd54629b-72af-4c18-b7e1-1fcda1e4e6d4-telemeter-client-tls\") pod \"telemeter-client-fff57f85f-8qw8c\" (UID: \"fd54629b-72af-4c18-b7e1-1fcda1e4e6d4\") " pod="openshift-monitoring/telemeter-client-fff57f85f-8qw8c"
Sep 18 20:05:52.570267 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:52.570246    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/fd54629b-72af-4c18-b7e1-1fcda1e4e6d4-secret-telemeter-client\") pod \"telemeter-client-fff57f85f-8qw8c\" (UID: \"fd54629b-72af-4c18-b7e1-1fcda1e4e6d4\") " pod="openshift-monitoring/telemeter-client-fff57f85f-8qw8c"
Sep 18 20:05:52.575974 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:52.575954    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5mbvc\" (UniqueName: \"kubernetes.io/projected/fd54629b-72af-4c18-b7e1-1fcda1e4e6d4-kube-api-access-5mbvc\") pod \"telemeter-client-fff57f85f-8qw8c\" (UID: \"fd54629b-72af-4c18-b7e1-1fcda1e4e6d4\") " pod="openshift-monitoring/telemeter-client-fff57f85f-8qw8c"
Sep 18 20:05:52.714590 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:52.714535    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-fff57f85f-8qw8c"
Sep 18 20:05:53.025666 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:53.025430    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-fff57f85f-8qw8c"]
Sep 18 20:05:53.328293 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:53.328210    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-fff57f85f-8qw8c" event={"ID":"fd54629b-72af-4c18-b7e1-1fcda1e4e6d4","Type":"ContainerStarted","Data":"3c0b3bb1722ad55d5231e604a6f7c5d74fb9b4995d8d05b40f58af69f2e29a4e"}
Sep 18 20:05:53.342206 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:53.342162    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:05:53.627430 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:53.627350    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:05:53.683319 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:53.683277    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:05:53.779101 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:53.779066    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/13532b96-fcd8-4f12-a4df-b4041abe4cf3-metrics-tls\") pod \"dns-default-l8bvt\" (UID: \"13532b96-fcd8-4f12-a4df-b4041abe4cf3\") " pod="openshift-dns/dns-default-l8bvt"
Sep 18 20:05:53.781693 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:53.781664    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/13532b96-fcd8-4f12-a4df-b4041abe4cf3-metrics-tls\") pod \"dns-default-l8bvt\" (UID: \"13532b96-fcd8-4f12-a4df-b4041abe4cf3\") " pod="openshift-dns/dns-default-l8bvt"
Sep 18 20:05:53.913201 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:53.913129    2532 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-dw2ht\""
Sep 18 20:05:53.921218 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:53.921190    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-l8bvt"
Sep 18 20:05:54.229234 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:54.229199    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-l8bvt"]
Sep 18 20:05:54.259645 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:05:54.259606    2532 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod13532b96_fcd8_4f12_a4df_b4041abe4cf3.slice/crio-b7ceed3abc3b1424ddbbac1000f109f10f4e4c31e8bc08b3f9c822ad36bb3111 WatchSource:0}: Error finding container b7ceed3abc3b1424ddbbac1000f109f10f4e4c31e8bc08b3f9c822ad36bb3111: Status 404 returned error can't find the container with id b7ceed3abc3b1424ddbbac1000f109f10f4e4c31e8bc08b3f9c822ad36bb3111
Sep 18 20:05:54.334851 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:54.334813    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-l8bvt" event={"ID":"13532b96-fcd8-4f12-a4df-b4041abe4cf3","Type":"ContainerStarted","Data":"b7ceed3abc3b1424ddbbac1000f109f10f4e4c31e8bc08b3f9c822ad36bb3111"}
Sep 18 20:05:54.508780 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:54.508675    2532 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-64795c474b-cmxpf" podUID="b09e90e3-0343-40d8-af09-03c0cdcfaf36" containerName="registry" containerID="cri-o://2c8253355e6fff30052637655e4509c52a9417482433a6be0db8bfc71e7fe503" gracePeriod=30
Sep 18 20:05:54.668523 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:54.668440    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:05:54.769858 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:54.769828    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:05:54.792903 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:54.792877    2532 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-64795c474b-cmxpf"
Sep 18 20:05:54.870430 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:54.870330    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:05:54.893186 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:54.890525    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/b09e90e3-0343-40d8-af09-03c0cdcfaf36-image-registry-private-configuration\") pod \"b09e90e3-0343-40d8-af09-03c0cdcfaf36\" (UID: \"b09e90e3-0343-40d8-af09-03c0cdcfaf36\") "
Sep 18 20:05:54.893186 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:54.890586    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nqb9w\" (UniqueName: \"kubernetes.io/projected/b09e90e3-0343-40d8-af09-03c0cdcfaf36-kube-api-access-nqb9w\") pod \"b09e90e3-0343-40d8-af09-03c0cdcfaf36\" (UID: \"b09e90e3-0343-40d8-af09-03c0cdcfaf36\") "
Sep 18 20:05:54.893186 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:54.890626    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/b09e90e3-0343-40d8-af09-03c0cdcfaf36-registry-certificates\") pod \"b09e90e3-0343-40d8-af09-03c0cdcfaf36\" (UID: \"b09e90e3-0343-40d8-af09-03c0cdcfaf36\") "
Sep 18 20:05:54.893186 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:54.890664    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/b09e90e3-0343-40d8-af09-03c0cdcfaf36-installation-pull-secrets\") pod \"b09e90e3-0343-40d8-af09-03c0cdcfaf36\" (UID: \"b09e90e3-0343-40d8-af09-03c0cdcfaf36\") "
Sep 18 20:05:54.893186 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:54.890700    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/b09e90e3-0343-40d8-af09-03c0cdcfaf36-ca-trust-extracted\") pod \"b09e90e3-0343-40d8-af09-03c0cdcfaf36\" (UID: \"b09e90e3-0343-40d8-af09-03c0cdcfaf36\") "
Sep 18 20:05:54.893186 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:54.890742    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/b09e90e3-0343-40d8-af09-03c0cdcfaf36-trusted-ca\") pod \"b09e90e3-0343-40d8-af09-03c0cdcfaf36\" (UID: \"b09e90e3-0343-40d8-af09-03c0cdcfaf36\") "
Sep 18 20:05:54.893186 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:54.890774    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/b09e90e3-0343-40d8-af09-03c0cdcfaf36-bound-sa-token\") pod \"b09e90e3-0343-40d8-af09-03c0cdcfaf36\" (UID: \"b09e90e3-0343-40d8-af09-03c0cdcfaf36\") "
Sep 18 20:05:54.893186 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:54.890853    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/b09e90e3-0343-40d8-af09-03c0cdcfaf36-registry-tls\") pod \"b09e90e3-0343-40d8-af09-03c0cdcfaf36\" (UID: \"b09e90e3-0343-40d8-af09-03c0cdcfaf36\") "
Sep 18 20:05:54.893186 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:54.891224    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b09e90e3-0343-40d8-af09-03c0cdcfaf36-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "b09e90e3-0343-40d8-af09-03c0cdcfaf36" (UID: "b09e90e3-0343-40d8-af09-03c0cdcfaf36"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 20:05:54.893789 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:54.893303    2532 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/b09e90e3-0343-40d8-af09-03c0cdcfaf36-registry-certificates\") on node \"ip-10-0-130-81.ec2.internal\" DevicePath \"\""
Sep 18 20:05:54.897115 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:54.894621    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b09e90e3-0343-40d8-af09-03c0cdcfaf36-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "b09e90e3-0343-40d8-af09-03c0cdcfaf36" (UID: "b09e90e3-0343-40d8-af09-03c0cdcfaf36"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 20:05:54.897115 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:54.895277    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b09e90e3-0343-40d8-af09-03c0cdcfaf36-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "b09e90e3-0343-40d8-af09-03c0cdcfaf36" (UID: "b09e90e3-0343-40d8-af09-03c0cdcfaf36"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 20:05:54.897115 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:54.895820    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b09e90e3-0343-40d8-af09-03c0cdcfaf36-kube-api-access-nqb9w" (OuterVolumeSpecName: "kube-api-access-nqb9w") pod "b09e90e3-0343-40d8-af09-03c0cdcfaf36" (UID: "b09e90e3-0343-40d8-af09-03c0cdcfaf36"). InnerVolumeSpecName "kube-api-access-nqb9w". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 20:05:54.900161 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:54.900128    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b09e90e3-0343-40d8-af09-03c0cdcfaf36-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "b09e90e3-0343-40d8-af09-03c0cdcfaf36" (UID: "b09e90e3-0343-40d8-af09-03c0cdcfaf36"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 20:05:54.900378 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:54.900354    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b09e90e3-0343-40d8-af09-03c0cdcfaf36-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "b09e90e3-0343-40d8-af09-03c0cdcfaf36" (UID: "b09e90e3-0343-40d8-af09-03c0cdcfaf36"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 20:05:54.902090 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:54.902052    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b09e90e3-0343-40d8-af09-03c0cdcfaf36-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "b09e90e3-0343-40d8-af09-03c0cdcfaf36" (UID: "b09e90e3-0343-40d8-af09-03c0cdcfaf36"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 20:05:54.904025 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:54.903999    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b09e90e3-0343-40d8-af09-03c0cdcfaf36-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "b09e90e3-0343-40d8-af09-03c0cdcfaf36" (UID: "b09e90e3-0343-40d8-af09-03c0cdcfaf36"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 20:05:54.994063 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:54.994027    2532 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/b09e90e3-0343-40d8-af09-03c0cdcfaf36-installation-pull-secrets\") on node \"ip-10-0-130-81.ec2.internal\" DevicePath \"\""
Sep 18 20:05:54.994063 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:54.994059    2532 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/b09e90e3-0343-40d8-af09-03c0cdcfaf36-ca-trust-extracted\") on node \"ip-10-0-130-81.ec2.internal\" DevicePath \"\""
Sep 18 20:05:54.994297 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:54.994072    2532 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/b09e90e3-0343-40d8-af09-03c0cdcfaf36-trusted-ca\") on node \"ip-10-0-130-81.ec2.internal\" DevicePath \"\""
Sep 18 20:05:54.994297 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:54.994084    2532 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/b09e90e3-0343-40d8-af09-03c0cdcfaf36-bound-sa-token\") on node \"ip-10-0-130-81.ec2.internal\" DevicePath \"\""
Sep 18 20:05:54.994297 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:54.994095    2532 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/b09e90e3-0343-40d8-af09-03c0cdcfaf36-registry-tls\") on node \"ip-10-0-130-81.ec2.internal\" DevicePath \"\""
Sep 18 20:05:54.994297 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:54.994106    2532 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/b09e90e3-0343-40d8-af09-03c0cdcfaf36-image-registry-private-configuration\") on node \"ip-10-0-130-81.ec2.internal\" DevicePath \"\""
Sep 18 20:05:54.994297 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:54.994118    2532 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nqb9w\" (UniqueName: \"kubernetes.io/projected/b09e90e3-0343-40d8-af09-03c0cdcfaf36-kube-api-access-nqb9w\") on node \"ip-10-0-130-81.ec2.internal\" DevicePath \"\""
Sep 18 20:05:55.341033 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:55.340995    2532 generic.go:356] "Generic (PLEG): container finished" podID="b09e90e3-0343-40d8-af09-03c0cdcfaf36" containerID="2c8253355e6fff30052637655e4509c52a9417482433a6be0db8bfc71e7fe503" exitCode=0
Sep 18 20:05:55.341226 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:55.341060    2532 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-64795c474b-cmxpf"
Sep 18 20:05:55.341226 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:55.341085    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-64795c474b-cmxpf" event={"ID":"b09e90e3-0343-40d8-af09-03c0cdcfaf36","Type":"ContainerDied","Data":"2c8253355e6fff30052637655e4509c52a9417482433a6be0db8bfc71e7fe503"}
Sep 18 20:05:55.341226 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:55.341145    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-64795c474b-cmxpf" event={"ID":"b09e90e3-0343-40d8-af09-03c0cdcfaf36","Type":"ContainerDied","Data":"98743b4ed8fc615d18c37bcdc4b80738f561b807af4b10d578b4545d6c2f3d4d"}
Sep 18 20:05:55.341226 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:55.341174    2532 scope.go:117] "RemoveContainer" containerID="2c8253355e6fff30052637655e4509c52a9417482433a6be0db8bfc71e7fe503"
Sep 18 20:05:55.342898 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:55.342826    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-647c4f6b67-8bh9b" event={"ID":"83993929-c843-41ee-825b-7362813e6a1d","Type":"ContainerStarted","Data":"8d3a34bfc2b0e2e2d744f396b52c8a9abf9159bbe3873909c459603df6eec3fe"}
Sep 18 20:05:55.352721 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:55.352701    2532 scope.go:117] "RemoveContainer" containerID="2c8253355e6fff30052637655e4509c52a9417482433a6be0db8bfc71e7fe503"
Sep 18 20:05:55.353120 ip-10-0-130-81 kubenswrapper[2532]: E0918 20:05:55.353095    2532 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2c8253355e6fff30052637655e4509c52a9417482433a6be0db8bfc71e7fe503\": container with ID starting with 2c8253355e6fff30052637655e4509c52a9417482433a6be0db8bfc71e7fe503 not found: ID does not exist" containerID="2c8253355e6fff30052637655e4509c52a9417482433a6be0db8bfc71e7fe503"
Sep 18 20:05:55.353199 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:55.353130    2532 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2c8253355e6fff30052637655e4509c52a9417482433a6be0db8bfc71e7fe503"} err="failed to get container status \"2c8253355e6fff30052637655e4509c52a9417482433a6be0db8bfc71e7fe503\": rpc error: code = NotFound desc = could not find container \"2c8253355e6fff30052637655e4509c52a9417482433a6be0db8bfc71e7fe503\": container with ID starting with 2c8253355e6fff30052637655e4509c52a9417482433a6be0db8bfc71e7fe503 not found: ID does not exist"
Sep 18 20:05:55.361875 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:55.361758    2532 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-647c4f6b67-8bh9b" podStartSLOduration=2.150052288 podStartE2EDuration="4.361741741s" totalImagesPullingTime="2.211689453s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 20:05:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 20:05:55.361666223 +0000 UTC m=+287.127055623" watchObservedRunningTime="2026-09-18 20:05:55.361741741 +0000 UTC m=+287.127131143"
Sep 18 20:05:55.374648 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:55.374621    2532 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-64795c474b-cmxpf"]
Sep 18 20:05:55.380341 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:55.380322    2532 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-64795c474b-cmxpf"]
Sep 18 20:05:56.305579 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:56.305543    2532 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b09e90e3-0343-40d8-af09-03c0cdcfaf36" path="/var/lib/kubelet/pods/b09e90e3-0343-40d8-af09-03c0cdcfaf36/volumes"
Sep 18 20:05:57.353908 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:57.353821    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-fff57f85f-8qw8c" event={"ID":"fd54629b-72af-4c18-b7e1-1fcda1e4e6d4","Type":"ContainerStarted","Data":"a548763f1154d2a48fd70899ae3ff47966e1f830035b21b7370e41bfa5d1356d"}
Sep 18 20:05:57.355634 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:57.355608    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-l8bvt" event={"ID":"13532b96-fcd8-4f12-a4df-b4041abe4cf3","Type":"ContainerStarted","Data":"dfd6b97a3e2d2200ce61a49790a38479d91283d80ff96468bef36fdc6e5eb074"}
Sep 18 20:05:57.355757 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:57.355650    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-l8bvt" event={"ID":"13532b96-fcd8-4f12-a4df-b4041abe4cf3","Type":"ContainerStarted","Data":"538f43d334244bcdcb6bea7a7017e556428e698a803e9e7ae034e7a6864c8ed6"}
Sep 18 20:05:57.355821 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:57.355754    2532 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-l8bvt"
Sep 18 20:05:57.371725 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:57.371678    2532 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-l8bvt" podStartSLOduration=251.858821965 podStartE2EDuration="4m14.371665476s" totalImagesPullingTime="2.512843511s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 20:01:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 20:05:57.370840084 +0000 UTC m=+289.136229484" watchObservedRunningTime="2026-09-18 20:05:57.371665476 +0000 UTC m=+289.137054901"
Sep 18 20:05:59.366220 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:59.366184    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-fff57f85f-8qw8c" event={"ID":"fd54629b-72af-4c18-b7e1-1fcda1e4e6d4","Type":"ContainerStarted","Data":"4fcbf68cd1a77cc3a579cedfa7f0f7ef6c7a99acc1794adf859008c6eff1e991"}
Sep 18 20:05:59.366699 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:59.366235    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-fff57f85f-8qw8c" event={"ID":"fd54629b-72af-4c18-b7e1-1fcda1e4e6d4","Type":"ContainerStarted","Data":"22f6dec443baa5be71ff9db7bd6b747ea78dbed5192020a75e399abd6cc42abc"}
Sep 18 20:05:59.388112 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:59.388068    2532 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-fff57f85f-8qw8c" podStartSLOduration=2.233251506 podStartE2EDuration="7.388052524s" totalImagesPullingTime="5.154801018s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 20:05:52 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 20:05:59.385858082 +0000 UTC m=+291.151247481" watchObservedRunningTime="2026-09-18 20:05:59.388052524 +0000 UTC m=+291.153441922"
Sep 18 20:05:59.837188 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:59.837155    2532 patch_prober.go:28] interesting pod/image-registry-5546d96886-g8mqv container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Sep 18 20:05:59.837353 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:05:59.837215    2532 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-5546d96886-g8mqv" podUID="20b90eb2-bbca-49d7-b2f3-c8be70bc619e" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 18 20:06:01.219107 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:06:01.219077    2532 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-5546d96886-g8mqv"
Sep 18 20:06:01.341096 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:06:01.341059    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:06:02.955901 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:06:02.955861    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:06:03.102009 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:06:03.101977    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:06:03.205072 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:06:03.205036    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:06:03.314242 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:06:03.314160    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:06:03.414685 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:06:03.414658    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:06:04.318066 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:06:04.318031    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:06:04.368255 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:06:04.368230    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:06:07.362510 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:06:07.362466    2532 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-l8bvt"
Sep 18 20:06:07.387408 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:06:07.387376    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:06:07.484413 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:06:07.484377    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:06:10.235581 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:06:10.235554    2532 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-m7wdd_32e33d39-7826-499d-bc01-0487b7580e7a/console-operator/1.log"
Sep 18 20:06:10.235966 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:06:10.235785    2532 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-m7wdd_32e33d39-7826-499d-bc01-0487b7580e7a/console-operator/1.log"
Sep 18 20:06:10.244776 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:06:10.244756    2532 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rtw9n_73fbf16b-3549-4a6c-a959-b633237ebd0c/ovn-acl-logging/0.log"
Sep 18 20:06:10.244895 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:06:10.244866    2532 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rtw9n_73fbf16b-3549-4a6c-a959-b633237ebd0c/ovn-acl-logging/0.log"
Sep 18 20:06:10.247503 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:06:10.247475    2532 kubelet.go:1648] "Image garbage collection succeeded"
Sep 18 20:06:10.371469 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:06:10.371329    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:06:10.476582 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:06:10.476549    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:06:10.578475 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:06:10.578404    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:06:10.679260 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:06:10.679226    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:06:11.773400 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:06:11.773363    2532 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-647c4f6b67-8bh9b"
Sep 18 20:06:11.773820 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:06:11.773682    2532 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-647c4f6b67-8bh9b"
Sep 18 20:06:13.389021 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:06:13.388990    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:06:13.493362 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:06:13.493319    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:06:16.264891 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:06:16.264858    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:06:16.318293 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:06:16.318262    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:06:16.482357 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:06:16.482325    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:06:16.582975 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:06:16.582903    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:06:17.432202 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:06:17.432167    2532 generic.go:356] "Generic (PLEG): container finished" podID="8e4262e9-3e90-4ed5-a3ac-4ea829f4c702" containerID="0a3c364aab1f04804f39e1c53e6458aa42a110f68cc6bde97be32fdc55ff3113" exitCode=0
Sep 18 20:06:17.432202 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:06:17.432205    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-57cdc4b768-gr8p9" event={"ID":"8e4262e9-3e90-4ed5-a3ac-4ea829f4c702","Type":"ContainerDied","Data":"0a3c364aab1f04804f39e1c53e6458aa42a110f68cc6bde97be32fdc55ff3113"}
Sep 18 20:06:17.432739 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:06:17.432583    2532 scope.go:117] "RemoveContainer" containerID="0a3c364aab1f04804f39e1c53e6458aa42a110f68cc6bde97be32fdc55ff3113"
Sep 18 20:06:18.437310 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:06:18.437276    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-57cdc4b768-gr8p9" event={"ID":"8e4262e9-3e90-4ed5-a3ac-4ea829f4c702","Type":"ContainerStarted","Data":"b948deaff5a02f204b863af18fbbcbb8796144e1d0543fd3aea9b9c58829d871"}
Sep 18 20:06:28.367303 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:06:28.367270    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:06:28.437067 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:06:28.437034    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:06:31.264605 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:06:31.264548    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:06:31.364903 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:06:31.364871    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:06:31.779339 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:06:31.779304    2532 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-647c4f6b67-8bh9b"
Sep 18 20:06:31.783081 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:06:31.783056    2532 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-647c4f6b67-8bh9b"
Sep 18 20:07:03.218452 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:07:03.218361    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:07:03.278325 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:07:03.278292    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:07:14.497708 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:07:14.497674    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:07:14.598791 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:07:14.598759    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:07:15.190471 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:07:15.190435    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:07:15.292810 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:07:15.292779    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:07:18.180845 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:07:18.180811    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:07:18.240724 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:07:18.240691    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:07:18.310763 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:07:18.310732    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:07:18.382597 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:07:18.382565    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:07:18.917839 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:07:18.917807    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:07:19.018451 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:07:19.018420    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:07:28.679089 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:07:28.679053    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:07:28.736674 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:07:28.736642    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:07:28.792536 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:07:28.792502    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:07:31.268990 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:07:31.268947    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:07:31.369632 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:07:31.369599    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:07:53.297915 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:07:53.297882    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:07:53.396245 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:07:53.396212    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:07:53.970904 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:07:53.970872    2532 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rqpkw"]
Sep 18 20:07:53.971160 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:07:53.971149    2532 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b09e90e3-0343-40d8-af09-03c0cdcfaf36" containerName="registry"
Sep 18 20:07:53.971204 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:07:53.971163    2532 state_mem.go:107] "Deleted CPUSet assignment" podUID="b09e90e3-0343-40d8-af09-03c0cdcfaf36" containerName="registry"
Sep 18 20:07:53.971237 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:07:53.971218    2532 memory_manager.go:368] "RemoveStaleState removing state" podUID="b09e90e3-0343-40d8-af09-03c0cdcfaf36" containerName="registry"
Sep 18 20:07:53.972924 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:07:53.972907    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rqpkw"
Sep 18 20:07:53.975323 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:07:53.975299    2532 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Sep 18 20:07:53.976322 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:07:53.976303    2532 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Sep 18 20:07:53.976432 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:07:53.976302    2532 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-62r9t\""
Sep 18 20:07:53.982634 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:07:53.982611    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rqpkw"]
Sep 18 20:07:54.017615 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:07:54.017571    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zb7hw\" (UniqueName: \"kubernetes.io/projected/f3c09fc5-3ccb-4ff9-888e-5ff3e50910ff-kube-api-access-zb7hw\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rqpkw\" (UID: \"f3c09fc5-3ccb-4ff9-888e-5ff3e50910ff\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rqpkw"
Sep 18 20:07:54.017793 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:07:54.017648    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f3c09fc5-3ccb-4ff9-888e-5ff3e50910ff-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rqpkw\" (UID: \"f3c09fc5-3ccb-4ff9-888e-5ff3e50910ff\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rqpkw"
Sep 18 20:07:54.017793 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:07:54.017685    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f3c09fc5-3ccb-4ff9-888e-5ff3e50910ff-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rqpkw\" (UID: \"f3c09fc5-3ccb-4ff9-888e-5ff3e50910ff\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rqpkw"
Sep 18 20:07:54.118679 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:07:54.118642    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f3c09fc5-3ccb-4ff9-888e-5ff3e50910ff-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rqpkw\" (UID: \"f3c09fc5-3ccb-4ff9-888e-5ff3e50910ff\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rqpkw"
Sep 18 20:07:54.118861 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:07:54.118687    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f3c09fc5-3ccb-4ff9-888e-5ff3e50910ff-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rqpkw\" (UID: \"f3c09fc5-3ccb-4ff9-888e-5ff3e50910ff\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rqpkw"
Sep 18 20:07:54.118861 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:07:54.118724    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zb7hw\" (UniqueName: \"kubernetes.io/projected/f3c09fc5-3ccb-4ff9-888e-5ff3e50910ff-kube-api-access-zb7hw\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rqpkw\" (UID: \"f3c09fc5-3ccb-4ff9-888e-5ff3e50910ff\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rqpkw"
Sep 18 20:07:54.119119 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:07:54.119095    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f3c09fc5-3ccb-4ff9-888e-5ff3e50910ff-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rqpkw\" (UID: \"f3c09fc5-3ccb-4ff9-888e-5ff3e50910ff\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rqpkw"
Sep 18 20:07:54.119187 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:07:54.119115    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f3c09fc5-3ccb-4ff9-888e-5ff3e50910ff-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rqpkw\" (UID: \"f3c09fc5-3ccb-4ff9-888e-5ff3e50910ff\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rqpkw"
Sep 18 20:07:54.129472 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:07:54.129447    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zb7hw\" (UniqueName: \"kubernetes.io/projected/f3c09fc5-3ccb-4ff9-888e-5ff3e50910ff-kube-api-access-zb7hw\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rqpkw\" (UID: \"f3c09fc5-3ccb-4ff9-888e-5ff3e50910ff\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rqpkw"
Sep 18 20:07:54.282037 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:07:54.281955    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rqpkw"
Sep 18 20:07:54.550092 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:07:54.550067    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rqpkw"]
Sep 18 20:07:54.552586 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:07:54.552552    2532 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf3c09fc5_3ccb_4ff9_888e_5ff3e50910ff.slice/crio-740f0c58b339a4644625f7f4dbb2db1fa3925db95f73f4dd76e971bad5d84aaf WatchSource:0}: Error finding container 740f0c58b339a4644625f7f4dbb2db1fa3925db95f73f4dd76e971bad5d84aaf: Status 404 returned error can't find the container with id 740f0c58b339a4644625f7f4dbb2db1fa3925db95f73f4dd76e971bad5d84aaf
Sep 18 20:07:54.556888 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:07:54.556872    2532 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 18 20:07:54.786525 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:07:54.786477    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rqpkw" event={"ID":"f3c09fc5-3ccb-4ff9-888e-5ff3e50910ff","Type":"ContainerStarted","Data":"740f0c58b339a4644625f7f4dbb2db1fa3925db95f73f4dd76e971bad5d84aaf"}
Sep 18 20:07:57.883872 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:07:57.883835    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:08:01.816967 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:08:01.816925    2532 generic.go:356] "Generic (PLEG): container finished" podID="f3c09fc5-3ccb-4ff9-888e-5ff3e50910ff" containerID="9b609cb471b4c5910ed3830a3370c4af276eea8aa78abcfff5955182e2337f27" exitCode=0
Sep 18 20:08:01.817379 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:08:01.817017    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rqpkw" event={"ID":"f3c09fc5-3ccb-4ff9-888e-5ff3e50910ff","Type":"ContainerDied","Data":"9b609cb471b4c5910ed3830a3370c4af276eea8aa78abcfff5955182e2337f27"}
Sep 18 20:08:03.244216 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:08:03.244180    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:08:04.831738 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:08:04.831706    2532 generic.go:356] "Generic (PLEG): container finished" podID="f3c09fc5-3ccb-4ff9-888e-5ff3e50910ff" containerID="e270a0075cd3873611a83b386dc5ba93efea3ab3956bfc75970e373be5920f9f" exitCode=0
Sep 18 20:08:04.832129 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:08:04.831764    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rqpkw" event={"ID":"f3c09fc5-3ccb-4ff9-888e-5ff3e50910ff","Type":"ContainerDied","Data":"e270a0075cd3873611a83b386dc5ba93efea3ab3956bfc75970e373be5920f9f"}
Sep 18 20:08:11.862379 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:08:11.862344    2532 generic.go:356] "Generic (PLEG): container finished" podID="f3c09fc5-3ccb-4ff9-888e-5ff3e50910ff" containerID="0f24f993683b6ff2c053221286b8ee49f89a713ba46284df8b460733e2dbf3e9" exitCode=0
Sep 18 20:08:11.862779 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:08:11.862396    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rqpkw" event={"ID":"f3c09fc5-3ccb-4ff9-888e-5ff3e50910ff","Type":"ContainerDied","Data":"0f24f993683b6ff2c053221286b8ee49f89a713ba46284df8b460733e2dbf3e9"}
Sep 18 20:08:13.000531 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:08:13.000507    2532 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rqpkw"
Sep 18 20:08:13.082983 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:08:13.082937    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zb7hw\" (UniqueName: \"kubernetes.io/projected/f3c09fc5-3ccb-4ff9-888e-5ff3e50910ff-kube-api-access-zb7hw\") pod \"f3c09fc5-3ccb-4ff9-888e-5ff3e50910ff\" (UID: \"f3c09fc5-3ccb-4ff9-888e-5ff3e50910ff\") "
Sep 18 20:08:13.083122 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:08:13.083058    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f3c09fc5-3ccb-4ff9-888e-5ff3e50910ff-bundle\") pod \"f3c09fc5-3ccb-4ff9-888e-5ff3e50910ff\" (UID: \"f3c09fc5-3ccb-4ff9-888e-5ff3e50910ff\") "
Sep 18 20:08:13.083122 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:08:13.083110    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f3c09fc5-3ccb-4ff9-888e-5ff3e50910ff-util\") pod \"f3c09fc5-3ccb-4ff9-888e-5ff3e50910ff\" (UID: \"f3c09fc5-3ccb-4ff9-888e-5ff3e50910ff\") "
Sep 18 20:08:13.083765 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:08:13.083736    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f3c09fc5-3ccb-4ff9-888e-5ff3e50910ff-bundle" (OuterVolumeSpecName: "bundle") pod "f3c09fc5-3ccb-4ff9-888e-5ff3e50910ff" (UID: "f3c09fc5-3ccb-4ff9-888e-5ff3e50910ff"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 20:08:13.085161 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:08:13.085138    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f3c09fc5-3ccb-4ff9-888e-5ff3e50910ff-kube-api-access-zb7hw" (OuterVolumeSpecName: "kube-api-access-zb7hw") pod "f3c09fc5-3ccb-4ff9-888e-5ff3e50910ff" (UID: "f3c09fc5-3ccb-4ff9-888e-5ff3e50910ff"). InnerVolumeSpecName "kube-api-access-zb7hw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 20:08:13.088194 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:08:13.088172    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f3c09fc5-3ccb-4ff9-888e-5ff3e50910ff-util" (OuterVolumeSpecName: "util") pod "f3c09fc5-3ccb-4ff9-888e-5ff3e50910ff" (UID: "f3c09fc5-3ccb-4ff9-888e-5ff3e50910ff"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 20:08:13.183872 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:08:13.183781    2532 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f3c09fc5-3ccb-4ff9-888e-5ff3e50910ff-util\") on node \"ip-10-0-130-81.ec2.internal\" DevicePath \"\""
Sep 18 20:08:13.183872 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:08:13.183825    2532 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zb7hw\" (UniqueName: \"kubernetes.io/projected/f3c09fc5-3ccb-4ff9-888e-5ff3e50910ff-kube-api-access-zb7hw\") on node \"ip-10-0-130-81.ec2.internal\" DevicePath \"\""
Sep 18 20:08:13.183872 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:08:13.183836    2532 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f3c09fc5-3ccb-4ff9-888e-5ff3e50910ff-bundle\") on node \"ip-10-0-130-81.ec2.internal\" DevicePath \"\""
Sep 18 20:08:13.876964 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:08:13.876936    2532 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rqpkw"
Sep 18 20:08:13.876964 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:08:13.876948    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rqpkw" event={"ID":"f3c09fc5-3ccb-4ff9-888e-5ff3e50910ff","Type":"ContainerDied","Data":"740f0c58b339a4644625f7f4dbb2db1fa3925db95f73f4dd76e971bad5d84aaf"}
Sep 18 20:08:13.877166 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:08:13.876997    2532 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="740f0c58b339a4644625f7f4dbb2db1fa3925db95f73f4dd76e971bad5d84aaf"
Sep 18 20:08:14.958733 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:08:14.958704    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:08:15.739321 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:08:15.739289    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:08:16.846872 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:08:16.846835    2532 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-r8q77"]
Sep 18 20:08:16.847245 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:08:16.847102    2532 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f3c09fc5-3ccb-4ff9-888e-5ff3e50910ff" containerName="pull"
Sep 18 20:08:16.847245 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:08:16.847112    2532 state_mem.go:107] "Deleted CPUSet assignment" podUID="f3c09fc5-3ccb-4ff9-888e-5ff3e50910ff" containerName="pull"
Sep 18 20:08:16.847245 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:08:16.847123    2532 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f3c09fc5-3ccb-4ff9-888e-5ff3e50910ff" containerName="extract"
Sep 18 20:08:16.847245 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:08:16.847129    2532 state_mem.go:107] "Deleted CPUSet assignment" podUID="f3c09fc5-3ccb-4ff9-888e-5ff3e50910ff" containerName="extract"
Sep 18 20:08:16.847245 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:08:16.847136    2532 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f3c09fc5-3ccb-4ff9-888e-5ff3e50910ff" containerName="util"
Sep 18 20:08:16.847245 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:08:16.847141    2532 state_mem.go:107] "Deleted CPUSet assignment" podUID="f3c09fc5-3ccb-4ff9-888e-5ff3e50910ff" containerName="util"
Sep 18 20:08:16.847245 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:08:16.847185    2532 memory_manager.go:368] "RemoveStaleState removing state" podUID="f3c09fc5-3ccb-4ff9-888e-5ff3e50910ff" containerName="extract"
Sep 18 20:08:16.891458 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:08:16.891433    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-r8q77"]
Sep 18 20:08:16.891598 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:08:16.891556    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-r8q77"
Sep 18 20:08:16.894066 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:08:16.894046    2532 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-67ksk\""
Sep 18 20:08:16.894336 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:08:16.894322    2532 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\""
Sep 18 20:08:16.894450 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:08:16.894433    2532 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\""
Sep 18 20:08:17.016519 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:08:17.016475    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f03cdc71-f660-4a6f-a8a1-2357f5130346-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-r8q77\" (UID: \"f03cdc71-f660-4a6f-a8a1-2357f5130346\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-r8q77"
Sep 18 20:08:17.016519 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:08:17.016522    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9g48w\" (UniqueName: \"kubernetes.io/projected/f03cdc71-f660-4a6f-a8a1-2357f5130346-kube-api-access-9g48w\") pod \"cert-manager-operator-controller-manager-55c444bf86-r8q77\" (UID: \"f03cdc71-f660-4a6f-a8a1-2357f5130346\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-r8q77"
Sep 18 20:08:17.117911 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:08:17.117822    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f03cdc71-f660-4a6f-a8a1-2357f5130346-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-r8q77\" (UID: \"f03cdc71-f660-4a6f-a8a1-2357f5130346\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-r8q77"
Sep 18 20:08:17.117911 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:08:17.117859    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9g48w\" (UniqueName: \"kubernetes.io/projected/f03cdc71-f660-4a6f-a8a1-2357f5130346-kube-api-access-9g48w\") pod \"cert-manager-operator-controller-manager-55c444bf86-r8q77\" (UID: \"f03cdc71-f660-4a6f-a8a1-2357f5130346\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-r8q77"
Sep 18 20:08:17.118258 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:08:17.118236    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f03cdc71-f660-4a6f-a8a1-2357f5130346-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-r8q77\" (UID: \"f03cdc71-f660-4a6f-a8a1-2357f5130346\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-r8q77"
Sep 18 20:08:17.125778 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:08:17.125754    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9g48w\" (UniqueName: \"kubernetes.io/projected/f03cdc71-f660-4a6f-a8a1-2357f5130346-kube-api-access-9g48w\") pod \"cert-manager-operator-controller-manager-55c444bf86-r8q77\" (UID: \"f03cdc71-f660-4a6f-a8a1-2357f5130346\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-r8q77"
Sep 18 20:08:17.199720 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:08:17.199689    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-r8q77"
Sep 18 20:08:17.494034 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:08:17.494010    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-r8q77"]
Sep 18 20:08:17.496797 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:08:17.496764    2532 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf03cdc71_f660_4a6f_a8a1_2357f5130346.slice/crio-f47ffab60575df64ac7c3d66d629a516eacd18b01431e9fb3e01f0d357d3e955 WatchSource:0}: Error finding container f47ffab60575df64ac7c3d66d629a516eacd18b01431e9fb3e01f0d357d3e955: Status 404 returned error can't find the container with id f47ffab60575df64ac7c3d66d629a516eacd18b01431e9fb3e01f0d357d3e955
Sep 18 20:08:17.893498 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:08:17.893400    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-r8q77" event={"ID":"f03cdc71-f660-4a6f-a8a1-2357f5130346","Type":"ContainerStarted","Data":"f47ffab60575df64ac7c3d66d629a516eacd18b01431e9fb3e01f0d357d3e955"}
Sep 18 20:08:18.586420 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:08:18.586394    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:08:18.693288 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:08:18.693254    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:08:19.903742 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:08:19.903662    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-r8q77" event={"ID":"f03cdc71-f660-4a6f-a8a1-2357f5130346","Type":"ContainerStarted","Data":"20eb73506eede94882c47eb8af499a3cbfcaba43bf7491594620655d7364a771"}
Sep 18 20:08:20.123889 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:08:20.123858    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:08:20.221222 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:08:20.221192    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:08:20.235766 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:08:20.235707    2532 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-r8q77" podStartSLOduration=2.257775157 podStartE2EDuration="4.235684099s" totalImagesPullingTime="1.977908942s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 20:08:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 20:08:19.928648136 +0000 UTC m=+431.694037538" watchObservedRunningTime="2026-09-18 20:08:20.235684099 +0000 UTC m=+432.001073489"
Sep 18 20:08:20.237167 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:08:20.237145    2532 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-4nqpk"]
Sep 18 20:08:20.261974 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:08:20.261948    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-4nqpk"]
Sep 18 20:08:20.262107 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:08:20.262060    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-4nqpk"
Sep 18 20:08:20.265032 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:08:20.265015    2532 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Sep 18 20:08:20.265886 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:08:20.265868    2532 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Sep 18 20:08:20.349684 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:08:20.349649    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2b4sf\" (UniqueName: \"kubernetes.io/projected/98d2ff5b-ebbc-4f09-9e35-bac42de09fb9-kube-api-access-2b4sf\") pod \"cert-manager-webhook-6486ffd997-4nqpk\" (UID: \"98d2ff5b-ebbc-4f09-9e35-bac42de09fb9\") " pod="cert-manager/cert-manager-webhook-6486ffd997-4nqpk"
Sep 18 20:08:20.349850 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:08:20.349717    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/98d2ff5b-ebbc-4f09-9e35-bac42de09fb9-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-4nqpk\" (UID: \"98d2ff5b-ebbc-4f09-9e35-bac42de09fb9\") " pod="cert-manager/cert-manager-webhook-6486ffd997-4nqpk"
Sep 18 20:08:20.411959 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:08:20.411929    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:08:20.451100 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:08:20.451063    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2b4sf\" (UniqueName: \"kubernetes.io/projected/98d2ff5b-ebbc-4f09-9e35-bac42de09fb9-kube-api-access-2b4sf\") pod \"cert-manager-webhook-6486ffd997-4nqpk\" (UID: \"98d2ff5b-ebbc-4f09-9e35-bac42de09fb9\") " pod="cert-manager/cert-manager-webhook-6486ffd997-4nqpk"
Sep 18 20:08:20.451100 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:08:20.451105    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/98d2ff5b-ebbc-4f09-9e35-bac42de09fb9-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-4nqpk\" (UID: \"98d2ff5b-ebbc-4f09-9e35-bac42de09fb9\") " pod="cert-manager/cert-manager-webhook-6486ffd997-4nqpk"
Sep 18 20:08:20.471837 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:08:20.471775    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/98d2ff5b-ebbc-4f09-9e35-bac42de09fb9-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-4nqpk\" (UID: \"98d2ff5b-ebbc-4f09-9e35-bac42de09fb9\") " pod="cert-manager/cert-manager-webhook-6486ffd997-4nqpk"
Sep 18 20:08:20.471970 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:08:20.471949    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2b4sf\" (UniqueName: \"kubernetes.io/projected/98d2ff5b-ebbc-4f09-9e35-bac42de09fb9-kube-api-access-2b4sf\") pod \"cert-manager-webhook-6486ffd997-4nqpk\" (UID: \"98d2ff5b-ebbc-4f09-9e35-bac42de09fb9\") " pod="cert-manager/cert-manager-webhook-6486ffd997-4nqpk"
Sep 18 20:08:20.521262 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:08:20.521225    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:08:20.586622 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:08:20.586584    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-4nqpk"
Sep 18 20:08:20.878530 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:08:20.878344    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-4nqpk"]
Sep 18 20:08:20.881162 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:08:20.881138    2532 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod98d2ff5b_ebbc_4f09_9e35_bac42de09fb9.slice/crio-67f7b647074e2a95fe80c28ecefa47a9291c07967ceeebf4f420f04874fcdaf4 WatchSource:0}: Error finding container 67f7b647074e2a95fe80c28ecefa47a9291c07967ceeebf4f420f04874fcdaf4: Status 404 returned error can't find the container with id 67f7b647074e2a95fe80c28ecefa47a9291c07967ceeebf4f420f04874fcdaf4
Sep 18 20:08:20.909208 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:08:20.909183    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-4nqpk" event={"ID":"98d2ff5b-ebbc-4f09-9e35-bac42de09fb9","Type":"ContainerStarted","Data":"67f7b647074e2a95fe80c28ecefa47a9291c07967ceeebf4f420f04874fcdaf4"}
Sep 18 20:08:21.058199 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:08:21.058167    2532 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fpj2p8"]
Sep 18 20:08:21.082517 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:08:21.082481    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fpj2p8"]
Sep 18 20:08:21.082672 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:08:21.082576    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fpj2p8"
Sep 18 20:08:21.087310 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:08:21.087283    2532 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Sep 18 20:08:21.087445 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:08:21.087433    2532 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-62r9t\""
Sep 18 20:08:21.087696 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:08:21.087673    2532 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Sep 18 20:08:21.159940 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:08:21.159855    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/350c9004-bd27-4ff9-88fd-3c06cf4eae0b-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fpj2p8\" (UID: \"350c9004-bd27-4ff9-88fd-3c06cf4eae0b\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fpj2p8"
Sep 18 20:08:21.159940 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:08:21.159909    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/350c9004-bd27-4ff9-88fd-3c06cf4eae0b-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fpj2p8\" (UID: \"350c9004-bd27-4ff9-88fd-3c06cf4eae0b\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fpj2p8"
Sep 18 20:08:21.160142 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:08:21.160022    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l8slp\" (UniqueName: \"kubernetes.io/projected/350c9004-bd27-4ff9-88fd-3c06cf4eae0b-kube-api-access-l8slp\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fpj2p8\" (UID: \"350c9004-bd27-4ff9-88fd-3c06cf4eae0b\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fpj2p8"
Sep 18 20:08:21.260352 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:08:21.260318    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l8slp\" (UniqueName: \"kubernetes.io/projected/350c9004-bd27-4ff9-88fd-3c06cf4eae0b-kube-api-access-l8slp\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fpj2p8\" (UID: \"350c9004-bd27-4ff9-88fd-3c06cf4eae0b\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fpj2p8"
Sep 18 20:08:21.260352 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:08:21.260355    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/350c9004-bd27-4ff9-88fd-3c06cf4eae0b-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fpj2p8\" (UID: \"350c9004-bd27-4ff9-88fd-3c06cf4eae0b\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fpj2p8"
Sep 18 20:08:21.260641 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:08:21.260383    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/350c9004-bd27-4ff9-88fd-3c06cf4eae0b-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fpj2p8\" (UID: \"350c9004-bd27-4ff9-88fd-3c06cf4eae0b\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fpj2p8"
Sep 18 20:08:21.260766 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:08:21.260747    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/350c9004-bd27-4ff9-88fd-3c06cf4eae0b-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fpj2p8\" (UID: \"350c9004-bd27-4ff9-88fd-3c06cf4eae0b\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fpj2p8"
Sep 18 20:08:21.260822 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:08:21.260775    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/350c9004-bd27-4ff9-88fd-3c06cf4eae0b-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fpj2p8\" (UID: \"350c9004-bd27-4ff9-88fd-3c06cf4eae0b\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fpj2p8"
Sep 18 20:08:21.271353 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:08:21.271329    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l8slp\" (UniqueName: \"kubernetes.io/projected/350c9004-bd27-4ff9-88fd-3c06cf4eae0b-kube-api-access-l8slp\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fpj2p8\" (UID: \"350c9004-bd27-4ff9-88fd-3c06cf4eae0b\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fpj2p8"
Sep 18 20:08:21.391886 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:08:21.391853    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fpj2p8"
Sep 18 20:08:21.668419 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:08:21.668343    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fpj2p8"]
Sep 18 20:08:21.671473 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:08:21.671446    2532 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod350c9004_bd27_4ff9_88fd_3c06cf4eae0b.slice/crio-c743051e0e0dc0f988b557cc399d77b6cd3e22a24df1fc24318706a0436219e0 WatchSource:0}: Error finding container c743051e0e0dc0f988b557cc399d77b6cd3e22a24df1fc24318706a0436219e0: Status 404 returned error can't find the container with id c743051e0e0dc0f988b557cc399d77b6cd3e22a24df1fc24318706a0436219e0
Sep 18 20:08:21.917166 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:08:21.917126    2532 generic.go:356] "Generic (PLEG): container finished" podID="350c9004-bd27-4ff9-88fd-3c06cf4eae0b" containerID="d93099cdcd4c38139ed3e794b22305d2cec7961e5d68c487c4cbd1915f4ef854" exitCode=0
Sep 18 20:08:21.917647 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:08:21.917221    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fpj2p8" event={"ID":"350c9004-bd27-4ff9-88fd-3c06cf4eae0b","Type":"ContainerDied","Data":"d93099cdcd4c38139ed3e794b22305d2cec7961e5d68c487c4cbd1915f4ef854"}
Sep 18 20:08:21.917647 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:08:21.917295    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fpj2p8" event={"ID":"350c9004-bd27-4ff9-88fd-3c06cf4eae0b","Type":"ContainerStarted","Data":"c743051e0e0dc0f988b557cc399d77b6cd3e22a24df1fc24318706a0436219e0"}
Sep 18 20:08:22.498882 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:08:22.498747    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:08:22.906411 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:08:22.906325    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:08:24.935325 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:08:24.935227    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-4nqpk" event={"ID":"98d2ff5b-ebbc-4f09-9e35-bac42de09fb9","Type":"ContainerStarted","Data":"7436dc11a536bfeec5f5a5d755e81ddaf6282b3b97dc9760c1fe0cc76135406b"}
Sep 18 20:08:24.935801 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:08:24.935396    2532 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-4nqpk"
Sep 18 20:08:24.937311 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:08:24.937287    2532 generic.go:356] "Generic (PLEG): container finished" podID="350c9004-bd27-4ff9-88fd-3c06cf4eae0b" containerID="34f0152ca6e9eba6dfc4822c6a7ac19db1c0bb3cd2f6cfec3bcd0cd576b21287" exitCode=0
Sep 18 20:08:24.937431 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:08:24.937316    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fpj2p8" event={"ID":"350c9004-bd27-4ff9-88fd-3c06cf4eae0b","Type":"ContainerDied","Data":"34f0152ca6e9eba6dfc4822c6a7ac19db1c0bb3cd2f6cfec3bcd0cd576b21287"}
Sep 18 20:08:24.950739 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:08:24.950686    2532 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-4nqpk" podStartSLOduration=1.342329878 podStartE2EDuration="4.950669205s" totalImagesPullingTime="3.608339327s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 20:08:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 20:08:24.949958699 +0000 UTC m=+436.715348099" watchObservedRunningTime="2026-09-18 20:08:24.950669205 +0000 UTC m=+436.716058608"
Sep 18 20:08:25.367734 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:08:25.367692    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:08:25.429501 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:08:25.429455    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:08:25.943435 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:08:25.943399    2532 generic.go:356] "Generic (PLEG): container finished" podID="350c9004-bd27-4ff9-88fd-3c06cf4eae0b" containerID="dd9b306f6ba372bd9fd7bb721946309f040b722af5326364d3e60ea816356e73" exitCode=0
Sep 18 20:08:25.943921 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:08:25.943503    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fpj2p8" event={"ID":"350c9004-bd27-4ff9-88fd-3c06cf4eae0b","Type":"ContainerDied","Data":"dd9b306f6ba372bd9fd7bb721946309f040b722af5326364d3e60ea816356e73"}
Sep 18 20:08:26.875360 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:08:26.875329    2532 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-8l6g6"]
Sep 18 20:08:26.879654 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:08:26.879631    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-8l6g6"
Sep 18 20:08:26.881926 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:08:26.881904    2532 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-fk8sv\""
Sep 18 20:08:26.885669 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:08:26.885650    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-8l6g6"]
Sep 18 20:08:27.010040 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:08:27.009987    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/64c3bdb9-aad3-4af3-b72a-096580f54260-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-8l6g6\" (UID: \"64c3bdb9-aad3-4af3-b72a-096580f54260\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-8l6g6"
Sep 18 20:08:27.010385 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:08:27.010098    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kr4sx\" (UniqueName: \"kubernetes.io/projected/64c3bdb9-aad3-4af3-b72a-096580f54260-kube-api-access-kr4sx\") pod \"cert-manager-cainjector-559cdd8c87-8l6g6\" (UID: \"64c3bdb9-aad3-4af3-b72a-096580f54260\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-8l6g6"
Sep 18 20:08:27.069894 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:08:27.069872    2532 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fpj2p8"
Sep 18 20:08:27.111032 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:08:27.110999    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kr4sx\" (UniqueName: \"kubernetes.io/projected/64c3bdb9-aad3-4af3-b72a-096580f54260-kube-api-access-kr4sx\") pod \"cert-manager-cainjector-559cdd8c87-8l6g6\" (UID: \"64c3bdb9-aad3-4af3-b72a-096580f54260\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-8l6g6"
Sep 18 20:08:27.111180 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:08:27.111044    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/64c3bdb9-aad3-4af3-b72a-096580f54260-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-8l6g6\" (UID: \"64c3bdb9-aad3-4af3-b72a-096580f54260\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-8l6g6"
Sep 18 20:08:27.119386 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:08:27.119361    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/64c3bdb9-aad3-4af3-b72a-096580f54260-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-8l6g6\" (UID: \"64c3bdb9-aad3-4af3-b72a-096580f54260\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-8l6g6"
Sep 18 20:08:27.119707 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:08:27.119686    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kr4sx\" (UniqueName: \"kubernetes.io/projected/64c3bdb9-aad3-4af3-b72a-096580f54260-kube-api-access-kr4sx\") pod \"cert-manager-cainjector-559cdd8c87-8l6g6\" (UID: \"64c3bdb9-aad3-4af3-b72a-096580f54260\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-8l6g6"
Sep 18 20:08:27.189732 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:08:27.189634    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-8l6g6"
Sep 18 20:08:27.212079 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:08:27.212049    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/350c9004-bd27-4ff9-88fd-3c06cf4eae0b-bundle\") pod \"350c9004-bd27-4ff9-88fd-3c06cf4eae0b\" (UID: \"350c9004-bd27-4ff9-88fd-3c06cf4eae0b\") "
Sep 18 20:08:27.212217 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:08:27.212106    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/350c9004-bd27-4ff9-88fd-3c06cf4eae0b-util\") pod \"350c9004-bd27-4ff9-88fd-3c06cf4eae0b\" (UID: \"350c9004-bd27-4ff9-88fd-3c06cf4eae0b\") "
Sep 18 20:08:27.212217 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:08:27.212196    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-l8slp\" (UniqueName: \"kubernetes.io/projected/350c9004-bd27-4ff9-88fd-3c06cf4eae0b-kube-api-access-l8slp\") pod \"350c9004-bd27-4ff9-88fd-3c06cf4eae0b\" (UID: \"350c9004-bd27-4ff9-88fd-3c06cf4eae0b\") "
Sep 18 20:08:27.212447 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:08:27.212425    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/350c9004-bd27-4ff9-88fd-3c06cf4eae0b-bundle" (OuterVolumeSpecName: "bundle") pod "350c9004-bd27-4ff9-88fd-3c06cf4eae0b" (UID: "350c9004-bd27-4ff9-88fd-3c06cf4eae0b"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 20:08:27.214229 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:08:27.214204    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/350c9004-bd27-4ff9-88fd-3c06cf4eae0b-kube-api-access-l8slp" (OuterVolumeSpecName: "kube-api-access-l8slp") pod "350c9004-bd27-4ff9-88fd-3c06cf4eae0b" (UID: "350c9004-bd27-4ff9-88fd-3c06cf4eae0b"). InnerVolumeSpecName "kube-api-access-l8slp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 20:08:27.216369 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:08:27.216346    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/350c9004-bd27-4ff9-88fd-3c06cf4eae0b-util" (OuterVolumeSpecName: "util") pod "350c9004-bd27-4ff9-88fd-3c06cf4eae0b" (UID: "350c9004-bd27-4ff9-88fd-3c06cf4eae0b"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 20:08:27.313279 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:08:27.313231    2532 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l8slp\" (UniqueName: \"kubernetes.io/projected/350c9004-bd27-4ff9-88fd-3c06cf4eae0b-kube-api-access-l8slp\") on node \"ip-10-0-130-81.ec2.internal\" DevicePath \"\""
Sep 18 20:08:27.313279 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:08:27.313272    2532 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/350c9004-bd27-4ff9-88fd-3c06cf4eae0b-bundle\") on node \"ip-10-0-130-81.ec2.internal\" DevicePath \"\""
Sep 18 20:08:27.313279 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:08:27.313281    2532 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/350c9004-bd27-4ff9-88fd-3c06cf4eae0b-util\") on node \"ip-10-0-130-81.ec2.internal\" DevicePath \"\""
Sep 18 20:08:27.458647 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:08:27.458621    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-8l6g6"]
Sep 18 20:08:27.461273 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:08:27.461240    2532 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod64c3bdb9_aad3_4af3_b72a_096580f54260.slice/crio-3b28c7af939c61cfff25575a0b749d488b166e3b5ca064fb797230f66e885aaa WatchSource:0}: Error finding container 3b28c7af939c61cfff25575a0b749d488b166e3b5ca064fb797230f66e885aaa: Status 404 returned error can't find the container with id 3b28c7af939c61cfff25575a0b749d488b166e3b5ca064fb797230f66e885aaa
Sep 18 20:08:27.952797 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:08:27.952700    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-8l6g6" event={"ID":"64c3bdb9-aad3-4af3-b72a-096580f54260","Type":"ContainerStarted","Data":"93e2f7e5fd5c95d449ccef1f2fc79efdd5a85d276342ae31387d17bb494981c1"}
Sep 18 20:08:27.952797 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:08:27.952749    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-8l6g6" event={"ID":"64c3bdb9-aad3-4af3-b72a-096580f54260","Type":"ContainerStarted","Data":"3b28c7af939c61cfff25575a0b749d488b166e3b5ca064fb797230f66e885aaa"}
Sep 18 20:08:27.954631 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:08:27.954608    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fpj2p8" event={"ID":"350c9004-bd27-4ff9-88fd-3c06cf4eae0b","Type":"ContainerDied","Data":"c743051e0e0dc0f988b557cc399d77b6cd3e22a24df1fc24318706a0436219e0"}
Sep 18 20:08:27.954723 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:08:27.954637    2532 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fpj2p8"
Sep 18 20:08:27.954723 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:08:27.954671    2532 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c743051e0e0dc0f988b557cc399d77b6cd3e22a24df1fc24318706a0436219e0"
Sep 18 20:08:27.968322 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:08:27.968285    2532 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-8l6g6" podStartSLOduration=1.968272465 podStartE2EDuration="1.968272465s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 20:08:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 20:08:27.967236789 +0000 UTC m=+439.732626192" watchObservedRunningTime="2026-09-18 20:08:27.968272465 +0000 UTC m=+439.733661864"
Sep 18 20:08:28.234029 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:08:28.233987    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:08:28.347978 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:08:28.347940    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:08:30.716206 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:08:30.716154    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:08:30.945773 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:08:30.945745    2532 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-4nqpk"
Sep 18 20:08:32.931934 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:08:32.931904    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:08:33.032954 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:08:33.032924    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:08:38.456568 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:08:38.456526    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:08:38.560225 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:08:38.560193    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:08:40.360800 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:08:40.360770    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:08:40.445998 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:08:40.445966    2532 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aflct42"]
Sep 18 20:08:40.446331 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:08:40.446314    2532 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="350c9004-bd27-4ff9-88fd-3c06cf4eae0b" containerName="util"
Sep 18 20:08:40.446405 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:08:40.446333    2532 state_mem.go:107] "Deleted CPUSet assignment" podUID="350c9004-bd27-4ff9-88fd-3c06cf4eae0b" containerName="util"
Sep 18 20:08:40.446405 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:08:40.446351    2532 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="350c9004-bd27-4ff9-88fd-3c06cf4eae0b" containerName="extract"
Sep 18 20:08:40.446405 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:08:40.446359    2532 state_mem.go:107] "Deleted CPUSet assignment" podUID="350c9004-bd27-4ff9-88fd-3c06cf4eae0b" containerName="extract"
Sep 18 20:08:40.446405 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:08:40.446393    2532 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="350c9004-bd27-4ff9-88fd-3c06cf4eae0b" containerName="pull"
Sep 18 20:08:40.446405 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:08:40.446401    2532 state_mem.go:107] "Deleted CPUSet assignment" podUID="350c9004-bd27-4ff9-88fd-3c06cf4eae0b" containerName="pull"
Sep 18 20:08:40.446673 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:08:40.446476    2532 memory_manager.go:368] "RemoveStaleState removing state" podUID="350c9004-bd27-4ff9-88fd-3c06cf4eae0b" containerName="extract"
Sep 18 20:08:40.449933 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:08:40.449911    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aflct42"
Sep 18 20:08:40.452354 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:08:40.452336    2532 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-62r9t\""
Sep 18 20:08:40.452354 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:08:40.452344    2532 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Sep 18 20:08:40.453212 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:08:40.453198    2532 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Sep 18 20:08:40.459668 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:08:40.459645    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aflct42"]
Sep 18 20:08:40.520228 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:08:40.520198    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vxqjr\" (UniqueName: \"kubernetes.io/projected/2df468ec-934e-45ed-b23a-a726ee242754-kube-api-access-vxqjr\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aflct42\" (UID: \"2df468ec-934e-45ed-b23a-a726ee242754\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aflct42"
Sep 18 20:08:40.520373 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:08:40.520243    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2df468ec-934e-45ed-b23a-a726ee242754-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aflct42\" (UID: \"2df468ec-934e-45ed-b23a-a726ee242754\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aflct42"
Sep 18 20:08:40.520373 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:08:40.520308    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2df468ec-934e-45ed-b23a-a726ee242754-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aflct42\" (UID: \"2df468ec-934e-45ed-b23a-a726ee242754\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aflct42"
Sep 18 20:08:40.621378 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:08:40.621291    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vxqjr\" (UniqueName: \"kubernetes.io/projected/2df468ec-934e-45ed-b23a-a726ee242754-kube-api-access-vxqjr\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aflct42\" (UID: \"2df468ec-934e-45ed-b23a-a726ee242754\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aflct42"
Sep 18 20:08:40.621378 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:08:40.621348    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2df468ec-934e-45ed-b23a-a726ee242754-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aflct42\" (UID: \"2df468ec-934e-45ed-b23a-a726ee242754\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aflct42"
Sep 18 20:08:40.621601 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:08:40.621400    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2df468ec-934e-45ed-b23a-a726ee242754-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aflct42\" (UID: \"2df468ec-934e-45ed-b23a-a726ee242754\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aflct42"
Sep 18 20:08:40.621784 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:08:40.621769    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2df468ec-934e-45ed-b23a-a726ee242754-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aflct42\" (UID: \"2df468ec-934e-45ed-b23a-a726ee242754\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aflct42"
Sep 18 20:08:40.621831 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:08:40.621813    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2df468ec-934e-45ed-b23a-a726ee242754-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aflct42\" (UID: \"2df468ec-934e-45ed-b23a-a726ee242754\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aflct42"
Sep 18 20:08:40.629422 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:08:40.629396    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vxqjr\" (UniqueName: \"kubernetes.io/projected/2df468ec-934e-45ed-b23a-a726ee242754-kube-api-access-vxqjr\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aflct42\" (UID: \"2df468ec-934e-45ed-b23a-a726ee242754\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aflct42"
Sep 18 20:08:40.758689 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:08:40.758635    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aflct42"
Sep 18 20:08:41.026395 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:08:41.026371    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aflct42"]
Sep 18 20:08:41.028453 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:08:41.028426    2532 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2df468ec_934e_45ed_b23a_a726ee242754.slice/crio-6815e9bc8920acbc412f9badb09df3505d0827b1b956fd380ed700d40f9ad639 WatchSource:0}: Error finding container 6815e9bc8920acbc412f9badb09df3505d0827b1b956fd380ed700d40f9ad639: Status 404 returned error can't find the container with id 6815e9bc8920acbc412f9badb09df3505d0827b1b956fd380ed700d40f9ad639
Sep 18 20:08:41.940736 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:08:41.940656    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:08:41.992393 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:08:41.992366    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:08:42.014970 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:08:42.014944    2532 generic.go:356] "Generic (PLEG): container finished" podID="2df468ec-934e-45ed-b23a-a726ee242754" containerID="5eaeb5050cef6f1771956ea75c12093673ac779f7929977fe857b0bb6ca6bced" exitCode=0
Sep 18 20:08:42.015099 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:08:42.014993    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aflct42" event={"ID":"2df468ec-934e-45ed-b23a-a726ee242754","Type":"ContainerDied","Data":"5eaeb5050cef6f1771956ea75c12093673ac779f7929977fe857b0bb6ca6bced"}
Sep 18 20:08:42.015099 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:08:42.015028    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aflct42" event={"ID":"2df468ec-934e-45ed-b23a-a726ee242754","Type":"ContainerStarted","Data":"6815e9bc8920acbc412f9badb09df3505d0827b1b956fd380ed700d40f9ad639"}
Sep 18 20:08:43.021777 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:08:43.021738    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aflct42" event={"ID":"2df468ec-934e-45ed-b23a-a726ee242754","Type":"ContainerStarted","Data":"b79b7b3af6884a5ca793c764b8242f4824b4485059ee481dbb7e8a9da77464e9"}
Sep 18 20:08:44.027254 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:08:44.027220    2532 generic.go:356] "Generic (PLEG): container finished" podID="2df468ec-934e-45ed-b23a-a726ee242754" containerID="b79b7b3af6884a5ca793c764b8242f4824b4485059ee481dbb7e8a9da77464e9" exitCode=0
Sep 18 20:08:44.027669 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:08:44.027303    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aflct42" event={"ID":"2df468ec-934e-45ed-b23a-a726ee242754","Type":"ContainerDied","Data":"b79b7b3af6884a5ca793c764b8242f4824b4485059ee481dbb7e8a9da77464e9"}
Sep 18 20:08:45.034697 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:08:45.034657    2532 generic.go:356] "Generic (PLEG): container finished" podID="2df468ec-934e-45ed-b23a-a726ee242754" containerID="bc865fd3322094fd4ac6e07fbd9dd290ed20f88e4f9c8a49eef9a51ea5b0dfa0" exitCode=0
Sep 18 20:08:45.035107 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:08:45.034699    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aflct42" event={"ID":"2df468ec-934e-45ed-b23a-a726ee242754","Type":"ContainerDied","Data":"bc865fd3322094fd4ac6e07fbd9dd290ed20f88e4f9c8a49eef9a51ea5b0dfa0"}
Sep 18 20:08:46.169374 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:08:46.169347    2532 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aflct42"
Sep 18 20:08:46.260184 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:08:46.260150    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vxqjr\" (UniqueName: \"kubernetes.io/projected/2df468ec-934e-45ed-b23a-a726ee242754-kube-api-access-vxqjr\") pod \"2df468ec-934e-45ed-b23a-a726ee242754\" (UID: \"2df468ec-934e-45ed-b23a-a726ee242754\") "
Sep 18 20:08:46.260347 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:08:46.260196    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2df468ec-934e-45ed-b23a-a726ee242754-util\") pod \"2df468ec-934e-45ed-b23a-a726ee242754\" (UID: \"2df468ec-934e-45ed-b23a-a726ee242754\") "
Sep 18 20:08:46.260347 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:08:46.260212    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2df468ec-934e-45ed-b23a-a726ee242754-bundle\") pod \"2df468ec-934e-45ed-b23a-a726ee242754\" (UID: \"2df468ec-934e-45ed-b23a-a726ee242754\") "
Sep 18 20:08:46.261010 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:08:46.260980    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2df468ec-934e-45ed-b23a-a726ee242754-bundle" (OuterVolumeSpecName: "bundle") pod "2df468ec-934e-45ed-b23a-a726ee242754" (UID: "2df468ec-934e-45ed-b23a-a726ee242754"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 20:08:46.262374 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:08:46.262348    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2df468ec-934e-45ed-b23a-a726ee242754-kube-api-access-vxqjr" (OuterVolumeSpecName: "kube-api-access-vxqjr") pod "2df468ec-934e-45ed-b23a-a726ee242754" (UID: "2df468ec-934e-45ed-b23a-a726ee242754"). InnerVolumeSpecName "kube-api-access-vxqjr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 20:08:46.266168 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:08:46.266136    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2df468ec-934e-45ed-b23a-a726ee242754-util" (OuterVolumeSpecName: "util") pod "2df468ec-934e-45ed-b23a-a726ee242754" (UID: "2df468ec-934e-45ed-b23a-a726ee242754"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 20:08:46.361333 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:08:46.361301    2532 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vxqjr\" (UniqueName: \"kubernetes.io/projected/2df468ec-934e-45ed-b23a-a726ee242754-kube-api-access-vxqjr\") on node \"ip-10-0-130-81.ec2.internal\" DevicePath \"\""
Sep 18 20:08:46.361333 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:08:46.361328    2532 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2df468ec-934e-45ed-b23a-a726ee242754-util\") on node \"ip-10-0-130-81.ec2.internal\" DevicePath \"\""
Sep 18 20:08:46.361333 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:08:46.361339    2532 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2df468ec-934e-45ed-b23a-a726ee242754-bundle\") on node \"ip-10-0-130-81.ec2.internal\" DevicePath \"\""
Sep 18 20:08:47.046289 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:08:47.046263    2532 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aflct42"
Sep 18 20:08:47.046463 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:08:47.046259    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aflct42" event={"ID":"2df468ec-934e-45ed-b23a-a726ee242754","Type":"ContainerDied","Data":"6815e9bc8920acbc412f9badb09df3505d0827b1b956fd380ed700d40f9ad639"}
Sep 18 20:08:47.046463 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:08:47.046379    2532 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6815e9bc8920acbc412f9badb09df3505d0827b1b956fd380ed700d40f9ad639"
Sep 18 20:08:53.809351 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:08:53.809314    2532 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0t5vv6"]
Sep 18 20:08:53.809932 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:08:53.809719    2532 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2df468ec-934e-45ed-b23a-a726ee242754" containerName="util"
Sep 18 20:08:53.809932 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:08:53.809736    2532 state_mem.go:107] "Deleted CPUSet assignment" podUID="2df468ec-934e-45ed-b23a-a726ee242754" containerName="util"
Sep 18 20:08:53.809932 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:08:53.809763    2532 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2df468ec-934e-45ed-b23a-a726ee242754" containerName="extract"
Sep 18 20:08:53.809932 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:08:53.809772    2532 state_mem.go:107] "Deleted CPUSet assignment" podUID="2df468ec-934e-45ed-b23a-a726ee242754" containerName="extract"
Sep 18 20:08:53.809932 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:08:53.809791    2532 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2df468ec-934e-45ed-b23a-a726ee242754" containerName="pull"
Sep 18 20:08:53.809932 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:08:53.809799    2532 state_mem.go:107] "Deleted CPUSet assignment" podUID="2df468ec-934e-45ed-b23a-a726ee242754" containerName="pull"
Sep 18 20:08:53.809932 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:08:53.809877    2532 memory_manager.go:368] "RemoveStaleState removing state" podUID="2df468ec-934e-45ed-b23a-a726ee242754" containerName="extract"
Sep 18 20:08:53.813902 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:08:53.813880    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0t5vv6"
Sep 18 20:08:53.816399 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:08:53.816378    2532 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Sep 18 20:08:53.816517 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:08:53.816415    2532 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Sep 18 20:08:53.816517 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:08:53.816477    2532 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-62r9t\""
Sep 18 20:08:53.821263 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:08:53.821243    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0t5vv6"]
Sep 18 20:08:53.909844 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:08:53.909801    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-44m6d\" (UniqueName: \"kubernetes.io/projected/75e08b59-588b-4e59-8c5d-47c290d91ab2-kube-api-access-44m6d\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0t5vv6\" (UID: \"75e08b59-588b-4e59-8c5d-47c290d91ab2\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0t5vv6"
Sep 18 20:08:53.910032 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:08:53.909850    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/75e08b59-588b-4e59-8c5d-47c290d91ab2-util\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0t5vv6\" (UID: \"75e08b59-588b-4e59-8c5d-47c290d91ab2\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0t5vv6"
Sep 18 20:08:53.910032 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:08:53.909930    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/75e08b59-588b-4e59-8c5d-47c290d91ab2-bundle\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0t5vv6\" (UID: \"75e08b59-588b-4e59-8c5d-47c290d91ab2\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0t5vv6"
Sep 18 20:08:54.011025 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:08:54.010991    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-44m6d\" (UniqueName: \"kubernetes.io/projected/75e08b59-588b-4e59-8c5d-47c290d91ab2-kube-api-access-44m6d\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0t5vv6\" (UID: \"75e08b59-588b-4e59-8c5d-47c290d91ab2\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0t5vv6"
Sep 18 20:08:54.011025 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:08:54.011028    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/75e08b59-588b-4e59-8c5d-47c290d91ab2-util\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0t5vv6\" (UID: \"75e08b59-588b-4e59-8c5d-47c290d91ab2\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0t5vv6"
Sep 18 20:08:54.011244 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:08:54.011068    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/75e08b59-588b-4e59-8c5d-47c290d91ab2-bundle\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0t5vv6\" (UID: \"75e08b59-588b-4e59-8c5d-47c290d91ab2\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0t5vv6"
Sep 18 20:08:54.011395 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:08:54.011380    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/75e08b59-588b-4e59-8c5d-47c290d91ab2-bundle\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0t5vv6\" (UID: \"75e08b59-588b-4e59-8c5d-47c290d91ab2\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0t5vv6"
Sep 18 20:08:54.011441 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:08:54.011424    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/75e08b59-588b-4e59-8c5d-47c290d91ab2-util\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0t5vv6\" (UID: \"75e08b59-588b-4e59-8c5d-47c290d91ab2\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0t5vv6"
Sep 18 20:08:54.019180 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:08:54.019154    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-44m6d\" (UniqueName: \"kubernetes.io/projected/75e08b59-588b-4e59-8c5d-47c290d91ab2-kube-api-access-44m6d\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0t5vv6\" (UID: \"75e08b59-588b-4e59-8c5d-47c290d91ab2\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0t5vv6"
Sep 18 20:08:54.123921 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:08:54.123832    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0t5vv6"
Sep 18 20:08:54.413712 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:08:54.413647    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0t5vv6"]
Sep 18 20:08:54.416326 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:08:54.416303    2532 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod75e08b59_588b_4e59_8c5d_47c290d91ab2.slice/crio-59f927cb93b69130575e6834d8fa85ce1960deeab50031e84a27b1e8d5cd1a4f WatchSource:0}: Error finding container 59f927cb93b69130575e6834d8fa85ce1960deeab50031e84a27b1e8d5cd1a4f: Status 404 returned error can't find the container with id 59f927cb93b69130575e6834d8fa85ce1960deeab50031e84a27b1e8d5cd1a4f
Sep 18 20:08:54.907966 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:08:54.907933    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:08:54.997042 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:08:54.997013    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:08:55.083367 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:08:55.083337    2532 generic.go:356] "Generic (PLEG): container finished" podID="75e08b59-588b-4e59-8c5d-47c290d91ab2" containerID="d32bce99e3049fe7b0580e16dbc2bf9fbba62acf3dd083474dc994ad12b69f30" exitCode=0
Sep 18 20:08:55.083551 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:08:55.083431    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0t5vv6" event={"ID":"75e08b59-588b-4e59-8c5d-47c290d91ab2","Type":"ContainerDied","Data":"d32bce99e3049fe7b0580e16dbc2bf9fbba62acf3dd083474dc994ad12b69f30"}
Sep 18 20:08:55.083551 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:08:55.083507    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0t5vv6" event={"ID":"75e08b59-588b-4e59-8c5d-47c290d91ab2","Type":"ContainerStarted","Data":"59f927cb93b69130575e6834d8fa85ce1960deeab50031e84a27b1e8d5cd1a4f"}
Sep 18 20:08:55.532624 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:08:55.532590    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:08:55.739607 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:08:55.739570    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:08:55.835149 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:08:55.835120    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:08:55.885267 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:08:55.885234    2532 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-668c7cd96d-6blck"]
Sep 18 20:08:55.888389 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:08:55.888373    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-668c7cd96d-6blck"
Sep 18 20:08:55.891436 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:08:55.891417    2532 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Sep 18 20:08:55.891584 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:08:55.891422    2532 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Sep 18 20:08:55.891584 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:08:55.891510    2532 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-ws545\""
Sep 18 20:08:55.891584 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:08:55.891510    2532 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Sep 18 20:08:55.891713 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:08:55.891665    2532 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Sep 18 20:08:55.911578 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:08:55.911555    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-668c7cd96d-6blck"]
Sep 18 20:08:55.925662 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:08:55.925637    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/e064dbb8-6962-4c10-a13f-5893654576f5-manifests\") pod \"opendatahub-operator-controller-manager-668c7cd96d-6blck\" (UID: \"e064dbb8-6962-4c10-a13f-5893654576f5\") " pod="opendatahub/opendatahub-operator-controller-manager-668c7cd96d-6blck"
Sep 18 20:08:55.925754 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:08:55.925675    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/e064dbb8-6962-4c10-a13f-5893654576f5-apiservice-cert\") pod \"opendatahub-operator-controller-manager-668c7cd96d-6blck\" (UID: \"e064dbb8-6962-4c10-a13f-5893654576f5\") " pod="opendatahub/opendatahub-operator-controller-manager-668c7cd96d-6blck"
Sep 18 20:08:55.925754 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:08:55.925717    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e064dbb8-6962-4c10-a13f-5893654576f5-tmp\") pod \"opendatahub-operator-controller-manager-668c7cd96d-6blck\" (UID: \"e064dbb8-6962-4c10-a13f-5893654576f5\") " pod="opendatahub/opendatahub-operator-controller-manager-668c7cd96d-6blck"
Sep 18 20:08:55.925754 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:08:55.925743    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gsr26\" (UniqueName: \"kubernetes.io/projected/e064dbb8-6962-4c10-a13f-5893654576f5-kube-api-access-gsr26\") pod \"opendatahub-operator-controller-manager-668c7cd96d-6blck\" (UID: \"e064dbb8-6962-4c10-a13f-5893654576f5\") " pod="opendatahub/opendatahub-operator-controller-manager-668c7cd96d-6blck"
Sep 18 20:08:55.925915 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:08:55.925814    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e064dbb8-6962-4c10-a13f-5893654576f5-webhook-cert\") pod \"opendatahub-operator-controller-manager-668c7cd96d-6blck\" (UID: \"e064dbb8-6962-4c10-a13f-5893654576f5\") " pod="opendatahub/opendatahub-operator-controller-manager-668c7cd96d-6blck"
Sep 18 20:08:56.026718 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:08:56.026680    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e064dbb8-6962-4c10-a13f-5893654576f5-webhook-cert\") pod \"opendatahub-operator-controller-manager-668c7cd96d-6blck\" (UID: \"e064dbb8-6962-4c10-a13f-5893654576f5\") " pod="opendatahub/opendatahub-operator-controller-manager-668c7cd96d-6blck"
Sep 18 20:08:56.026888 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:08:56.026745    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/e064dbb8-6962-4c10-a13f-5893654576f5-manifests\") pod \"opendatahub-operator-controller-manager-668c7cd96d-6blck\" (UID: \"e064dbb8-6962-4c10-a13f-5893654576f5\") " pod="opendatahub/opendatahub-operator-controller-manager-668c7cd96d-6blck"
Sep 18 20:08:56.026888 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:08:56.026767    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/e064dbb8-6962-4c10-a13f-5893654576f5-apiservice-cert\") pod \"opendatahub-operator-controller-manager-668c7cd96d-6blck\" (UID: \"e064dbb8-6962-4c10-a13f-5893654576f5\") " pod="opendatahub/opendatahub-operator-controller-manager-668c7cd96d-6blck"
Sep 18 20:08:56.026888 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:08:56.026792    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e064dbb8-6962-4c10-a13f-5893654576f5-tmp\") pod \"opendatahub-operator-controller-manager-668c7cd96d-6blck\" (UID: \"e064dbb8-6962-4c10-a13f-5893654576f5\") " pod="opendatahub/opendatahub-operator-controller-manager-668c7cd96d-6blck"
Sep 18 20:08:56.026888 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:08:56.026808    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gsr26\" (UniqueName: \"kubernetes.io/projected/e064dbb8-6962-4c10-a13f-5893654576f5-kube-api-access-gsr26\") pod \"opendatahub-operator-controller-manager-668c7cd96d-6blck\" (UID: \"e064dbb8-6962-4c10-a13f-5893654576f5\") " pod="opendatahub/opendatahub-operator-controller-manager-668c7cd96d-6blck"
Sep 18 20:08:56.027260 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:08:56.027224    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/e064dbb8-6962-4c10-a13f-5893654576f5-manifests\") pod \"opendatahub-operator-controller-manager-668c7cd96d-6blck\" (UID: \"e064dbb8-6962-4c10-a13f-5893654576f5\") " pod="opendatahub/opendatahub-operator-controller-manager-668c7cd96d-6blck"
Sep 18 20:08:56.029159 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:08:56.029129    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e064dbb8-6962-4c10-a13f-5893654576f5-tmp\") pod \"opendatahub-operator-controller-manager-668c7cd96d-6blck\" (UID: \"e064dbb8-6962-4c10-a13f-5893654576f5\") " pod="opendatahub/opendatahub-operator-controller-manager-668c7cd96d-6blck"
Sep 18 20:08:56.029341 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:08:56.029319    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/e064dbb8-6962-4c10-a13f-5893654576f5-apiservice-cert\") pod \"opendatahub-operator-controller-manager-668c7cd96d-6blck\" (UID: \"e064dbb8-6962-4c10-a13f-5893654576f5\") " pod="opendatahub/opendatahub-operator-controller-manager-668c7cd96d-6blck"
Sep 18 20:08:56.029422 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:08:56.029403    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e064dbb8-6962-4c10-a13f-5893654576f5-webhook-cert\") pod \"opendatahub-operator-controller-manager-668c7cd96d-6blck\" (UID: \"e064dbb8-6962-4c10-a13f-5893654576f5\") " pod="opendatahub/opendatahub-operator-controller-manager-668c7cd96d-6blck"
Sep 18 20:08:56.036674 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:08:56.036652    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gsr26\" (UniqueName: \"kubernetes.io/projected/e064dbb8-6962-4c10-a13f-5893654576f5-kube-api-access-gsr26\") pod \"opendatahub-operator-controller-manager-668c7cd96d-6blck\" (UID: \"e064dbb8-6962-4c10-a13f-5893654576f5\") " pod="opendatahub/opendatahub-operator-controller-manager-668c7cd96d-6blck"
Sep 18 20:08:56.080046 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:08:56.079969    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:08:56.089321 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:08:56.089295    2532 generic.go:356] "Generic (PLEG): container finished" podID="75e08b59-588b-4e59-8c5d-47c290d91ab2" containerID="23cd3c74f25efcee0b7db0aa8109e61cf57d7ddbba45f038f1f7e535176ba5ca" exitCode=0
Sep 18 20:08:56.089469 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:08:56.089357    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0t5vv6" event={"ID":"75e08b59-588b-4e59-8c5d-47c290d91ab2","Type":"ContainerDied","Data":"23cd3c74f25efcee0b7db0aa8109e61cf57d7ddbba45f038f1f7e535176ba5ca"}
Sep 18 20:08:56.180867 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:08:56.180832    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:08:56.198304 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:08:56.198279    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-668c7cd96d-6blck"
Sep 18 20:08:56.280882 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:08:56.280851    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:08:56.381887 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:08:56.381814    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:08:56.461837 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:08:56.461798    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:08:56.480866 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:08:56.480810    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-668c7cd96d-6blck"]
Sep 18 20:08:56.482915 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:08:56.482879    2532 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode064dbb8_6962_4c10_a13f_5893654576f5.slice/crio-461740e13ee9cb15111244882efe6c00778ad75c93a51ed95bbb7d4f770e4001 WatchSource:0}: Error finding container 461740e13ee9cb15111244882efe6c00778ad75c93a51ed95bbb7d4f770e4001: Status 404 returned error can't find the container with id 461740e13ee9cb15111244882efe6c00778ad75c93a51ed95bbb7d4f770e4001
Sep 18 20:08:57.098951 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:08:57.098912    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-668c7cd96d-6blck" event={"ID":"e064dbb8-6962-4c10-a13f-5893654576f5","Type":"ContainerStarted","Data":"461740e13ee9cb15111244882efe6c00778ad75c93a51ed95bbb7d4f770e4001"}
Sep 18 20:08:57.101534 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:08:57.101502    2532 generic.go:356] "Generic (PLEG): container finished" podID="75e08b59-588b-4e59-8c5d-47c290d91ab2" containerID="b27e7437fcf5e86b7242de6088ae7a085e8dfca283bad2eeeaa65a6107aa9f5b" exitCode=0
Sep 18 20:08:57.101656 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:08:57.101597    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0t5vv6" event={"ID":"75e08b59-588b-4e59-8c5d-47c290d91ab2","Type":"ContainerDied","Data":"b27e7437fcf5e86b7242de6088ae7a085e8dfca283bad2eeeaa65a6107aa9f5b"}
Sep 18 20:08:59.495624 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:08:59.495595    2532 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0t5vv6"
Sep 18 20:08:59.557040 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:08:59.557015    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/75e08b59-588b-4e59-8c5d-47c290d91ab2-util\") pod \"75e08b59-588b-4e59-8c5d-47c290d91ab2\" (UID: \"75e08b59-588b-4e59-8c5d-47c290d91ab2\") "
Sep 18 20:08:59.557125 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:08:59.557077    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-44m6d\" (UniqueName: \"kubernetes.io/projected/75e08b59-588b-4e59-8c5d-47c290d91ab2-kube-api-access-44m6d\") pod \"75e08b59-588b-4e59-8c5d-47c290d91ab2\" (UID: \"75e08b59-588b-4e59-8c5d-47c290d91ab2\") "
Sep 18 20:08:59.557125 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:08:59.557108    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/75e08b59-588b-4e59-8c5d-47c290d91ab2-bundle\") pod \"75e08b59-588b-4e59-8c5d-47c290d91ab2\" (UID: \"75e08b59-588b-4e59-8c5d-47c290d91ab2\") "
Sep 18 20:08:59.557800 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:08:59.557778    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/75e08b59-588b-4e59-8c5d-47c290d91ab2-bundle" (OuterVolumeSpecName: "bundle") pod "75e08b59-588b-4e59-8c5d-47c290d91ab2" (UID: "75e08b59-588b-4e59-8c5d-47c290d91ab2"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 20:08:59.558992 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:08:59.558972    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/75e08b59-588b-4e59-8c5d-47c290d91ab2-kube-api-access-44m6d" (OuterVolumeSpecName: "kube-api-access-44m6d") pod "75e08b59-588b-4e59-8c5d-47c290d91ab2" (UID: "75e08b59-588b-4e59-8c5d-47c290d91ab2"). InnerVolumeSpecName "kube-api-access-44m6d". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 20:08:59.562709 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:08:59.562677    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/75e08b59-588b-4e59-8c5d-47c290d91ab2-util" (OuterVolumeSpecName: "util") pod "75e08b59-588b-4e59-8c5d-47c290d91ab2" (UID: "75e08b59-588b-4e59-8c5d-47c290d91ab2"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 20:08:59.657830 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:08:59.657793    2532 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-44m6d\" (UniqueName: \"kubernetes.io/projected/75e08b59-588b-4e59-8c5d-47c290d91ab2-kube-api-access-44m6d\") on node \"ip-10-0-130-81.ec2.internal\" DevicePath \"\""
Sep 18 20:08:59.657830 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:08:59.657824    2532 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/75e08b59-588b-4e59-8c5d-47c290d91ab2-bundle\") on node \"ip-10-0-130-81.ec2.internal\" DevicePath \"\""
Sep 18 20:08:59.657830 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:08:59.657833    2532 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/75e08b59-588b-4e59-8c5d-47c290d91ab2-util\") on node \"ip-10-0-130-81.ec2.internal\" DevicePath \"\""
Sep 18 20:09:00.123350 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:09:00.123308    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-668c7cd96d-6blck" event={"ID":"e064dbb8-6962-4c10-a13f-5893654576f5","Type":"ContainerStarted","Data":"ad0bb7b7c0d2348c6e465e6884546f3b1f4bda07960bb7e9554d9fbad70ca50b"}
Sep 18 20:09:00.125162 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:09:00.125137    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0t5vv6" event={"ID":"75e08b59-588b-4e59-8c5d-47c290d91ab2","Type":"ContainerDied","Data":"59f927cb93b69130575e6834d8fa85ce1960deeab50031e84a27b1e8d5cd1a4f"}
Sep 18 20:09:00.125256 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:09:00.125166    2532 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0t5vv6"
Sep 18 20:09:00.125256 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:09:00.125176    2532 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="59f927cb93b69130575e6834d8fa85ce1960deeab50031e84a27b1e8d5cd1a4f"
Sep 18 20:09:00.826278 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:09:00.826250    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:09:01.133865 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:09:01.133826    2532 generic.go:356] "Generic (PLEG): container finished" podID="e064dbb8-6962-4c10-a13f-5893654576f5" containerID="ad0bb7b7c0d2348c6e465e6884546f3b1f4bda07960bb7e9554d9fbad70ca50b" exitCode=0
Sep 18 20:09:01.134093 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:09:01.133879    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-668c7cd96d-6blck" event={"ID":"e064dbb8-6962-4c10-a13f-5893654576f5","Type":"ContainerDied","Data":"ad0bb7b7c0d2348c6e465e6884546f3b1f4bda07960bb7e9554d9fbad70ca50b"}
Sep 18 20:09:02.140717 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:09:02.140685    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-668c7cd96d-6blck" event={"ID":"e064dbb8-6962-4c10-a13f-5893654576f5","Type":"ContainerStarted","Data":"f71636233ee5e2b9f4b979390122b33737bfaa704a1db83ade4ef7017e4076f7"}
Sep 18 20:09:02.141091 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:09:02.140776    2532 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-668c7cd96d-6blck"
Sep 18 20:09:02.163610 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:09:02.163562    2532 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-668c7cd96d-6blck" podStartSLOduration=2.768196453 podStartE2EDuration="7.16354823s" totalImagesPullingTime="3.383833964s" totalInitContainerRuntime="1.011517813s" isStatefulPod=false podCreationTimestamp="2026-09-18 20:08:55 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 20:09:02.161171655 +0000 UTC m=+473.926561054" watchObservedRunningTime="2026-09-18 20:09:02.16354823 +0000 UTC m=+473.928937630"
Sep 18 20:09:06.623481 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:09:06.623448    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:09:06.720866 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:09:06.720836    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:09:13.148448 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:09:13.148417    2532 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-668c7cd96d-6blck"
Sep 18 20:09:15.459431 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:09:15.459398    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:09:15.588094 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:09:15.588062    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:09:15.782234 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:09:15.782147    2532 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-66b8956864-v2z9s"]
Sep 18 20:09:15.782650 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:09:15.782627    2532 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="75e08b59-588b-4e59-8c5d-47c290d91ab2" containerName="extract"
Sep 18 20:09:15.782650 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:09:15.782650    2532 state_mem.go:107] "Deleted CPUSet assignment" podUID="75e08b59-588b-4e59-8c5d-47c290d91ab2" containerName="extract"
Sep 18 20:09:15.782798 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:09:15.782665    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:09:15.782798 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:09:15.782691    2532 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="75e08b59-588b-4e59-8c5d-47c290d91ab2" containerName="util"
Sep 18 20:09:15.782798 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:09:15.782700    2532 state_mem.go:107] "Deleted CPUSet assignment" podUID="75e08b59-588b-4e59-8c5d-47c290d91ab2" containerName="util"
Sep 18 20:09:15.782798 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:09:15.782714    2532 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="75e08b59-588b-4e59-8c5d-47c290d91ab2" containerName="pull"
Sep 18 20:09:15.782798 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:09:15.782723    2532 state_mem.go:107] "Deleted CPUSet assignment" podUID="75e08b59-588b-4e59-8c5d-47c290d91ab2" containerName="pull"
Sep 18 20:09:15.782798 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:09:15.782790    2532 memory_manager.go:368] "RemoveStaleState removing state" podUID="75e08b59-588b-4e59-8c5d-47c290d91ab2" containerName="extract"
Sep 18 20:09:15.787926 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:09:15.787911    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-66b8956864-v2z9s"
Sep 18 20:09:15.792233 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:09:15.792211    2532 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Sep 18 20:09:15.792597 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:09:15.792579    2532 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Sep 18 20:09:15.792597 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:09:15.792590    2532 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Sep 18 20:09:15.792841 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:09:15.792823    2532 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Sep 18 20:09:15.806276 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:09:15.806254    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-66b8956864-v2z9s"]
Sep 18 20:09:15.885179 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:09:15.885109    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a26ec002-d403-4adf-b6c9-e8e74c7c812f-tmp\") pod \"kube-auth-proxy-66b8956864-v2z9s\" (UID: \"a26ec002-d403-4adf-b6c9-e8e74c7c812f\") " pod="openshift-ingress/kube-auth-proxy-66b8956864-v2z9s"
Sep 18 20:09:15.885179 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:09:15.885167    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6pl5b\" (UniqueName: \"kubernetes.io/projected/a26ec002-d403-4adf-b6c9-e8e74c7c812f-kube-api-access-6pl5b\") pod \"kube-auth-proxy-66b8956864-v2z9s\" (UID: \"a26ec002-d403-4adf-b6c9-e8e74c7c812f\") " pod="openshift-ingress/kube-auth-proxy-66b8956864-v2z9s"
Sep 18 20:09:15.885445 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:09:15.885264    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a26ec002-d403-4adf-b6c9-e8e74c7c812f-tls-certs\") pod \"kube-auth-proxy-66b8956864-v2z9s\" (UID: \"a26ec002-d403-4adf-b6c9-e8e74c7c812f\") " pod="openshift-ingress/kube-auth-proxy-66b8956864-v2z9s"
Sep 18 20:09:15.986310 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:09:15.986275    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a26ec002-d403-4adf-b6c9-e8e74c7c812f-tls-certs\") pod \"kube-auth-proxy-66b8956864-v2z9s\" (UID: \"a26ec002-d403-4adf-b6c9-e8e74c7c812f\") " pod="openshift-ingress/kube-auth-proxy-66b8956864-v2z9s"
Sep 18 20:09:15.986463 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:09:15.986340    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a26ec002-d403-4adf-b6c9-e8e74c7c812f-tmp\") pod \"kube-auth-proxy-66b8956864-v2z9s\" (UID: \"a26ec002-d403-4adf-b6c9-e8e74c7c812f\") " pod="openshift-ingress/kube-auth-proxy-66b8956864-v2z9s"
Sep 18 20:09:15.986463 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:09:15.986407    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6pl5b\" (UniqueName: \"kubernetes.io/projected/a26ec002-d403-4adf-b6c9-e8e74c7c812f-kube-api-access-6pl5b\") pod \"kube-auth-proxy-66b8956864-v2z9s\" (UID: \"a26ec002-d403-4adf-b6c9-e8e74c7c812f\") " pod="openshift-ingress/kube-auth-proxy-66b8956864-v2z9s"
Sep 18 20:09:15.986463 ip-10-0-130-81 kubenswrapper[2532]: E0918 20:09:15.986428    2532 secret.go:189] Couldn't get secret openshift-ingress/kube-auth-proxy-tls: secret "kube-auth-proxy-tls" not found
Sep 18 20:09:15.986668 ip-10-0-130-81 kubenswrapper[2532]: E0918 20:09:15.986536    2532 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a26ec002-d403-4adf-b6c9-e8e74c7c812f-tls-certs podName:a26ec002-d403-4adf-b6c9-e8e74c7c812f nodeName:}" failed. No retries permitted until 2026-09-18 20:09:16.48651244 +0000 UTC m=+488.251901834 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/a26ec002-d403-4adf-b6c9-e8e74c7c812f-tls-certs") pod "kube-auth-proxy-66b8956864-v2z9s" (UID: "a26ec002-d403-4adf-b6c9-e8e74c7c812f") : secret "kube-auth-proxy-tls" not found
Sep 18 20:09:15.988613 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:09:15.988594    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a26ec002-d403-4adf-b6c9-e8e74c7c812f-tmp\") pod \"kube-auth-proxy-66b8956864-v2z9s\" (UID: \"a26ec002-d403-4adf-b6c9-e8e74c7c812f\") " pod="openshift-ingress/kube-auth-proxy-66b8956864-v2z9s"
Sep 18 20:09:15.999475 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:09:15.999451    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6pl5b\" (UniqueName: \"kubernetes.io/projected/a26ec002-d403-4adf-b6c9-e8e74c7c812f-kube-api-access-6pl5b\") pod \"kube-auth-proxy-66b8956864-v2z9s\" (UID: \"a26ec002-d403-4adf-b6c9-e8e74c7c812f\") " pod="openshift-ingress/kube-auth-proxy-66b8956864-v2z9s"
Sep 18 20:09:16.082214 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:09:16.082139    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:09:16.490853 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:09:16.490809    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a26ec002-d403-4adf-b6c9-e8e74c7c812f-tls-certs\") pod \"kube-auth-proxy-66b8956864-v2z9s\" (UID: \"a26ec002-d403-4adf-b6c9-e8e74c7c812f\") " pod="openshift-ingress/kube-auth-proxy-66b8956864-v2z9s"
Sep 18 20:09:16.493323 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:09:16.493299    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a26ec002-d403-4adf-b6c9-e8e74c7c812f-tls-certs\") pod \"kube-auth-proxy-66b8956864-v2z9s\" (UID: \"a26ec002-d403-4adf-b6c9-e8e74c7c812f\") " pod="openshift-ingress/kube-auth-proxy-66b8956864-v2z9s"
Sep 18 20:09:16.697388 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:09:16.697354    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-66b8956864-v2z9s"
Sep 18 20:09:16.970969 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:09:16.970947    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-66b8956864-v2z9s"]
Sep 18 20:09:16.973238 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:09:16.973208    2532 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda26ec002_d403_4adf_b6c9_e8e74c7c812f.slice/crio-15f5d8e1dc85a8aef5c525b2331310a142a1eae03adcb7995edf1b7235476da7 WatchSource:0}: Error finding container 15f5d8e1dc85a8aef5c525b2331310a142a1eae03adcb7995edf1b7235476da7: Status 404 returned error can't find the container with id 15f5d8e1dc85a8aef5c525b2331310a142a1eae03adcb7995edf1b7235476da7
Sep 18 20:09:17.066581 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:09:17.066449    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:09:17.078985 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:09:17.078960    2532 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-vjvzh"]
Sep 18 20:09:17.081643 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:09:17.081626    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-vjvzh"
Sep 18 20:09:17.084162 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:09:17.084140    2532 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Sep 18 20:09:17.084265 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:09:17.084165    2532 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Sep 18 20:09:17.084265 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:09:17.084164    2532 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Sep 18 20:09:17.084265 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:09:17.084165    2532 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Sep 18 20:09:17.084654 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:09:17.084637    2532 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-zgmmj\""
Sep 18 20:09:17.095984 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:09:17.095962    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-vjvzh"]
Sep 18 20:09:17.127455 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:09:17.127429    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:09:17.197206 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:09:17.197172    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mfhpw\" (UniqueName: \"kubernetes.io/projected/fd5e3b65-98ce-4df6-b671-facce9b83cbe-kube-api-access-mfhpw\") pod \"istiod-openshift-gateway-7b7995d5cd-vjvzh\" (UID: \"fd5e3b65-98ce-4df6-b671-facce9b83cbe\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-vjvzh"
Sep 18 20:09:17.197369 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:09:17.197228    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/fd5e3b65-98ce-4df6-b671-facce9b83cbe-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-vjvzh\" (UID: \"fd5e3b65-98ce-4df6-b671-facce9b83cbe\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-vjvzh"
Sep 18 20:09:17.197369 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:09:17.197283    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/fd5e3b65-98ce-4df6-b671-facce9b83cbe-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-vjvzh\" (UID: \"fd5e3b65-98ce-4df6-b671-facce9b83cbe\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-vjvzh"
Sep 18 20:09:17.197369 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:09:17.197324    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/fd5e3b65-98ce-4df6-b671-facce9b83cbe-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-vjvzh\" (UID: \"fd5e3b65-98ce-4df6-b671-facce9b83cbe\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-vjvzh"
Sep 18 20:09:17.197525 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:09:17.197380    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/fd5e3b65-98ce-4df6-b671-facce9b83cbe-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-vjvzh\" (UID: \"fd5e3b65-98ce-4df6-b671-facce9b83cbe\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-vjvzh"
Sep 18 20:09:17.197525 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:09:17.197409    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/fd5e3b65-98ce-4df6-b671-facce9b83cbe-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-vjvzh\" (UID: \"fd5e3b65-98ce-4df6-b671-facce9b83cbe\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-vjvzh"
Sep 18 20:09:17.197525 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:09:17.197435    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fd5e3b65-98ce-4df6-b671-facce9b83cbe-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-vjvzh\" (UID: \"fd5e3b65-98ce-4df6-b671-facce9b83cbe\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-vjvzh"
Sep 18 20:09:17.207840 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:09:17.207815    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-66b8956864-v2z9s" event={"ID":"a26ec002-d403-4adf-b6c9-e8e74c7c812f","Type":"ContainerStarted","Data":"15f5d8e1dc85a8aef5c525b2331310a142a1eae03adcb7995edf1b7235476da7"}
Sep 18 20:09:17.298342 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:09:17.298303    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/fd5e3b65-98ce-4df6-b671-facce9b83cbe-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-vjvzh\" (UID: \"fd5e3b65-98ce-4df6-b671-facce9b83cbe\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-vjvzh"
Sep 18 20:09:17.298342 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:09:17.298345    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/fd5e3b65-98ce-4df6-b671-facce9b83cbe-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-vjvzh\" (UID: \"fd5e3b65-98ce-4df6-b671-facce9b83cbe\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-vjvzh"
Sep 18 20:09:17.298592 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:09:17.298389    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/fd5e3b65-98ce-4df6-b671-facce9b83cbe-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-vjvzh\" (UID: \"fd5e3b65-98ce-4df6-b671-facce9b83cbe\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-vjvzh"
Sep 18 20:09:17.298592 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:09:17.298406    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/fd5e3b65-98ce-4df6-b671-facce9b83cbe-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-vjvzh\" (UID: \"fd5e3b65-98ce-4df6-b671-facce9b83cbe\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-vjvzh"
Sep 18 20:09:17.298592 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:09:17.298539    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fd5e3b65-98ce-4df6-b671-facce9b83cbe-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-vjvzh\" (UID: \"fd5e3b65-98ce-4df6-b671-facce9b83cbe\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-vjvzh"
Sep 18 20:09:17.298741 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:09:17.298620    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mfhpw\" (UniqueName: \"kubernetes.io/projected/fd5e3b65-98ce-4df6-b671-facce9b83cbe-kube-api-access-mfhpw\") pod \"istiod-openshift-gateway-7b7995d5cd-vjvzh\" (UID: \"fd5e3b65-98ce-4df6-b671-facce9b83cbe\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-vjvzh"
Sep 18 20:09:17.298741 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:09:17.298659    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/fd5e3b65-98ce-4df6-b671-facce9b83cbe-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-vjvzh\" (UID: \"fd5e3b65-98ce-4df6-b671-facce9b83cbe\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-vjvzh"
Sep 18 20:09:17.299173 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:09:17.299142    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/fd5e3b65-98ce-4df6-b671-facce9b83cbe-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-vjvzh\" (UID: \"fd5e3b65-98ce-4df6-b671-facce9b83cbe\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-vjvzh"
Sep 18 20:09:17.300756 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:09:17.300729    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/fd5e3b65-98ce-4df6-b671-facce9b83cbe-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-vjvzh\" (UID: \"fd5e3b65-98ce-4df6-b671-facce9b83cbe\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-vjvzh"
Sep 18 20:09:17.300871 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:09:17.300854    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/fd5e3b65-98ce-4df6-b671-facce9b83cbe-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-vjvzh\" (UID: \"fd5e3b65-98ce-4df6-b671-facce9b83cbe\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-vjvzh"
Sep 18 20:09:17.301099 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:09:17.301083    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/fd5e3b65-98ce-4df6-b671-facce9b83cbe-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-vjvzh\" (UID: \"fd5e3b65-98ce-4df6-b671-facce9b83cbe\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-vjvzh"
Sep 18 20:09:17.301171 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:09:17.301152    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/fd5e3b65-98ce-4df6-b671-facce9b83cbe-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-vjvzh\" (UID: \"fd5e3b65-98ce-4df6-b671-facce9b83cbe\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-vjvzh"
Sep 18 20:09:17.312127 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:09:17.312102    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fd5e3b65-98ce-4df6-b671-facce9b83cbe-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-vjvzh\" (UID: \"fd5e3b65-98ce-4df6-b671-facce9b83cbe\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-vjvzh"
Sep 18 20:09:17.316896 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:09:17.316845    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mfhpw\" (UniqueName: \"kubernetes.io/projected/fd5e3b65-98ce-4df6-b671-facce9b83cbe-kube-api-access-mfhpw\") pod \"istiod-openshift-gateway-7b7995d5cd-vjvzh\" (UID: \"fd5e3b65-98ce-4df6-b671-facce9b83cbe\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-vjvzh"
Sep 18 20:09:17.390869 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:09:17.390839    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-vjvzh"
Sep 18 20:09:17.664570 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:09:17.664538    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-vjvzh"]
Sep 18 20:09:17.666754 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:09:17.666722    2532 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfd5e3b65_98ce_4df6_b671_facce9b83cbe.slice/crio-4b2f76d84a8803f64082ed7b2fd8fe072d0229568312f7c32ea582900f2c8ee5 WatchSource:0}: Error finding container 4b2f76d84a8803f64082ed7b2fd8fe072d0229568312f7c32ea582900f2c8ee5: Status 404 returned error can't find the container with id 4b2f76d84a8803f64082ed7b2fd8fe072d0229568312f7c32ea582900f2c8ee5
Sep 18 20:09:17.995386 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:09:17.995357    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:09:18.216142 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:09:18.216104    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-vjvzh" event={"ID":"fd5e3b65-98ce-4df6-b671-facce9b83cbe","Type":"ContainerStarted","Data":"4b2f76d84a8803f64082ed7b2fd8fe072d0229568312f7c32ea582900f2c8ee5"}
Sep 18 20:09:20.858791 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:09:20.858756    2532 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 18 20:09:20.859062 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:09:20.858827    2532 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 18 20:09:21.233098 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:09:21.233058    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-66b8956864-v2z9s" event={"ID":"a26ec002-d403-4adf-b6c9-e8e74c7c812f","Type":"ContainerStarted","Data":"3f1b07ceda1a1e2a49edbabae0ac23c318ae48e1822541c06348e60a5d4f7f34"}
Sep 18 20:09:21.234651 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:09:21.234626    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-vjvzh" event={"ID":"fd5e3b65-98ce-4df6-b671-facce9b83cbe","Type":"ContainerStarted","Data":"5af9667c94901c5c37cacd8f9fc57336b70c52e8cdc53d0a2f1c81d436b4d65d"}
Sep 18 20:09:21.234785 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:09:21.234669    2532 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-vjvzh"
Sep 18 20:09:21.236400 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:09:21.236376    2532 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7b7995d5cd-vjvzh container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 18 20:09:21.236504 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:09:21.236419    2532 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-vjvzh" podUID="fd5e3b65-98ce-4df6-b671-facce9b83cbe" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 18 20:09:21.250709 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:09:21.250665    2532 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-66b8956864-v2z9s" podStartSLOduration=2.374939318 podStartE2EDuration="6.250649963s" totalImagesPullingTime="3.875710645s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 20:09:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 20:09:21.250466128 +0000 UTC m=+493.015855528" watchObservedRunningTime="2026-09-18 20:09:21.250649963 +0000 UTC m=+493.016039361"
Sep 18 20:09:21.270273 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:09:21.270229    2532 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-vjvzh" podStartSLOduration=1.081244943 podStartE2EDuration="4.270216193s" totalImagesPullingTime="3.18897125s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 20:09:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 20:09:21.268448335 +0000 UTC m=+493.033837735" watchObservedRunningTime="2026-09-18 20:09:21.270216193 +0000 UTC m=+493.035605593"
Sep 18 20:09:21.465730 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:09:21.465691    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:09:21.528916 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:09:21.528806    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:09:22.067872 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:09:22.067840    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:09:22.239816 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:09:22.239783    2532 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-vjvzh"
Sep 18 20:09:22.251993 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:09:22.251968    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:09:22.369681 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:09:22.369607    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:09:26.511061 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:09:26.511033    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:09:26.569552 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:09:26.569520    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:09:26.670187 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:09:26.670153    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:09:26.999467 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:09:26.999431    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:09:27.182465 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:09:27.182433    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:09:27.538671 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:09:27.538635    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:09:34.789813 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:09:34.789778    2532 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-76d6fbff56-qvklf"]
Sep 18 20:09:34.799799 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:09:34.799767    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-76d6fbff56-qvklf"
Sep 18 20:09:34.801275 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:09:34.801237    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-76d6fbff56-qvklf"]
Sep 18 20:09:34.804264 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:09:34.804239    2532 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Sep 18 20:09:34.804406 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:09:34.804301    2532 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Sep 18 20:09:34.804406 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:09:34.804334    2532 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-bbght\""
Sep 18 20:09:34.804406 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:09:34.804239    2532 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Sep 18 20:09:34.804406 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:09:34.804377    2532 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Sep 18 20:09:34.804669 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:09:34.804656    2532 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Sep 18 20:09:34.950642 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:09:34.950608    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/b0c74281-fc9b-4303-b135-41059fa4e970-metrics-cert\") pod \"lws-controller-manager-76d6fbff56-qvklf\" (UID: \"b0c74281-fc9b-4303-b135-41059fa4e970\") " pod="openshift-lws-operator/lws-controller-manager-76d6fbff56-qvklf"
Sep 18 20:09:34.950642 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:09:34.950643    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bmj6w\" (UniqueName: \"kubernetes.io/projected/b0c74281-fc9b-4303-b135-41059fa4e970-kube-api-access-bmj6w\") pod \"lws-controller-manager-76d6fbff56-qvklf\" (UID: \"b0c74281-fc9b-4303-b135-41059fa4e970\") " pod="openshift-lws-operator/lws-controller-manager-76d6fbff56-qvklf"
Sep 18 20:09:34.950862 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:09:34.950678    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b0c74281-fc9b-4303-b135-41059fa4e970-cert\") pod \"lws-controller-manager-76d6fbff56-qvklf\" (UID: \"b0c74281-fc9b-4303-b135-41059fa4e970\") " pod="openshift-lws-operator/lws-controller-manager-76d6fbff56-qvklf"
Sep 18 20:09:34.950862 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:09:34.950733    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/b0c74281-fc9b-4303-b135-41059fa4e970-manager-config\") pod \"lws-controller-manager-76d6fbff56-qvklf\" (UID: \"b0c74281-fc9b-4303-b135-41059fa4e970\") " pod="openshift-lws-operator/lws-controller-manager-76d6fbff56-qvklf"
Sep 18 20:09:35.052188 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:09:35.052101    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/b0c74281-fc9b-4303-b135-41059fa4e970-manager-config\") pod \"lws-controller-manager-76d6fbff56-qvklf\" (UID: \"b0c74281-fc9b-4303-b135-41059fa4e970\") " pod="openshift-lws-operator/lws-controller-manager-76d6fbff56-qvklf"
Sep 18 20:09:35.052188 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:09:35.052167    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/b0c74281-fc9b-4303-b135-41059fa4e970-metrics-cert\") pod \"lws-controller-manager-76d6fbff56-qvklf\" (UID: \"b0c74281-fc9b-4303-b135-41059fa4e970\") " pod="openshift-lws-operator/lws-controller-manager-76d6fbff56-qvklf"
Sep 18 20:09:35.052188 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:09:35.052184    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bmj6w\" (UniqueName: \"kubernetes.io/projected/b0c74281-fc9b-4303-b135-41059fa4e970-kube-api-access-bmj6w\") pod \"lws-controller-manager-76d6fbff56-qvklf\" (UID: \"b0c74281-fc9b-4303-b135-41059fa4e970\") " pod="openshift-lws-operator/lws-controller-manager-76d6fbff56-qvklf"
Sep 18 20:09:35.052461 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:09:35.052219    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b0c74281-fc9b-4303-b135-41059fa4e970-cert\") pod \"lws-controller-manager-76d6fbff56-qvklf\" (UID: \"b0c74281-fc9b-4303-b135-41059fa4e970\") " pod="openshift-lws-operator/lws-controller-manager-76d6fbff56-qvklf"
Sep 18 20:09:35.052813 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:09:35.052788    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/b0c74281-fc9b-4303-b135-41059fa4e970-manager-config\") pod \"lws-controller-manager-76d6fbff56-qvklf\" (UID: \"b0c74281-fc9b-4303-b135-41059fa4e970\") " pod="openshift-lws-operator/lws-controller-manager-76d6fbff56-qvklf"
Sep 18 20:09:35.054770 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:09:35.054741    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/b0c74281-fc9b-4303-b135-41059fa4e970-metrics-cert\") pod \"lws-controller-manager-76d6fbff56-qvklf\" (UID: \"b0c74281-fc9b-4303-b135-41059fa4e970\") " pod="openshift-lws-operator/lws-controller-manager-76d6fbff56-qvklf"
Sep 18 20:09:35.054862 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:09:35.054783    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b0c74281-fc9b-4303-b135-41059fa4e970-cert\") pod \"lws-controller-manager-76d6fbff56-qvklf\" (UID: \"b0c74281-fc9b-4303-b135-41059fa4e970\") " pod="openshift-lws-operator/lws-controller-manager-76d6fbff56-qvklf"
Sep 18 20:09:35.073395 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:09:35.073372    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bmj6w\" (UniqueName: \"kubernetes.io/projected/b0c74281-fc9b-4303-b135-41059fa4e970-kube-api-access-bmj6w\") pod \"lws-controller-manager-76d6fbff56-qvklf\" (UID: \"b0c74281-fc9b-4303-b135-41059fa4e970\") " pod="openshift-lws-operator/lws-controller-manager-76d6fbff56-qvklf"
Sep 18 20:09:35.109974 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:09:35.109944    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-76d6fbff56-qvklf"
Sep 18 20:09:35.383690 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:09:35.383665    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-76d6fbff56-qvklf"]
Sep 18 20:09:35.386256 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:09:35.386230    2532 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb0c74281_fc9b_4303_b135_41059fa4e970.slice/crio-b363f08c048b686735c54ca2b52b185a38cf302b98b10f1f521eaf2dc58fdd77 WatchSource:0}: Error finding container b363f08c048b686735c54ca2b52b185a38cf302b98b10f1f521eaf2dc58fdd77: Status 404 returned error can't find the container with id b363f08c048b686735c54ca2b52b185a38cf302b98b10f1f521eaf2dc58fdd77
Sep 18 20:09:36.161451 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:09:36.161371    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:09:36.263042 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:09:36.262892    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:09:36.309452 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:09:36.309418    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-76d6fbff56-qvklf" event={"ID":"b0c74281-fc9b-4303-b135-41059fa4e970","Type":"ContainerStarted","Data":"b363f08c048b686735c54ca2b52b185a38cf302b98b10f1f521eaf2dc58fdd77"}
Sep 18 20:09:36.366962 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:09:36.366925    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:09:36.718476 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:09:36.718445    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:09:37.315875 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:09:37.315789    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-76d6fbff56-qvklf" event={"ID":"b0c74281-fc9b-4303-b135-41059fa4e970","Type":"ContainerStarted","Data":"1fcf2c91b2c04fe350e056a50b9db36f8a88b70742c83420014976dadd0d263d"}
Sep 18 20:09:37.316223 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:09:37.315949    2532 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-76d6fbff56-qvklf"
Sep 18 20:09:37.347118 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:09:37.347064    2532 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-76d6fbff56-qvklf" podStartSLOduration=1.7847539380000002 podStartE2EDuration="3.347048609s" totalImagesPullingTime="1.562294671s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 20:09:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 20:09:37.344809348 +0000 UTC m=+509.110198748" watchObservedRunningTime="2026-09-18 20:09:37.347048609 +0000 UTC m=+509.112438007"
Sep 18 20:09:37.473379 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:09:37.473340    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:09:37.567108 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:09:37.567031    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:09:37.666910 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:09:37.666879    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:09:38.147445 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:09:38.147408    2532 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-vjvzh"]
Sep 18 20:09:38.147656 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:09:38.147634    2532 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-vjvzh" podUID="fd5e3b65-98ce-4df6-b671-facce9b83cbe" containerName="discovery" containerID="cri-o://5af9667c94901c5c37cacd8f9fc57336b70c52e8cdc53d0a2f1c81d436b4d65d" gracePeriod=30
Sep 18 20:09:38.322777 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:09:38.322743    2532 generic.go:356] "Generic (PLEG): container finished" podID="fd5e3b65-98ce-4df6-b671-facce9b83cbe" containerID="5af9667c94901c5c37cacd8f9fc57336b70c52e8cdc53d0a2f1c81d436b4d65d" exitCode=0
Sep 18 20:09:38.323119 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:09:38.322806    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-vjvzh" event={"ID":"fd5e3b65-98ce-4df6-b671-facce9b83cbe","Type":"ContainerDied","Data":"5af9667c94901c5c37cacd8f9fc57336b70c52e8cdc53d0a2f1c81d436b4d65d"}
Sep 18 20:09:38.393785 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:09:38.393762    2532 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-vjvzh"
Sep 18 20:09:38.483369 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:09:38.483336    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fd5e3b65-98ce-4df6-b671-facce9b83cbe-istio-token\") pod \"fd5e3b65-98ce-4df6-b671-facce9b83cbe\" (UID: \"fd5e3b65-98ce-4df6-b671-facce9b83cbe\") "
Sep 18 20:09:38.483601 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:09:38.483383    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/fd5e3b65-98ce-4df6-b671-facce9b83cbe-local-certs\") pod \"fd5e3b65-98ce-4df6-b671-facce9b83cbe\" (UID: \"fd5e3b65-98ce-4df6-b671-facce9b83cbe\") "
Sep 18 20:09:38.483601 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:09:38.483412    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/fd5e3b65-98ce-4df6-b671-facce9b83cbe-cacerts\") pod \"fd5e3b65-98ce-4df6-b671-facce9b83cbe\" (UID: \"fd5e3b65-98ce-4df6-b671-facce9b83cbe\") "
Sep 18 20:09:38.483601 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:09:38.483437    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/fd5e3b65-98ce-4df6-b671-facce9b83cbe-istio-csr-dns-cert\") pod \"fd5e3b65-98ce-4df6-b671-facce9b83cbe\" (UID: \"fd5e3b65-98ce-4df6-b671-facce9b83cbe\") "
Sep 18 20:09:38.483601 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:09:38.483460    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mfhpw\" (UniqueName: \"kubernetes.io/projected/fd5e3b65-98ce-4df6-b671-facce9b83cbe-kube-api-access-mfhpw\") pod \"fd5e3b65-98ce-4df6-b671-facce9b83cbe\" (UID: \"fd5e3b65-98ce-4df6-b671-facce9b83cbe\") "
Sep 18 20:09:38.483601 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:09:38.483476    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/fd5e3b65-98ce-4df6-b671-facce9b83cbe-istio-kubeconfig\") pod \"fd5e3b65-98ce-4df6-b671-facce9b83cbe\" (UID: \"fd5e3b65-98ce-4df6-b671-facce9b83cbe\") "
Sep 18 20:09:38.483601 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:09:38.483547    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/fd5e3b65-98ce-4df6-b671-facce9b83cbe-istio-csr-ca-configmap\") pod \"fd5e3b65-98ce-4df6-b671-facce9b83cbe\" (UID: \"fd5e3b65-98ce-4df6-b671-facce9b83cbe\") "
Sep 18 20:09:38.484061 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:09:38.484029    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fd5e3b65-98ce-4df6-b671-facce9b83cbe-istio-csr-ca-configmap" (OuterVolumeSpecName: "istio-csr-ca-configmap") pod "fd5e3b65-98ce-4df6-b671-facce9b83cbe" (UID: "fd5e3b65-98ce-4df6-b671-facce9b83cbe"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 20:09:38.486099 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:09:38.486069    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fd5e3b65-98ce-4df6-b671-facce9b83cbe-local-certs" (OuterVolumeSpecName: "local-certs") pod "fd5e3b65-98ce-4df6-b671-facce9b83cbe" (UID: "fd5e3b65-98ce-4df6-b671-facce9b83cbe"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 20:09:38.486099 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:09:38.486071    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fd5e3b65-98ce-4df6-b671-facce9b83cbe-istio-csr-dns-cert" (OuterVolumeSpecName: "istio-csr-dns-cert") pod "fd5e3b65-98ce-4df6-b671-facce9b83cbe" (UID: "fd5e3b65-98ce-4df6-b671-facce9b83cbe"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 20:09:38.486265 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:09:38.486136    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fd5e3b65-98ce-4df6-b671-facce9b83cbe-istio-kubeconfig" (OuterVolumeSpecName: "istio-kubeconfig") pod "fd5e3b65-98ce-4df6-b671-facce9b83cbe" (UID: "fd5e3b65-98ce-4df6-b671-facce9b83cbe"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 20:09:38.486265 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:09:38.486157    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fd5e3b65-98ce-4df6-b671-facce9b83cbe-istio-token" (OuterVolumeSpecName: "istio-token") pod "fd5e3b65-98ce-4df6-b671-facce9b83cbe" (UID: "fd5e3b65-98ce-4df6-b671-facce9b83cbe"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 20:09:38.486375 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:09:38.486264    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fd5e3b65-98ce-4df6-b671-facce9b83cbe-kube-api-access-mfhpw" (OuterVolumeSpecName: "kube-api-access-mfhpw") pod "fd5e3b65-98ce-4df6-b671-facce9b83cbe" (UID: "fd5e3b65-98ce-4df6-b671-facce9b83cbe"). InnerVolumeSpecName "kube-api-access-mfhpw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 20:09:38.486509 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:09:38.486470    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fd5e3b65-98ce-4df6-b671-facce9b83cbe-cacerts" (OuterVolumeSpecName: "cacerts") pod "fd5e3b65-98ce-4df6-b671-facce9b83cbe" (UID: "fd5e3b65-98ce-4df6-b671-facce9b83cbe"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 20:09:38.584918 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:09:38.584882    2532 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/fd5e3b65-98ce-4df6-b671-facce9b83cbe-istio-csr-ca-configmap\") on node \"ip-10-0-130-81.ec2.internal\" DevicePath \"\""
Sep 18 20:09:38.584918 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:09:38.584912    2532 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fd5e3b65-98ce-4df6-b671-facce9b83cbe-istio-token\") on node \"ip-10-0-130-81.ec2.internal\" DevicePath \"\""
Sep 18 20:09:38.584918 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:09:38.584923    2532 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/fd5e3b65-98ce-4df6-b671-facce9b83cbe-local-certs\") on node \"ip-10-0-130-81.ec2.internal\" DevicePath \"\""
Sep 18 20:09:38.584918 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:09:38.584930    2532 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/fd5e3b65-98ce-4df6-b671-facce9b83cbe-cacerts\") on node \"ip-10-0-130-81.ec2.internal\" DevicePath \"\""
Sep 18 20:09:38.585173 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:09:38.584939    2532 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/fd5e3b65-98ce-4df6-b671-facce9b83cbe-istio-csr-dns-cert\") on node \"ip-10-0-130-81.ec2.internal\" DevicePath \"\""
Sep 18 20:09:38.585173 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:09:38.584948    2532 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mfhpw\" (UniqueName: \"kubernetes.io/projected/fd5e3b65-98ce-4df6-b671-facce9b83cbe-kube-api-access-mfhpw\") on node \"ip-10-0-130-81.ec2.internal\" DevicePath \"\""
Sep 18 20:09:38.585173 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:09:38.584956    2532 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/fd5e3b65-98ce-4df6-b671-facce9b83cbe-istio-kubeconfig\") on node \"ip-10-0-130-81.ec2.internal\" DevicePath \"\""
Sep 18 20:09:39.329528 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:09:39.329475    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-vjvzh" event={"ID":"fd5e3b65-98ce-4df6-b671-facce9b83cbe","Type":"ContainerDied","Data":"4b2f76d84a8803f64082ed7b2fd8fe072d0229568312f7c32ea582900f2c8ee5"}
Sep 18 20:09:39.329932 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:09:39.329538    2532 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-vjvzh"
Sep 18 20:09:39.329932 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:09:39.329544    2532 scope.go:117] "RemoveContainer" containerID="5af9667c94901c5c37cacd8f9fc57336b70c52e8cdc53d0a2f1c81d436b4d65d"
Sep 18 20:09:39.353199 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:09:39.353170    2532 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-vjvzh"]
Sep 18 20:09:39.362687 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:09:39.362642    2532 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-vjvzh"]
Sep 18 20:09:40.305551 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:09:40.305518    2532 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fd5e3b65-98ce-4df6-b671-facce9b83cbe" path="/var/lib/kubelet/pods/fd5e3b65-98ce-4df6-b671-facce9b83cbe/volumes"
Sep 18 20:09:48.325791 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:09:48.325755    2532 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-76d6fbff56-qvklf"
Sep 18 20:09:51.382326 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:09:51.382284    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:10:27.573695 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:10:27.573664    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:10:27.670864 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:10:27.670830    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:10:31.270419 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:10:31.270381    2532 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7p5s95"]
Sep 18 20:10:31.270817 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:10:31.270690    2532 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fd5e3b65-98ce-4df6-b671-facce9b83cbe" containerName="discovery"
Sep 18 20:10:31.270817 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:10:31.270700    2532 state_mem.go:107] "Deleted CPUSet assignment" podUID="fd5e3b65-98ce-4df6-b671-facce9b83cbe" containerName="discovery"
Sep 18 20:10:31.270817 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:10:31.270760    2532 memory_manager.go:368] "RemoveStaleState removing state" podUID="fd5e3b65-98ce-4df6-b671-facce9b83cbe" containerName="discovery"
Sep 18 20:10:31.273738 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:10:31.273721    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7p5s95"
Sep 18 20:10:31.276700 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:10:31.276672    2532 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-62r9t\""
Sep 18 20:10:31.276819 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:10:31.276723    2532 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Sep 18 20:10:31.277898 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:10:31.277862    2532 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Sep 18 20:10:31.283725 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:10:31.283701    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7p5s95"]
Sep 18 20:10:31.380230 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:10:31.380193    2532 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6qbglf"]
Sep 18 20:10:31.383409 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:10:31.383390    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6qbglf"
Sep 18 20:10:31.392848 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:10:31.392826    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6qbglf"]
Sep 18 20:10:31.427945 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:10:31.427913    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/afdf6281-6feb-413a-a24c-4e7036a664aa-bundle\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7p5s95\" (UID: \"afdf6281-6feb-413a-a24c-4e7036a664aa\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7p5s95"
Sep 18 20:10:31.428118 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:10:31.427959    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/afdf6281-6feb-413a-a24c-4e7036a664aa-util\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7p5s95\" (UID: \"afdf6281-6feb-413a-a24c-4e7036a664aa\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7p5s95"
Sep 18 20:10:31.428118 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:10:31.427982    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jbvxc\" (UniqueName: \"kubernetes.io/projected/afdf6281-6feb-413a-a24c-4e7036a664aa-kube-api-access-jbvxc\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7p5s95\" (UID: \"afdf6281-6feb-413a-a24c-4e7036a664aa\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7p5s95"
Sep 18 20:10:31.479847 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:10:31.479817    2532 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c7t654"]
Sep 18 20:10:31.483171 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:10:31.483149    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c7t654"
Sep 18 20:10:31.491134 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:10:31.491110    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c7t654"]
Sep 18 20:10:31.529473 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:10:31.529371    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/43022f68-06c7-4063-9106-bdc5575f6f8f-bundle\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6qbglf\" (UID: \"43022f68-06c7-4063-9106-bdc5575f6f8f\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6qbglf"
Sep 18 20:10:31.529473 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:10:31.529428    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/afdf6281-6feb-413a-a24c-4e7036a664aa-bundle\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7p5s95\" (UID: \"afdf6281-6feb-413a-a24c-4e7036a664aa\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7p5s95"
Sep 18 20:10:31.529473 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:10:31.529468    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/43022f68-06c7-4063-9106-bdc5575f6f8f-util\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6qbglf\" (UID: \"43022f68-06c7-4063-9106-bdc5575f6f8f\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6qbglf"
Sep 18 20:10:31.529772 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:10:31.529542    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/afdf6281-6feb-413a-a24c-4e7036a664aa-util\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7p5s95\" (UID: \"afdf6281-6feb-413a-a24c-4e7036a664aa\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7p5s95"
Sep 18 20:10:31.529772 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:10:31.529574    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jbvxc\" (UniqueName: \"kubernetes.io/projected/afdf6281-6feb-413a-a24c-4e7036a664aa-kube-api-access-jbvxc\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7p5s95\" (UID: \"afdf6281-6feb-413a-a24c-4e7036a664aa\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7p5s95"
Sep 18 20:10:31.529772 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:10:31.529610    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rjqgd\" (UniqueName: \"kubernetes.io/projected/43022f68-06c7-4063-9106-bdc5575f6f8f-kube-api-access-rjqgd\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6qbglf\" (UID: \"43022f68-06c7-4063-9106-bdc5575f6f8f\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6qbglf"
Sep 18 20:10:31.529921 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:10:31.529895    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/afdf6281-6feb-413a-a24c-4e7036a664aa-bundle\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7p5s95\" (UID: \"afdf6281-6feb-413a-a24c-4e7036a664aa\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7p5s95"
Sep 18 20:10:31.529957 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:10:31.529922    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/afdf6281-6feb-413a-a24c-4e7036a664aa-util\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7p5s95\" (UID: \"afdf6281-6feb-413a-a24c-4e7036a664aa\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7p5s95"
Sep 18 20:10:31.538294 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:10:31.538261    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jbvxc\" (UniqueName: \"kubernetes.io/projected/afdf6281-6feb-413a-a24c-4e7036a664aa-kube-api-access-jbvxc\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7p5s95\" (UID: \"afdf6281-6feb-413a-a24c-4e7036a664aa\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7p5s95"
Sep 18 20:10:31.573961 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:10:31.573926    2532 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847amxx8k"]
Sep 18 20:10:31.577526 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:10:31.577509    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847amxx8k"
Sep 18 20:10:31.583239 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:10:31.583203    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7p5s95"
Sep 18 20:10:31.585452 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:10:31.585424    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847amxx8k"]
Sep 18 20:10:31.630609 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:10:31.630570    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/43022f68-06c7-4063-9106-bdc5575f6f8f-bundle\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6qbglf\" (UID: \"43022f68-06c7-4063-9106-bdc5575f6f8f\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6qbglf"
Sep 18 20:10:31.630787 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:10:31.630621    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b7ad160d-df03-4024-8c5d-236858118f78-util\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c7t654\" (UID: \"b7ad160d-df03-4024-8c5d-236858118f78\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c7t654"
Sep 18 20:10:31.630787 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:10:31.630667    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/43022f68-06c7-4063-9106-bdc5575f6f8f-util\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6qbglf\" (UID: \"43022f68-06c7-4063-9106-bdc5575f6f8f\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6qbglf"
Sep 18 20:10:31.630787 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:10:31.630698    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b7ad160d-df03-4024-8c5d-236858118f78-bundle\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c7t654\" (UID: \"b7ad160d-df03-4024-8c5d-236858118f78\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c7t654"
Sep 18 20:10:31.630787 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:10:31.630764    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m59vr\" (UniqueName: \"kubernetes.io/projected/b7ad160d-df03-4024-8c5d-236858118f78-kube-api-access-m59vr\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c7t654\" (UID: \"b7ad160d-df03-4024-8c5d-236858118f78\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c7t654"
Sep 18 20:10:31.630941 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:10:31.630794    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rjqgd\" (UniqueName: \"kubernetes.io/projected/43022f68-06c7-4063-9106-bdc5575f6f8f-kube-api-access-rjqgd\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6qbglf\" (UID: \"43022f68-06c7-4063-9106-bdc5575f6f8f\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6qbglf"
Sep 18 20:10:31.631034 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:10:31.631008    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/43022f68-06c7-4063-9106-bdc5575f6f8f-bundle\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6qbglf\" (UID: \"43022f68-06c7-4063-9106-bdc5575f6f8f\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6qbglf"
Sep 18 20:10:31.631074 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:10:31.631020    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/43022f68-06c7-4063-9106-bdc5575f6f8f-util\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6qbglf\" (UID: \"43022f68-06c7-4063-9106-bdc5575f6f8f\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6qbglf"
Sep 18 20:10:31.645371 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:10:31.645337    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rjqgd\" (UniqueName: \"kubernetes.io/projected/43022f68-06c7-4063-9106-bdc5575f6f8f-kube-api-access-rjqgd\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6qbglf\" (UID: \"43022f68-06c7-4063-9106-bdc5575f6f8f\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6qbglf"
Sep 18 20:10:31.692628 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:10:31.692600    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6qbglf"
Sep 18 20:10:31.731989 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:10:31.731956    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1ab8829f-b8e3-462f-87a9-35f4bcc18ba4-bundle\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847amxx8k\" (UID: \"1ab8829f-b8e3-462f-87a9-35f4bcc18ba4\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847amxx8k"
Sep 18 20:10:31.732145 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:10:31.731999    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b7ad160d-df03-4024-8c5d-236858118f78-bundle\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c7t654\" (UID: \"b7ad160d-df03-4024-8c5d-236858118f78\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c7t654"
Sep 18 20:10:31.732145 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:10:31.732037    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m59vr\" (UniqueName: \"kubernetes.io/projected/b7ad160d-df03-4024-8c5d-236858118f78-kube-api-access-m59vr\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c7t654\" (UID: \"b7ad160d-df03-4024-8c5d-236858118f78\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c7t654"
Sep 18 20:10:31.732145 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:10:31.732094    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b7ad160d-df03-4024-8c5d-236858118f78-util\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c7t654\" (UID: \"b7ad160d-df03-4024-8c5d-236858118f78\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c7t654"
Sep 18 20:10:31.732145 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:10:31.732123    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1ab8829f-b8e3-462f-87a9-35f4bcc18ba4-util\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847amxx8k\" (UID: \"1ab8829f-b8e3-462f-87a9-35f4bcc18ba4\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847amxx8k"
Sep 18 20:10:31.732279 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:10:31.732143    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n29qh\" (UniqueName: \"kubernetes.io/projected/1ab8829f-b8e3-462f-87a9-35f4bcc18ba4-kube-api-access-n29qh\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847amxx8k\" (UID: \"1ab8829f-b8e3-462f-87a9-35f4bcc18ba4\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847amxx8k"
Sep 18 20:10:31.732340 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:10:31.732318    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b7ad160d-df03-4024-8c5d-236858118f78-bundle\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c7t654\" (UID: \"b7ad160d-df03-4024-8c5d-236858118f78\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c7t654"
Sep 18 20:10:31.732391 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:10:31.732378    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b7ad160d-df03-4024-8c5d-236858118f78-util\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c7t654\" (UID: \"b7ad160d-df03-4024-8c5d-236858118f78\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c7t654"
Sep 18 20:10:31.741906 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:10:31.741761    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m59vr\" (UniqueName: \"kubernetes.io/projected/b7ad160d-df03-4024-8c5d-236858118f78-kube-api-access-m59vr\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c7t654\" (UID: \"b7ad160d-df03-4024-8c5d-236858118f78\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c7t654"
Sep 18 20:10:31.792956 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:10:31.792890    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c7t654"
Sep 18 20:10:31.833441 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:10:31.833406    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1ab8829f-b8e3-462f-87a9-35f4bcc18ba4-util\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847amxx8k\" (UID: \"1ab8829f-b8e3-462f-87a9-35f4bcc18ba4\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847amxx8k"
Sep 18 20:10:31.833441 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:10:31.833444    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n29qh\" (UniqueName: \"kubernetes.io/projected/1ab8829f-b8e3-462f-87a9-35f4bcc18ba4-kube-api-access-n29qh\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847amxx8k\" (UID: \"1ab8829f-b8e3-462f-87a9-35f4bcc18ba4\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847amxx8k"
Sep 18 20:10:31.833703 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:10:31.833465    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1ab8829f-b8e3-462f-87a9-35f4bcc18ba4-bundle\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847amxx8k\" (UID: \"1ab8829f-b8e3-462f-87a9-35f4bcc18ba4\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847amxx8k"
Sep 18 20:10:31.833813 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:10:31.833793    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1ab8829f-b8e3-462f-87a9-35f4bcc18ba4-util\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847amxx8k\" (UID: \"1ab8829f-b8e3-462f-87a9-35f4bcc18ba4\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847amxx8k"
Sep 18 20:10:31.833875 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:10:31.833826    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1ab8829f-b8e3-462f-87a9-35f4bcc18ba4-bundle\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847amxx8k\" (UID: \"1ab8829f-b8e3-462f-87a9-35f4bcc18ba4\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847amxx8k"
Sep 18 20:10:31.844585 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:10:31.843711    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n29qh\" (UniqueName: \"kubernetes.io/projected/1ab8829f-b8e3-462f-87a9-35f4bcc18ba4-kube-api-access-n29qh\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847amxx8k\" (UID: \"1ab8829f-b8e3-462f-87a9-35f4bcc18ba4\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847amxx8k"
Sep 18 20:10:31.867073 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:10:31.866729    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7p5s95"]
Sep 18 20:10:31.869187 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:10:31.869163    2532 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podafdf6281_6feb_413a_a24c_4e7036a664aa.slice/crio-4dff3613cb00dc84b8ea28ff2155b3572110e2a92aac9d90dd552a3f6a7a8958 WatchSource:0}: Error finding container 4dff3613cb00dc84b8ea28ff2155b3572110e2a92aac9d90dd552a3f6a7a8958: Status 404 returned error can't find the container with id 4dff3613cb00dc84b8ea28ff2155b3572110e2a92aac9d90dd552a3f6a7a8958
Sep 18 20:10:31.892693 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:10:31.892673    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847amxx8k"
Sep 18 20:10:31.968749 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:10:31.968718    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6qbglf"]
Sep 18 20:10:31.971031 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:10:31.971004    2532 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod43022f68_06c7_4063_9106_bdc5575f6f8f.slice/crio-255fd9cb8211b4b588cf6d681bfd5bb611d5ecd81db43e3627ee59f326f16380 WatchSource:0}: Error finding container 255fd9cb8211b4b588cf6d681bfd5bb611d5ecd81db43e3627ee59f326f16380: Status 404 returned error can't find the container with id 255fd9cb8211b4b588cf6d681bfd5bb611d5ecd81db43e3627ee59f326f16380
Sep 18 20:10:32.062318 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:10:32.062297    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c7t654"]
Sep 18 20:10:32.064046 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:10:32.064021    2532 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb7ad160d_df03_4024_8c5d_236858118f78.slice/crio-98bf841ce3b0c3718062d4c8e2f3e55194dcdc5fdc1a14a01846d7a2a5a30b4d WatchSource:0}: Error finding container 98bf841ce3b0c3718062d4c8e2f3e55194dcdc5fdc1a14a01846d7a2a5a30b4d: Status 404 returned error can't find the container with id 98bf841ce3b0c3718062d4c8e2f3e55194dcdc5fdc1a14a01846d7a2a5a30b4d
Sep 18 20:10:32.163897 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:10:32.163766    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847amxx8k"]
Sep 18 20:10:32.174392 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:10:32.174363    2532 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1ab8829f_b8e3_462f_87a9_35f4bcc18ba4.slice/crio-6c16b5e3aa4ff585a9ae79f345cc7932a36cec21cdc7013ba375c9f8f90e41ba WatchSource:0}: Error finding container 6c16b5e3aa4ff585a9ae79f345cc7932a36cec21cdc7013ba375c9f8f90e41ba: Status 404 returned error can't find the container with id 6c16b5e3aa4ff585a9ae79f345cc7932a36cec21cdc7013ba375c9f8f90e41ba
Sep 18 20:10:32.571133 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:10:32.571053    2532 generic.go:356] "Generic (PLEG): container finished" podID="1ab8829f-b8e3-462f-87a9-35f4bcc18ba4" containerID="47ba4562691e991f0edf6b7f647f6ba466e5e15680854eac3812014e5fe8484a" exitCode=0
Sep 18 20:10:32.571581 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:10:32.571135    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847amxx8k" event={"ID":"1ab8829f-b8e3-462f-87a9-35f4bcc18ba4","Type":"ContainerDied","Data":"47ba4562691e991f0edf6b7f647f6ba466e5e15680854eac3812014e5fe8484a"}
Sep 18 20:10:32.571581 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:10:32.571189    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847amxx8k" event={"ID":"1ab8829f-b8e3-462f-87a9-35f4bcc18ba4","Type":"ContainerStarted","Data":"6c16b5e3aa4ff585a9ae79f345cc7932a36cec21cdc7013ba375c9f8f90e41ba"}
Sep 18 20:10:32.572980 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:10:32.572808    2532 generic.go:356] "Generic (PLEG): container finished" podID="b7ad160d-df03-4024-8c5d-236858118f78" containerID="60c05aadeb6da0cd04e44c6845b4065d1af86c42aa98f2c04ac670edb31abe5f" exitCode=0
Sep 18 20:10:32.572980 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:10:32.572897    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c7t654" event={"ID":"b7ad160d-df03-4024-8c5d-236858118f78","Type":"ContainerDied","Data":"60c05aadeb6da0cd04e44c6845b4065d1af86c42aa98f2c04ac670edb31abe5f"}
Sep 18 20:10:32.572980 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:10:32.572942    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c7t654" event={"ID":"b7ad160d-df03-4024-8c5d-236858118f78","Type":"ContainerStarted","Data":"98bf841ce3b0c3718062d4c8e2f3e55194dcdc5fdc1a14a01846d7a2a5a30b4d"}
Sep 18 20:10:32.574497 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:10:32.574464    2532 generic.go:356] "Generic (PLEG): container finished" podID="43022f68-06c7-4063-9106-bdc5575f6f8f" containerID="908bdc7905e93d3505ba5adf3c824c40a82a6f2865992174c58d73005a1f7b95" exitCode=0
Sep 18 20:10:32.574588 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:10:32.574506    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6qbglf" event={"ID":"43022f68-06c7-4063-9106-bdc5575f6f8f","Type":"ContainerDied","Data":"908bdc7905e93d3505ba5adf3c824c40a82a6f2865992174c58d73005a1f7b95"}
Sep 18 20:10:32.574588 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:10:32.574548    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6qbglf" event={"ID":"43022f68-06c7-4063-9106-bdc5575f6f8f","Type":"ContainerStarted","Data":"255fd9cb8211b4b588cf6d681bfd5bb611d5ecd81db43e3627ee59f326f16380"}
Sep 18 20:10:32.576276 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:10:32.576259    2532 generic.go:356] "Generic (PLEG): container finished" podID="afdf6281-6feb-413a-a24c-4e7036a664aa" containerID="9f816370ffe4d0e2f88888967808e651735898c5c8ba47a9524eab691a964b81" exitCode=0
Sep 18 20:10:32.576349 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:10:32.576284    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7p5s95" event={"ID":"afdf6281-6feb-413a-a24c-4e7036a664aa","Type":"ContainerDied","Data":"9f816370ffe4d0e2f88888967808e651735898c5c8ba47a9524eab691a964b81"}
Sep 18 20:10:32.576349 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:10:32.576308    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7p5s95" event={"ID":"afdf6281-6feb-413a-a24c-4e7036a664aa","Type":"ContainerStarted","Data":"4dff3613cb00dc84b8ea28ff2155b3572110e2a92aac9d90dd552a3f6a7a8958"}
Sep 18 20:10:34.589179 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:10:34.589146    2532 generic.go:356] "Generic (PLEG): container finished" podID="b7ad160d-df03-4024-8c5d-236858118f78" containerID="6f50106882164382ca3e1291f407cd742247c6e602feeb9f1c764326941c6ea3" exitCode=0
Sep 18 20:10:34.589633 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:10:34.589236    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c7t654" event={"ID":"b7ad160d-df03-4024-8c5d-236858118f78","Type":"ContainerDied","Data":"6f50106882164382ca3e1291f407cd742247c6e602feeb9f1c764326941c6ea3"}
Sep 18 20:10:34.591329 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:10:34.591306    2532 generic.go:356] "Generic (PLEG): container finished" podID="43022f68-06c7-4063-9106-bdc5575f6f8f" containerID="80526cf0440e577e97c0a7a106de4d7e9a6d8b86af7eaf0b057bdf8db6aa5332" exitCode=0
Sep 18 20:10:34.591438 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:10:34.591376    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6qbglf" event={"ID":"43022f68-06c7-4063-9106-bdc5575f6f8f","Type":"ContainerDied","Data":"80526cf0440e577e97c0a7a106de4d7e9a6d8b86af7eaf0b057bdf8db6aa5332"}
Sep 18 20:10:34.593551 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:10:34.593524    2532 generic.go:356] "Generic (PLEG): container finished" podID="afdf6281-6feb-413a-a24c-4e7036a664aa" containerID="7857a1750c397ea187e5f665385f9b1d213463792b3e0ecb60b349d539e3968a" exitCode=0
Sep 18 20:10:34.593651 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:10:34.593635    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7p5s95" event={"ID":"afdf6281-6feb-413a-a24c-4e7036a664aa","Type":"ContainerDied","Data":"7857a1750c397ea187e5f665385f9b1d213463792b3e0ecb60b349d539e3968a"}
Sep 18 20:10:34.596600 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:10:34.596574    2532 generic.go:356] "Generic (PLEG): container finished" podID="1ab8829f-b8e3-462f-87a9-35f4bcc18ba4" containerID="31f15090d58d1edc461ec1a26f6c841e406e648d82e9b09507b817d848944cbf" exitCode=0
Sep 18 20:10:34.596708 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:10:34.596657    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847amxx8k" event={"ID":"1ab8829f-b8e3-462f-87a9-35f4bcc18ba4","Type":"ContainerDied","Data":"31f15090d58d1edc461ec1a26f6c841e406e648d82e9b09507b817d848944cbf"}
Sep 18 20:10:35.603805 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:10:35.603771    2532 generic.go:356] "Generic (PLEG): container finished" podID="b7ad160d-df03-4024-8c5d-236858118f78" containerID="d0a09b469a9edaf879cbdeeea5d393e7b15199caf9d418379b94352dacb74406" exitCode=0
Sep 18 20:10:35.604222 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:10:35.603852    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c7t654" event={"ID":"b7ad160d-df03-4024-8c5d-236858118f78","Type":"ContainerDied","Data":"d0a09b469a9edaf879cbdeeea5d393e7b15199caf9d418379b94352dacb74406"}
Sep 18 20:10:35.605840 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:10:35.605820    2532 generic.go:356] "Generic (PLEG): container finished" podID="43022f68-06c7-4063-9106-bdc5575f6f8f" containerID="5a7a5e1faedc5cfc624a06b50787f515e08c1eec28eebfc77aede99909e316af" exitCode=0
Sep 18 20:10:35.605955 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:10:35.605845    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6qbglf" event={"ID":"43022f68-06c7-4063-9106-bdc5575f6f8f","Type":"ContainerDied","Data":"5a7a5e1faedc5cfc624a06b50787f515e08c1eec28eebfc77aede99909e316af"}
Sep 18 20:10:35.607806 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:10:35.607783    2532 generic.go:356] "Generic (PLEG): container finished" podID="afdf6281-6feb-413a-a24c-4e7036a664aa" containerID="3c2105f901b7cb24a392a3bb205f8cb464f8c06905e8957d5316fa1a2df7c8e4" exitCode=0
Sep 18 20:10:35.607888 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:10:35.607867    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7p5s95" event={"ID":"afdf6281-6feb-413a-a24c-4e7036a664aa","Type":"ContainerDied","Data":"3c2105f901b7cb24a392a3bb205f8cb464f8c06905e8957d5316fa1a2df7c8e4"}
Sep 18 20:10:35.609790 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:10:35.609772    2532 generic.go:356] "Generic (PLEG): container finished" podID="1ab8829f-b8e3-462f-87a9-35f4bcc18ba4" containerID="3abab26288737bcd5c80d2a111d0bad8e46ba868ddbe7142df22ca4a00a78501" exitCode=0
Sep 18 20:10:35.609885 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:10:35.609807    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847amxx8k" event={"ID":"1ab8829f-b8e3-462f-87a9-35f4bcc18ba4","Type":"ContainerDied","Data":"3abab26288737bcd5c80d2a111d0bad8e46ba868ddbe7142df22ca4a00a78501"}
Sep 18 20:10:36.801100 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:10:36.801067    2532 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c7t654"
Sep 18 20:10:36.804008 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:10:36.803988    2532 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7p5s95"
Sep 18 20:10:36.822409 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:10:36.822386    2532 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6qbglf"
Sep 18 20:10:36.825587 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:10:36.825568    2532 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847amxx8k"
Sep 18 20:10:36.875561 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:10:36.875455    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1ab8829f-b8e3-462f-87a9-35f4bcc18ba4-bundle\") pod \"1ab8829f-b8e3-462f-87a9-35f4bcc18ba4\" (UID: \"1ab8829f-b8e3-462f-87a9-35f4bcc18ba4\") "
Sep 18 20:10:36.875561 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:10:36.875525    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-m59vr\" (UniqueName: \"kubernetes.io/projected/b7ad160d-df03-4024-8c5d-236858118f78-kube-api-access-m59vr\") pod \"b7ad160d-df03-4024-8c5d-236858118f78\" (UID: \"b7ad160d-df03-4024-8c5d-236858118f78\") "
Sep 18 20:10:36.875561 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:10:36.875544    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1ab8829f-b8e3-462f-87a9-35f4bcc18ba4-util\") pod \"1ab8829f-b8e3-462f-87a9-35f4bcc18ba4\" (UID: \"1ab8829f-b8e3-462f-87a9-35f4bcc18ba4\") "
Sep 18 20:10:36.875561 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:10:36.875560    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b7ad160d-df03-4024-8c5d-236858118f78-bundle\") pod \"b7ad160d-df03-4024-8c5d-236858118f78\" (UID: \"b7ad160d-df03-4024-8c5d-236858118f78\") "
Sep 18 20:10:36.875884 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:10:36.875582    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/afdf6281-6feb-413a-a24c-4e7036a664aa-util\") pod \"afdf6281-6feb-413a-a24c-4e7036a664aa\" (UID: \"afdf6281-6feb-413a-a24c-4e7036a664aa\") "
Sep 18 20:10:36.875884 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:10:36.875598    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-n29qh\" (UniqueName: \"kubernetes.io/projected/1ab8829f-b8e3-462f-87a9-35f4bcc18ba4-kube-api-access-n29qh\") pod \"1ab8829f-b8e3-462f-87a9-35f4bcc18ba4\" (UID: \"1ab8829f-b8e3-462f-87a9-35f4bcc18ba4\") "
Sep 18 20:10:36.875884 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:10:36.875628    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/43022f68-06c7-4063-9106-bdc5575f6f8f-util\") pod \"43022f68-06c7-4063-9106-bdc5575f6f8f\" (UID: \"43022f68-06c7-4063-9106-bdc5575f6f8f\") "
Sep 18 20:10:36.875884 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:10:36.875817    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/43022f68-06c7-4063-9106-bdc5575f6f8f-bundle\") pod \"43022f68-06c7-4063-9106-bdc5575f6f8f\" (UID: \"43022f68-06c7-4063-9106-bdc5575f6f8f\") "
Sep 18 20:10:36.876082 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:10:36.875903    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rjqgd\" (UniqueName: \"kubernetes.io/projected/43022f68-06c7-4063-9106-bdc5575f6f8f-kube-api-access-rjqgd\") pod \"43022f68-06c7-4063-9106-bdc5575f6f8f\" (UID: \"43022f68-06c7-4063-9106-bdc5575f6f8f\") "
Sep 18 20:10:36.876082 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:10:36.875935    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b7ad160d-df03-4024-8c5d-236858118f78-util\") pod \"b7ad160d-df03-4024-8c5d-236858118f78\" (UID: \"b7ad160d-df03-4024-8c5d-236858118f78\") "
Sep 18 20:10:36.876384 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:10:36.876357    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b7ad160d-df03-4024-8c5d-236858118f78-bundle" (OuterVolumeSpecName: "bundle") pod "b7ad160d-df03-4024-8c5d-236858118f78" (UID: "b7ad160d-df03-4024-8c5d-236858118f78"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 20:10:36.876913 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:10:36.876888    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/43022f68-06c7-4063-9106-bdc5575f6f8f-bundle" (OuterVolumeSpecName: "bundle") pod "43022f68-06c7-4063-9106-bdc5575f6f8f" (UID: "43022f68-06c7-4063-9106-bdc5575f6f8f"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 20:10:36.876997 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:10:36.876946    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1ab8829f-b8e3-462f-87a9-35f4bcc18ba4-bundle" (OuterVolumeSpecName: "bundle") pod "1ab8829f-b8e3-462f-87a9-35f4bcc18ba4" (UID: "1ab8829f-b8e3-462f-87a9-35f4bcc18ba4"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 20:10:36.879201 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:10:36.879154    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1ab8829f-b8e3-462f-87a9-35f4bcc18ba4-kube-api-access-n29qh" (OuterVolumeSpecName: "kube-api-access-n29qh") pod "1ab8829f-b8e3-462f-87a9-35f4bcc18ba4" (UID: "1ab8829f-b8e3-462f-87a9-35f4bcc18ba4"). InnerVolumeSpecName "kube-api-access-n29qh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 20:10:36.879514 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:10:36.879384    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/43022f68-06c7-4063-9106-bdc5575f6f8f-kube-api-access-rjqgd" (OuterVolumeSpecName: "kube-api-access-rjqgd") pod "43022f68-06c7-4063-9106-bdc5575f6f8f" (UID: "43022f68-06c7-4063-9106-bdc5575f6f8f"). InnerVolumeSpecName "kube-api-access-rjqgd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 20:10:36.879514 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:10:36.879408    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b7ad160d-df03-4024-8c5d-236858118f78-kube-api-access-m59vr" (OuterVolumeSpecName: "kube-api-access-m59vr") pod "b7ad160d-df03-4024-8c5d-236858118f78" (UID: "b7ad160d-df03-4024-8c5d-236858118f78"). InnerVolumeSpecName "kube-api-access-m59vr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 20:10:36.884194 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:10:36.884165    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1ab8829f-b8e3-462f-87a9-35f4bcc18ba4-util" (OuterVolumeSpecName: "util") pod "1ab8829f-b8e3-462f-87a9-35f4bcc18ba4" (UID: "1ab8829f-b8e3-462f-87a9-35f4bcc18ba4"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 20:10:36.884298 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:10:36.884279    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b7ad160d-df03-4024-8c5d-236858118f78-util" (OuterVolumeSpecName: "util") pod "b7ad160d-df03-4024-8c5d-236858118f78" (UID: "b7ad160d-df03-4024-8c5d-236858118f78"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 20:10:36.885415 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:10:36.885372    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/43022f68-06c7-4063-9106-bdc5575f6f8f-util" (OuterVolumeSpecName: "util") pod "43022f68-06c7-4063-9106-bdc5575f6f8f" (UID: "43022f68-06c7-4063-9106-bdc5575f6f8f"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 20:10:36.885540 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:10:36.885525    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/afdf6281-6feb-413a-a24c-4e7036a664aa-util" (OuterVolumeSpecName: "util") pod "afdf6281-6feb-413a-a24c-4e7036a664aa" (UID: "afdf6281-6feb-413a-a24c-4e7036a664aa"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 20:10:36.977121 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:10:36.977088    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/afdf6281-6feb-413a-a24c-4e7036a664aa-bundle\") pod \"afdf6281-6feb-413a-a24c-4e7036a664aa\" (UID: \"afdf6281-6feb-413a-a24c-4e7036a664aa\") "
Sep 18 20:10:36.977276 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:10:36.977139    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jbvxc\" (UniqueName: \"kubernetes.io/projected/afdf6281-6feb-413a-a24c-4e7036a664aa-kube-api-access-jbvxc\") pod \"afdf6281-6feb-413a-a24c-4e7036a664aa\" (UID: \"afdf6281-6feb-413a-a24c-4e7036a664aa\") "
Sep 18 20:10:36.977276 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:10:36.977258    2532 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m59vr\" (UniqueName: \"kubernetes.io/projected/b7ad160d-df03-4024-8c5d-236858118f78-kube-api-access-m59vr\") on node \"ip-10-0-130-81.ec2.internal\" DevicePath \"\""
Sep 18 20:10:36.977276 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:10:36.977269    2532 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1ab8829f-b8e3-462f-87a9-35f4bcc18ba4-util\") on node \"ip-10-0-130-81.ec2.internal\" DevicePath \"\""
Sep 18 20:10:36.977397 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:10:36.977278    2532 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b7ad160d-df03-4024-8c5d-236858118f78-bundle\") on node \"ip-10-0-130-81.ec2.internal\" DevicePath \"\""
Sep 18 20:10:36.977397 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:10:36.977286    2532 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/afdf6281-6feb-413a-a24c-4e7036a664aa-util\") on node \"ip-10-0-130-81.ec2.internal\" DevicePath \"\""
Sep 18 20:10:36.977397 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:10:36.977293    2532 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n29qh\" (UniqueName: \"kubernetes.io/projected/1ab8829f-b8e3-462f-87a9-35f4bcc18ba4-kube-api-access-n29qh\") on node \"ip-10-0-130-81.ec2.internal\" DevicePath \"\""
Sep 18 20:10:36.977397 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:10:36.977302    2532 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/43022f68-06c7-4063-9106-bdc5575f6f8f-util\") on node \"ip-10-0-130-81.ec2.internal\" DevicePath \"\""
Sep 18 20:10:36.977397 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:10:36.977312    2532 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/43022f68-06c7-4063-9106-bdc5575f6f8f-bundle\") on node \"ip-10-0-130-81.ec2.internal\" DevicePath \"\""
Sep 18 20:10:36.977397 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:10:36.977319    2532 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rjqgd\" (UniqueName: \"kubernetes.io/projected/43022f68-06c7-4063-9106-bdc5575f6f8f-kube-api-access-rjqgd\") on node \"ip-10-0-130-81.ec2.internal\" DevicePath \"\""
Sep 18 20:10:36.977397 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:10:36.977327    2532 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b7ad160d-df03-4024-8c5d-236858118f78-util\") on node \"ip-10-0-130-81.ec2.internal\" DevicePath \"\""
Sep 18 20:10:36.977397 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:10:36.977335    2532 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1ab8829f-b8e3-462f-87a9-35f4bcc18ba4-bundle\") on node \"ip-10-0-130-81.ec2.internal\" DevicePath \"\""
Sep 18 20:10:36.977704 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:10:36.977680    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/afdf6281-6feb-413a-a24c-4e7036a664aa-bundle" (OuterVolumeSpecName: "bundle") pod "afdf6281-6feb-413a-a24c-4e7036a664aa" (UID: "afdf6281-6feb-413a-a24c-4e7036a664aa"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 20:10:36.979238 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:10:36.979218    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/afdf6281-6feb-413a-a24c-4e7036a664aa-kube-api-access-jbvxc" (OuterVolumeSpecName: "kube-api-access-jbvxc") pod "afdf6281-6feb-413a-a24c-4e7036a664aa" (UID: "afdf6281-6feb-413a-a24c-4e7036a664aa"). InnerVolumeSpecName "kube-api-access-jbvxc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 20:10:37.078584 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:10:37.078548    2532 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jbvxc\" (UniqueName: \"kubernetes.io/projected/afdf6281-6feb-413a-a24c-4e7036a664aa-kube-api-access-jbvxc\") on node \"ip-10-0-130-81.ec2.internal\" DevicePath \"\""
Sep 18 20:10:37.078584 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:10:37.078580    2532 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/afdf6281-6feb-413a-a24c-4e7036a664aa-bundle\") on node \"ip-10-0-130-81.ec2.internal\" DevicePath \"\""
Sep 18 20:10:37.622127 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:10:37.622099    2532 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847amxx8k"
Sep 18 20:10:37.622127 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:10:37.622104    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847amxx8k" event={"ID":"1ab8829f-b8e3-462f-87a9-35f4bcc18ba4","Type":"ContainerDied","Data":"6c16b5e3aa4ff585a9ae79f345cc7932a36cec21cdc7013ba375c9f8f90e41ba"}
Sep 18 20:10:37.622332 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:10:37.622153    2532 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6c16b5e3aa4ff585a9ae79f345cc7932a36cec21cdc7013ba375c9f8f90e41ba"
Sep 18 20:10:37.624111 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:10:37.624094    2532 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c7t654"
Sep 18 20:10:37.624248 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:10:37.624122    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c7t654" event={"ID":"b7ad160d-df03-4024-8c5d-236858118f78","Type":"ContainerDied","Data":"98bf841ce3b0c3718062d4c8e2f3e55194dcdc5fdc1a14a01846d7a2a5a30b4d"}
Sep 18 20:10:37.624248 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:10:37.624157    2532 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="98bf841ce3b0c3718062d4c8e2f3e55194dcdc5fdc1a14a01846d7a2a5a30b4d"
Sep 18 20:10:37.626130 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:10:37.626110    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6qbglf" event={"ID":"43022f68-06c7-4063-9106-bdc5575f6f8f","Type":"ContainerDied","Data":"255fd9cb8211b4b588cf6d681bfd5bb611d5ecd81db43e3627ee59f326f16380"}
Sep 18 20:10:37.626236 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:10:37.626131    2532 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6qbglf"
Sep 18 20:10:37.626236 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:10:37.626139    2532 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="255fd9cb8211b4b588cf6d681bfd5bb611d5ecd81db43e3627ee59f326f16380"
Sep 18 20:10:37.628166 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:10:37.628150    2532 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7p5s95"
Sep 18 20:10:37.628264 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:10:37.628152    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7p5s95" event={"ID":"afdf6281-6feb-413a-a24c-4e7036a664aa","Type":"ContainerDied","Data":"4dff3613cb00dc84b8ea28ff2155b3572110e2a92aac9d90dd552a3f6a7a8958"}
Sep 18 20:10:37.628315 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:10:37.628281    2532 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4dff3613cb00dc84b8ea28ff2155b3572110e2a92aac9d90dd552a3f6a7a8958"
Sep 18 20:10:40.946614 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:10:40.946585    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:10:41.091458 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:10:41.091428    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:10:41.176324 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:10:41.176291    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:10:41.276465 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:10:41.276436    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:10:41.377364 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:10:41.377334    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:10:41.745744 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:10:41.745711    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:10:41.843608 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:10:41.843574    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:10:43.208501 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:10:43.208460    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:10:43.258831 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:10:43.258804    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:10:44.754003 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:10:44.753970    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:10:44.986802 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:10:44.986770    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:10:45.147820 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:10:45.147737    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:10:45.762577 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:10:45.762539    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:10:47.428907 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:10:47.428876    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:10:49.155603 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:10:49.155570    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:10:50.186280 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:10:50.186206    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:10:51.373605 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:10:51.373567    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:10:51.542263 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:10:51.542229    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:10:51.736463 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:10:51.736427    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:10:52.060343 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:10:52.060269    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:10:52.166802 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:10:52.166769    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:10:52.804141 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:10:52.804099    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:10:54.355372 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:10:54.355341    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:10:54.456124 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:10:54.456088    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:10:54.820525 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:10:54.820469    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:10:54.884787 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:10:54.884754    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:10:56.177088 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:10:56.177057    2532 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-fbdd4f445-hnxk4"]
Sep 18 20:10:56.177594 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:10:56.177435    2532 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1ab8829f-b8e3-462f-87a9-35f4bcc18ba4" containerName="pull"
Sep 18 20:10:56.177594 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:10:56.177451    2532 state_mem.go:107] "Deleted CPUSet assignment" podUID="1ab8829f-b8e3-462f-87a9-35f4bcc18ba4" containerName="pull"
Sep 18 20:10:56.177594 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:10:56.177460    2532 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="afdf6281-6feb-413a-a24c-4e7036a664aa" containerName="util"
Sep 18 20:10:56.177594 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:10:56.177465    2532 state_mem.go:107] "Deleted CPUSet assignment" podUID="afdf6281-6feb-413a-a24c-4e7036a664aa" containerName="util"
Sep 18 20:10:56.177594 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:10:56.177474    2532 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="43022f68-06c7-4063-9106-bdc5575f6f8f" containerName="extract"
Sep 18 20:10:56.177594 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:10:56.177494    2532 state_mem.go:107] "Deleted CPUSet assignment" podUID="43022f68-06c7-4063-9106-bdc5575f6f8f" containerName="extract"
Sep 18 20:10:56.177594 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:10:56.177509    2532 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="43022f68-06c7-4063-9106-bdc5575f6f8f" containerName="util"
Sep 18 20:10:56.177594 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:10:56.177514    2532 state_mem.go:107] "Deleted CPUSet assignment" podUID="43022f68-06c7-4063-9106-bdc5575f6f8f" containerName="util"
Sep 18 20:10:56.177594 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:10:56.177520    2532 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1ab8829f-b8e3-462f-87a9-35f4bcc18ba4" containerName="util"
Sep 18 20:10:56.177594 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:10:56.177525    2532 state_mem.go:107] "Deleted CPUSet assignment" podUID="1ab8829f-b8e3-462f-87a9-35f4bcc18ba4" containerName="util"
Sep 18 20:10:56.177594 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:10:56.177531    2532 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b7ad160d-df03-4024-8c5d-236858118f78" containerName="extract"
Sep 18 20:10:56.177594 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:10:56.177536    2532 state_mem.go:107] "Deleted CPUSet assignment" podUID="b7ad160d-df03-4024-8c5d-236858118f78" containerName="extract"
Sep 18 20:10:56.177594 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:10:56.177543    2532 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="afdf6281-6feb-413a-a24c-4e7036a664aa" containerName="pull"
Sep 18 20:10:56.177594 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:10:56.177549    2532 state_mem.go:107] "Deleted CPUSet assignment" podUID="afdf6281-6feb-413a-a24c-4e7036a664aa" containerName="pull"
Sep 18 20:10:56.177594 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:10:56.177554    2532 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b7ad160d-df03-4024-8c5d-236858118f78" containerName="pull"
Sep 18 20:10:56.177594 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:10:56.177562    2532 state_mem.go:107] "Deleted CPUSet assignment" podUID="b7ad160d-df03-4024-8c5d-236858118f78" containerName="pull"
Sep 18 20:10:56.177594 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:10:56.177570    2532 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="afdf6281-6feb-413a-a24c-4e7036a664aa" containerName="extract"
Sep 18 20:10:56.177594 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:10:56.177578    2532 state_mem.go:107] "Deleted CPUSet assignment" podUID="afdf6281-6feb-413a-a24c-4e7036a664aa" containerName="extract"
Sep 18 20:10:56.177594 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:10:56.177591    2532 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b7ad160d-df03-4024-8c5d-236858118f78" containerName="util"
Sep 18 20:10:56.177594 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:10:56.177595    2532 state_mem.go:107] "Deleted CPUSet assignment" podUID="b7ad160d-df03-4024-8c5d-236858118f78" containerName="util"
Sep 18 20:10:56.177594 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:10:56.177601    2532 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="43022f68-06c7-4063-9106-bdc5575f6f8f" containerName="pull"
Sep 18 20:10:56.177594 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:10:56.177606    2532 state_mem.go:107] "Deleted CPUSet assignment" podUID="43022f68-06c7-4063-9106-bdc5575f6f8f" containerName="pull"
Sep 18 20:10:56.178607 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:10:56.177612    2532 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1ab8829f-b8e3-462f-87a9-35f4bcc18ba4" containerName="extract"
Sep 18 20:10:56.178607 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:10:56.177618    2532 state_mem.go:107] "Deleted CPUSet assignment" podUID="1ab8829f-b8e3-462f-87a9-35f4bcc18ba4" containerName="extract"
Sep 18 20:10:56.178607 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:10:56.177684    2532 memory_manager.go:368] "RemoveStaleState removing state" podUID="43022f68-06c7-4063-9106-bdc5575f6f8f" containerName="extract"
Sep 18 20:10:56.178607 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:10:56.177692    2532 memory_manager.go:368] "RemoveStaleState removing state" podUID="1ab8829f-b8e3-462f-87a9-35f4bcc18ba4" containerName="extract"
Sep 18 20:10:56.178607 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:10:56.177699    2532 memory_manager.go:368] "RemoveStaleState removing state" podUID="afdf6281-6feb-413a-a24c-4e7036a664aa" containerName="extract"
Sep 18 20:10:56.178607 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:10:56.177704    2532 memory_manager.go:368] "RemoveStaleState removing state" podUID="b7ad160d-df03-4024-8c5d-236858118f78" containerName="extract"
Sep 18 20:10:56.179097 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:10:56.179077    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:10:56.180669 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:10:56.180654    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-hnxk4"
Sep 18 20:10:56.183248 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:10:56.183227    2532 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Sep 18 20:10:56.183370 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:10:56.183270    2532 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\""
Sep 18 20:10:56.184295 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:10:56.184279    2532 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\""
Sep 18 20:10:56.184406 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:10:56.184282    2532 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-75d4r\""
Sep 18 20:10:56.184406 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:10:56.184281    2532 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Sep 18 20:10:56.189551 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:10:56.189519    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-fbdd4f445-hnxk4"]
Sep 18 20:10:56.248254 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:10:56.248210    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/8d1b6ff3-8023-4e34-9e10-2bcc51ef5468-plugin-serving-cert\") pod \"kuadrant-console-plugin-fbdd4f445-hnxk4\" (UID: \"8d1b6ff3-8023-4e34-9e10-2bcc51ef5468\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-hnxk4"
Sep 18 20:10:56.248578 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:10:56.248552    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/8d1b6ff3-8023-4e34-9e10-2bcc51ef5468-nginx-conf\") pod \"kuadrant-console-plugin-fbdd4f445-hnxk4\" (UID: \"8d1b6ff3-8023-4e34-9e10-2bcc51ef5468\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-hnxk4"
Sep 18 20:10:56.248661 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:10:56.248633    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d7pnr\" (UniqueName: \"kubernetes.io/projected/8d1b6ff3-8023-4e34-9e10-2bcc51ef5468-kube-api-access-d7pnr\") pod \"kuadrant-console-plugin-fbdd4f445-hnxk4\" (UID: \"8d1b6ff3-8023-4e34-9e10-2bcc51ef5468\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-hnxk4"
Sep 18 20:10:56.274845 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:10:56.274813    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:10:56.349111 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:10:56.349077    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/8d1b6ff3-8023-4e34-9e10-2bcc51ef5468-plugin-serving-cert\") pod \"kuadrant-console-plugin-fbdd4f445-hnxk4\" (UID: \"8d1b6ff3-8023-4e34-9e10-2bcc51ef5468\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-hnxk4"
Sep 18 20:10:56.349323 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:10:56.349143    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/8d1b6ff3-8023-4e34-9e10-2bcc51ef5468-nginx-conf\") pod \"kuadrant-console-plugin-fbdd4f445-hnxk4\" (UID: \"8d1b6ff3-8023-4e34-9e10-2bcc51ef5468\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-hnxk4"
Sep 18 20:10:56.349323 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:10:56.349207    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d7pnr\" (UniqueName: \"kubernetes.io/projected/8d1b6ff3-8023-4e34-9e10-2bcc51ef5468-kube-api-access-d7pnr\") pod \"kuadrant-console-plugin-fbdd4f445-hnxk4\" (UID: \"8d1b6ff3-8023-4e34-9e10-2bcc51ef5468\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-hnxk4"
Sep 18 20:10:56.349896 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:10:56.349870    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/8d1b6ff3-8023-4e34-9e10-2bcc51ef5468-nginx-conf\") pod \"kuadrant-console-plugin-fbdd4f445-hnxk4\" (UID: \"8d1b6ff3-8023-4e34-9e10-2bcc51ef5468\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-hnxk4"
Sep 18 20:10:56.351587 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:10:56.351564    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/8d1b6ff3-8023-4e34-9e10-2bcc51ef5468-plugin-serving-cert\") pod \"kuadrant-console-plugin-fbdd4f445-hnxk4\" (UID: \"8d1b6ff3-8023-4e34-9e10-2bcc51ef5468\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-hnxk4"
Sep 18 20:10:56.356891 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:10:56.356872    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d7pnr\" (UniqueName: \"kubernetes.io/projected/8d1b6ff3-8023-4e34-9e10-2bcc51ef5468-kube-api-access-d7pnr\") pod \"kuadrant-console-plugin-fbdd4f445-hnxk4\" (UID: \"8d1b6ff3-8023-4e34-9e10-2bcc51ef5468\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-hnxk4"
Sep 18 20:10:56.491119 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:10:56.491091    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-hnxk4"
Sep 18 20:10:56.760860 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:10:56.760835    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-fbdd4f445-hnxk4"]
Sep 18 20:10:56.762345 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:10:56.762310    2532 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8d1b6ff3_8023_4e34_9e10_2bcc51ef5468.slice/crio-a540e50924fb2905b8f87c636990d6cada391002e0318b39a42a50237140c896 WatchSource:0}: Error finding container a540e50924fb2905b8f87c636990d6cada391002e0318b39a42a50237140c896: Status 404 returned error can't find the container with id a540e50924fb2905b8f87c636990d6cada391002e0318b39a42a50237140c896
Sep 18 20:10:57.738678 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:10:57.738614    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-hnxk4" event={"ID":"8d1b6ff3-8023-4e34-9e10-2bcc51ef5468","Type":"ContainerStarted","Data":"a540e50924fb2905b8f87c636990d6cada391002e0318b39a42a50237140c896"}
Sep 18 20:11:00.145863 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:11:00.145826    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:11:00.209712 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:11:00.209682    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:11:00.952688 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:11:00.952651    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:11:01.013653 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:11:01.013623    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:11:01.153748 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:11:01.153718    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:11:01.256328 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:11:01.256290    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:11:01.766231 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:11:01.766131    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-hnxk4" event={"ID":"8d1b6ff3-8023-4e34-9e10-2bcc51ef5468","Type":"ContainerStarted","Data":"348f2fd18c221c97a8c739289e04c952b559bc9c00733fd9e33de38e3f45fe3c"}
Sep 18 20:11:01.786052 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:11:01.785991    2532 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-hnxk4" podStartSLOduration=1.080483177 podStartE2EDuration="5.785972807s" totalImagesPullingTime="4.70548963s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 20:10:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 20:11:01.783372322 +0000 UTC m=+593.548761721" watchObservedRunningTime="2026-09-18 20:11:01.785972807 +0000 UTC m=+593.551362207"
Sep 18 20:11:06.445022 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:11:06.444989    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:11:07.774549 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:11:07.774513    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:11:08.861655 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:11:08.861617    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:11:08.961738 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:11:08.961706    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:11:10.276372 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:11:10.276342    2532 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-m7wdd_32e33d39-7826-499d-bc01-0487b7580e7a/console-operator/1.log"
Sep 18 20:11:10.277889 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:11:10.277869    2532 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-m7wdd_32e33d39-7826-499d-bc01-0487b7580e7a/console-operator/1.log"
Sep 18 20:11:10.285211 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:11:10.285190    2532 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rtw9n_73fbf16b-3549-4a6c-a959-b633237ebd0c/ovn-acl-logging/0.log"
Sep 18 20:11:10.286685 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:11:10.286662    2532 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rtw9n_73fbf16b-3549-4a6c-a959-b633237ebd0c/ovn-acl-logging/0.log"
Sep 18 20:11:14.374663 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:11:14.374634    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:11:14.474962 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:11:14.474933    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:11:14.981559 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:11:14.981531    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:11:15.046219 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:11:15.046183    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:11:19.404323 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:11:19.404293    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:11:19.504752 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:11:19.504708    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:11:19.794216 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:11:19.794186    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:11:19.894759 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:11:19.894728    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:11:20.185240 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:11:20.185162    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:11:20.285875 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:11:20.285846    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:11:26.188433 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:11:26.188356    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:11:26.341822 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:11:26.341778    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:11:26.542480 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:11:26.542445    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:11:26.603589 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:11:26.603555    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:11:28.630295 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:11:28.630246    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:11:28.687865 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:11:28.687830    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:11:34.854704 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:11:34.854661    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:11:35.241946 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:11:35.241909    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:11:36.577204 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:11:36.577168    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:11:36.691834 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:11:36.691794    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:11:36.759022 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:11:36.758994    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:11:36.989693 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:11:36.989658    2532 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-vptdl"]
Sep 18 20:11:36.994952 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:11:36.994927    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-84994f7ffc-vptdl"
Sep 18 20:11:36.997292 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:11:36.997269    2532 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Sep 18 20:11:37.000582 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:11:37.000552    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-vptdl"]
Sep 18 20:11:37.028326 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:11:37.028291    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-vptdl"]
Sep 18 20:11:37.090638 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:11:37.090595    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bnrkg\" (UniqueName: \"kubernetes.io/projected/a9d9b684-8e0c-404c-b00f-a18940fbfe2f-kube-api-access-bnrkg\") pod \"limitador-limitador-84994f7ffc-vptdl\" (UID: \"a9d9b684-8e0c-404c-b00f-a18940fbfe2f\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-vptdl"
Sep 18 20:11:37.090803 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:11:37.090688    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/a9d9b684-8e0c-404c-b00f-a18940fbfe2f-config-file\") pod \"limitador-limitador-84994f7ffc-vptdl\" (UID: \"a9d9b684-8e0c-404c-b00f-a18940fbfe2f\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-vptdl"
Sep 18 20:11:37.192086 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:11:37.192050    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bnrkg\" (UniqueName: \"kubernetes.io/projected/a9d9b684-8e0c-404c-b00f-a18940fbfe2f-kube-api-access-bnrkg\") pod \"limitador-limitador-84994f7ffc-vptdl\" (UID: \"a9d9b684-8e0c-404c-b00f-a18940fbfe2f\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-vptdl"
Sep 18 20:11:37.192263 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:11:37.192125    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/a9d9b684-8e0c-404c-b00f-a18940fbfe2f-config-file\") pod \"limitador-limitador-84994f7ffc-vptdl\" (UID: \"a9d9b684-8e0c-404c-b00f-a18940fbfe2f\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-vptdl"
Sep 18 20:11:37.192813 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:11:37.192795    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/a9d9b684-8e0c-404c-b00f-a18940fbfe2f-config-file\") pod \"limitador-limitador-84994f7ffc-vptdl\" (UID: \"a9d9b684-8e0c-404c-b00f-a18940fbfe2f\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-vptdl"
Sep 18 20:11:37.199869 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:11:37.199846    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bnrkg\" (UniqueName: \"kubernetes.io/projected/a9d9b684-8e0c-404c-b00f-a18940fbfe2f-kube-api-access-bnrkg\") pod \"limitador-limitador-84994f7ffc-vptdl\" (UID: \"a9d9b684-8e0c-404c-b00f-a18940fbfe2f\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-vptdl"
Sep 18 20:11:37.305891 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:11:37.305798    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-84994f7ffc-vptdl"
Sep 18 20:11:37.416563 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:11:37.416526    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:11:37.513075 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:11:37.513043    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:11:37.514179 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:11:37.514160    2532 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-59c974bd48-mpxz4"]
Sep 18 20:11:37.516928 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:11:37.516913    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-59c974bd48-mpxz4"
Sep 18 20:11:37.519252 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:11:37.519233    2532 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-c2nff\""
Sep 18 20:11:37.526799 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:11:37.526778    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-59c974bd48-mpxz4"]
Sep 18 20:11:37.577395 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:11:37.577367    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-vptdl"]
Sep 18 20:11:37.579902 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:11:37.579876    2532 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda9d9b684_8e0c_404c_b00f_a18940fbfe2f.slice/crio-3f6843992b0a6a9777aed08392e5dff7e1ec530a452bd01b3cc4634436ecc6ec WatchSource:0}: Error finding container 3f6843992b0a6a9777aed08392e5dff7e1ec530a452bd01b3cc4634436ecc6ec: Status 404 returned error can't find the container with id 3f6843992b0a6a9777aed08392e5dff7e1ec530a452bd01b3cc4634436ecc6ec
Sep 18 20:11:37.595534 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:11:37.595473    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bbb4k\" (UniqueName: \"kubernetes.io/projected/e7bb7b92-2cfd-481a-9299-ad9ffb0e3d13-kube-api-access-bbb4k\") pod \"authorino-59c974bd48-mpxz4\" (UID: \"e7bb7b92-2cfd-481a-9299-ad9ffb0e3d13\") " pod="kuadrant-system/authorino-59c974bd48-mpxz4"
Sep 18 20:11:37.696287 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:11:37.696253    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bbb4k\" (UniqueName: \"kubernetes.io/projected/e7bb7b92-2cfd-481a-9299-ad9ffb0e3d13-kube-api-access-bbb4k\") pod \"authorino-59c974bd48-mpxz4\" (UID: \"e7bb7b92-2cfd-481a-9299-ad9ffb0e3d13\") " pod="kuadrant-system/authorino-59c974bd48-mpxz4"
Sep 18 20:11:37.705163 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:11:37.705129    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bbb4k\" (UniqueName: \"kubernetes.io/projected/e7bb7b92-2cfd-481a-9299-ad9ffb0e3d13-kube-api-access-bbb4k\") pod \"authorino-59c974bd48-mpxz4\" (UID: \"e7bb7b92-2cfd-481a-9299-ad9ffb0e3d13\") " pod="kuadrant-system/authorino-59c974bd48-mpxz4"
Sep 18 20:11:37.826640 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:11:37.826556    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-59c974bd48-mpxz4"
Sep 18 20:11:37.941659 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:11:37.941616    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-84994f7ffc-vptdl" event={"ID":"a9d9b684-8e0c-404c-b00f-a18940fbfe2f","Type":"ContainerStarted","Data":"3f6843992b0a6a9777aed08392e5dff7e1ec530a452bd01b3cc4634436ecc6ec"}
Sep 18 20:11:38.097522 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:11:38.097481    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-59c974bd48-mpxz4"]
Sep 18 20:11:38.100293 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:11:38.100266    2532 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode7bb7b92_2cfd_481a_9299_ad9ffb0e3d13.slice/crio-4cb2caace5473d97179940bf744215228788d5075b309cb13724a78ca287c089 WatchSource:0}: Error finding container 4cb2caace5473d97179940bf744215228788d5075b309cb13724a78ca287c089: Status 404 returned error can't find the container with id 4cb2caace5473d97179940bf744215228788d5075b309cb13724a78ca287c089
Sep 18 20:11:38.951144 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:11:38.951072    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-59c974bd48-mpxz4" event={"ID":"e7bb7b92-2cfd-481a-9299-ad9ffb0e3d13","Type":"ContainerStarted","Data":"4cb2caace5473d97179940bf744215228788d5075b309cb13724a78ca287c089"}
Sep 18 20:11:40.963130 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:11:40.963096    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-59c974bd48-mpxz4" event={"ID":"e7bb7b92-2cfd-481a-9299-ad9ffb0e3d13","Type":"ContainerStarted","Data":"f74a4a306e4ee326c151a47b116d443a1f578beff5d8e7c91a079a487c487408"}
Sep 18 20:11:40.964606 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:11:40.964582    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-84994f7ffc-vptdl" event={"ID":"a9d9b684-8e0c-404c-b00f-a18940fbfe2f","Type":"ContainerStarted","Data":"4eb735041288563504ec66c5ae8ca5811d2f27123b856241a7931b4ffa838165"}
Sep 18 20:11:40.964716 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:11:40.964654    2532 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-84994f7ffc-vptdl"
Sep 18 20:11:40.979456 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:11:40.979405    2532 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-59c974bd48-mpxz4" podStartSLOduration=1.665029707 podStartE2EDuration="3.979391001s" totalImagesPullingTime="2.314361294s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 20:11:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 20:11:40.97660438 +0000 UTC m=+632.741993791" watchObservedRunningTime="2026-09-18 20:11:40.979391001 +0000 UTC m=+632.744780399"
Sep 18 20:11:40.993327 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:11:40.993279    2532 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-84994f7ffc-vptdl" podStartSLOduration=2.165220893 podStartE2EDuration="4.993253651s" totalImagesPullingTime="2.828032758s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 20:11:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 20:11:40.991675824 +0000 UTC m=+632.757065225" watchObservedRunningTime="2026-09-18 20:11:40.993253651 +0000 UTC m=+632.758643067"
Sep 18 20:11:51.970775 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:11:51.970742    2532 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-84994f7ffc-vptdl"
Sep 18 20:11:57.832411 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:11:57.832368    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:11:57.931343 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:11:57.931310    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:11:58.286566 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:11:58.286536    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:12:01.063942 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:12:01.063909    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:12:01.119507 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:12:01.119457    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:12:10.147671 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:12:10.147633    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:12:10.383713 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:12:10.383680    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:12:12.833016 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:12:12.832985    2532 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-59c974bd48-mpxz4"]
Sep 18 20:12:12.833441 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:12:12.833233    2532 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-59c974bd48-mpxz4" podUID="e7bb7b92-2cfd-481a-9299-ad9ffb0e3d13" containerName="authorino" containerID="cri-o://f74a4a306e4ee326c151a47b116d443a1f578beff5d8e7c91a079a487c487408" gracePeriod=30
Sep 18 20:12:13.070094 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:12:13.070067    2532 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-59c974bd48-mpxz4"
Sep 18 20:12:13.135420 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:12:13.135330    2532 generic.go:356] "Generic (PLEG): container finished" podID="e7bb7b92-2cfd-481a-9299-ad9ffb0e3d13" containerID="f74a4a306e4ee326c151a47b116d443a1f578beff5d8e7c91a079a487c487408" exitCode=0
Sep 18 20:12:13.135420 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:12:13.135393    2532 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-59c974bd48-mpxz4"
Sep 18 20:12:13.135643 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:12:13.135420    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-59c974bd48-mpxz4" event={"ID":"e7bb7b92-2cfd-481a-9299-ad9ffb0e3d13","Type":"ContainerDied","Data":"f74a4a306e4ee326c151a47b116d443a1f578beff5d8e7c91a079a487c487408"}
Sep 18 20:12:13.135643 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:12:13.135467    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-59c974bd48-mpxz4" event={"ID":"e7bb7b92-2cfd-481a-9299-ad9ffb0e3d13","Type":"ContainerDied","Data":"4cb2caace5473d97179940bf744215228788d5075b309cb13724a78ca287c089"}
Sep 18 20:12:13.135643 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:12:13.135507    2532 scope.go:117] "RemoveContainer" containerID="f74a4a306e4ee326c151a47b116d443a1f578beff5d8e7c91a079a487c487408"
Sep 18 20:12:13.143612 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:12:13.143591    2532 scope.go:117] "RemoveContainer" containerID="f74a4a306e4ee326c151a47b116d443a1f578beff5d8e7c91a079a487c487408"
Sep 18 20:12:13.143864 ip-10-0-130-81 kubenswrapper[2532]: E0918 20:12:13.143846    2532 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f74a4a306e4ee326c151a47b116d443a1f578beff5d8e7c91a079a487c487408\": container with ID starting with f74a4a306e4ee326c151a47b116d443a1f578beff5d8e7c91a079a487c487408 not found: ID does not exist" containerID="f74a4a306e4ee326c151a47b116d443a1f578beff5d8e7c91a079a487c487408"
Sep 18 20:12:13.143914 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:12:13.143874    2532 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f74a4a306e4ee326c151a47b116d443a1f578beff5d8e7c91a079a487c487408"} err="failed to get container status \"f74a4a306e4ee326c151a47b116d443a1f578beff5d8e7c91a079a487c487408\": rpc error: code = NotFound desc = could not find container \"f74a4a306e4ee326c151a47b116d443a1f578beff5d8e7c91a079a487c487408\": container with ID starting with f74a4a306e4ee326c151a47b116d443a1f578beff5d8e7c91a079a487c487408 not found: ID does not exist"
Sep 18 20:12:13.209806 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:12:13.209771    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bbb4k\" (UniqueName: \"kubernetes.io/projected/e7bb7b92-2cfd-481a-9299-ad9ffb0e3d13-kube-api-access-bbb4k\") pod \"e7bb7b92-2cfd-481a-9299-ad9ffb0e3d13\" (UID: \"e7bb7b92-2cfd-481a-9299-ad9ffb0e3d13\") "
Sep 18 20:12:13.211952 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:12:13.211922    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e7bb7b92-2cfd-481a-9299-ad9ffb0e3d13-kube-api-access-bbb4k" (OuterVolumeSpecName: "kube-api-access-bbb4k") pod "e7bb7b92-2cfd-481a-9299-ad9ffb0e3d13" (UID: "e7bb7b92-2cfd-481a-9299-ad9ffb0e3d13"). InnerVolumeSpecName "kube-api-access-bbb4k". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 20:12:13.310840 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:12:13.310813    2532 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bbb4k\" (UniqueName: \"kubernetes.io/projected/e7bb7b92-2cfd-481a-9299-ad9ffb0e3d13-kube-api-access-bbb4k\") on node \"ip-10-0-130-81.ec2.internal\" DevicePath \"\""
Sep 18 20:12:13.456378 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:12:13.456348    2532 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-59c974bd48-mpxz4"]
Sep 18 20:12:13.459566 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:12:13.459543    2532 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-59c974bd48-mpxz4"]
Sep 18 20:12:14.305933 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:12:14.305901    2532 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e7bb7b92-2cfd-481a-9299-ad9ffb0e3d13" path="/var/lib/kubelet/pods/e7bb7b92-2cfd-481a-9299-ad9ffb0e3d13/volumes"
Sep 18 20:12:17.420646 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:12:17.420607    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:12:17.546781 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:12:17.546746    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:12:17.630823 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:12:17.630795    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:12:17.688771 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:12:17.688692    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:12:21.006376 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:12:21.006345    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:12:21.107861 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:12:21.107831    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:12:50.034143 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:12:50.034110    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:12:50.088589 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:12:50.088501    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:12:53.541104 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:12:53.541029    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:12:53.631537 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:12:53.631477    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:12:55.032635 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:12:55.032606    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:12:55.133568 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:12:55.133536    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:12:55.242372 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:12:55.242339    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:12:55.416391 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:12:55.416310    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:12:55.522601 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:12:55.522569    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:12:55.627403 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:12:55.627369    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:12:55.786012 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:12:55.785978    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-vptdl"]
Sep 18 20:13:06.104827 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:13:06.104781    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:13:06.198623 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:13:06.198590    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:13:17.538344 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:13:17.538307    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:13:18.030785 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:13:18.030744    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-vptdl"]
Sep 18 20:13:24.021617 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:13:24.021575    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:13:24.699735 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:13:24.699702    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-vptdl"]
Sep 18 20:13:52.333042 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:13:52.333004    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:13:53.392983 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:13:53.392946    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-vptdl"]
Sep 18 20:14:00.835155 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:14:00.835122    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:14:02.094624 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:14:02.094588    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-vptdl"]
Sep 18 20:14:26.005151 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:14:26.005073    2532 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fv5rtb"]
Sep 18 20:14:26.005540 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:14:26.005380    2532 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e7bb7b92-2cfd-481a-9299-ad9ffb0e3d13" containerName="authorino"
Sep 18 20:14:26.005540 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:14:26.005392    2532 state_mem.go:107] "Deleted CPUSet assignment" podUID="e7bb7b92-2cfd-481a-9299-ad9ffb0e3d13" containerName="authorino"
Sep 18 20:14:26.005540 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:14:26.005475    2532 memory_manager.go:368] "RemoveStaleState removing state" podUID="e7bb7b92-2cfd-481a-9299-ad9ffb0e3d13" containerName="authorino"
Sep 18 20:14:26.008351 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:14:26.008333    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fv5rtb"
Sep 18 20:14:26.012373 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:14:26.012347    2532 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-m7b4l\""
Sep 18 20:14:26.012528 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:14:26.012416    2532 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Sep 18 20:14:26.012528 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:14:26.012354    2532 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Sep 18 20:14:26.012528 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:14:26.012347    2532 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\""
Sep 18 20:14:26.017532 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:14:26.017509    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fv5rtb"]
Sep 18 20:14:26.035855 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:14:26.035815    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:14:26.122090 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:14:26.122057    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c87a3f2e-928e-42eb-a6c1-c9b0d703a609-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fv5rtb\" (UID: \"c87a3f2e-928e-42eb-a6c1-c9b0d703a609\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fv5rtb"
Sep 18 20:14:26.122336 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:14:26.122096    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c87a3f2e-928e-42eb-a6c1-c9b0d703a609-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fv5rtb\" (UID: \"c87a3f2e-928e-42eb-a6c1-c9b0d703a609\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fv5rtb"
Sep 18 20:14:26.122336 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:14:26.122117    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c87a3f2e-928e-42eb-a6c1-c9b0d703a609-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fv5rtb\" (UID: \"c87a3f2e-928e-42eb-a6c1-c9b0d703a609\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fv5rtb"
Sep 18 20:14:26.122336 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:14:26.122175    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c87a3f2e-928e-42eb-a6c1-c9b0d703a609-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fv5rtb\" (UID: \"c87a3f2e-928e-42eb-a6c1-c9b0d703a609\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fv5rtb"
Sep 18 20:14:26.122336 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:14:26.122230    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c87a3f2e-928e-42eb-a6c1-c9b0d703a609-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fv5rtb\" (UID: \"c87a3f2e-928e-42eb-a6c1-c9b0d703a609\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fv5rtb"
Sep 18 20:14:26.122336 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:14:26.122298    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g8pdd\" (UniqueName: \"kubernetes.io/projected/c87a3f2e-928e-42eb-a6c1-c9b0d703a609-kube-api-access-g8pdd\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fv5rtb\" (UID: \"c87a3f2e-928e-42eb-a6c1-c9b0d703a609\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fv5rtb"
Sep 18 20:14:26.222751 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:14:26.222715    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c87a3f2e-928e-42eb-a6c1-c9b0d703a609-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fv5rtb\" (UID: \"c87a3f2e-928e-42eb-a6c1-c9b0d703a609\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fv5rtb"
Sep 18 20:14:26.222934 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:14:26.222760    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c87a3f2e-928e-42eb-a6c1-c9b0d703a609-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fv5rtb\" (UID: \"c87a3f2e-928e-42eb-a6c1-c9b0d703a609\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fv5rtb"
Sep 18 20:14:26.222934 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:14:26.222781    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c87a3f2e-928e-42eb-a6c1-c9b0d703a609-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fv5rtb\" (UID: \"c87a3f2e-928e-42eb-a6c1-c9b0d703a609\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fv5rtb"
Sep 18 20:14:26.222934 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:14:26.222800    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c87a3f2e-928e-42eb-a6c1-c9b0d703a609-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fv5rtb\" (UID: \"c87a3f2e-928e-42eb-a6c1-c9b0d703a609\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fv5rtb"
Sep 18 20:14:26.222934 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:14:26.222826    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c87a3f2e-928e-42eb-a6c1-c9b0d703a609-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fv5rtb\" (UID: \"c87a3f2e-928e-42eb-a6c1-c9b0d703a609\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fv5rtb"
Sep 18 20:14:26.222934 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:14:26.222853    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g8pdd\" (UniqueName: \"kubernetes.io/projected/c87a3f2e-928e-42eb-a6c1-c9b0d703a609-kube-api-access-g8pdd\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fv5rtb\" (UID: \"c87a3f2e-928e-42eb-a6c1-c9b0d703a609\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fv5rtb"
Sep 18 20:14:26.223226 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:14:26.223161    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c87a3f2e-928e-42eb-a6c1-c9b0d703a609-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fv5rtb\" (UID: \"c87a3f2e-928e-42eb-a6c1-c9b0d703a609\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fv5rtb"
Sep 18 20:14:26.223226 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:14:26.223166    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c87a3f2e-928e-42eb-a6c1-c9b0d703a609-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fv5rtb\" (UID: \"c87a3f2e-928e-42eb-a6c1-c9b0d703a609\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fv5rtb"
Sep 18 20:14:26.223304 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:14:26.223258    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c87a3f2e-928e-42eb-a6c1-c9b0d703a609-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fv5rtb\" (UID: \"c87a3f2e-928e-42eb-a6c1-c9b0d703a609\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fv5rtb"
Sep 18 20:14:26.225106 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:14:26.225082    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c87a3f2e-928e-42eb-a6c1-c9b0d703a609-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fv5rtb\" (UID: \"c87a3f2e-928e-42eb-a6c1-c9b0d703a609\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fv5rtb"
Sep 18 20:14:26.225351 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:14:26.225335    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c87a3f2e-928e-42eb-a6c1-c9b0d703a609-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fv5rtb\" (UID: \"c87a3f2e-928e-42eb-a6c1-c9b0d703a609\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fv5rtb"
Sep 18 20:14:26.230741 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:14:26.230722    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g8pdd\" (UniqueName: \"kubernetes.io/projected/c87a3f2e-928e-42eb-a6c1-c9b0d703a609-kube-api-access-g8pdd\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fv5rtb\" (UID: \"c87a3f2e-928e-42eb-a6c1-c9b0d703a609\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fv5rtb"
Sep 18 20:14:26.319733 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:14:26.319649    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fv5rtb"
Sep 18 20:14:26.589635 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:14:26.589611    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fv5rtb"]
Sep 18 20:14:26.591428 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:14:26.591398    2532 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc87a3f2e_928e_42eb_a6c1_c9b0d703a609.slice/crio-34be2909eee640845f57fac30fa2852a0b9764db8b0edc6269df1a50ad696579 WatchSource:0}: Error finding container 34be2909eee640845f57fac30fa2852a0b9764db8b0edc6269df1a50ad696579: Status 404 returned error can't find the container with id 34be2909eee640845f57fac30fa2852a0b9764db8b0edc6269df1a50ad696579
Sep 18 20:14:26.593151 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:14:26.593132    2532 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 18 20:14:26.793559 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:14:26.793516    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fv5rtb" event={"ID":"c87a3f2e-928e-42eb-a6c1-c9b0d703a609","Type":"ContainerStarted","Data":"34be2909eee640845f57fac30fa2852a0b9764db8b0edc6269df1a50ad696579"}
Sep 18 20:14:27.597664 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:14:27.597636    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-vptdl"]
Sep 18 20:14:27.800005 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:14:27.799916    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fv5rtb" event={"ID":"c87a3f2e-928e-42eb-a6c1-c9b0d703a609","Type":"ContainerStarted","Data":"9b5984e7a27e5da8c9ce74918f27f7f689d46c5fe58b02726ffbccf38caeae80"}
Sep 18 20:14:27.818287 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:14:27.818234    2532 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fv5rtb" podStartSLOduration=1.864241236 podStartE2EDuration="2.81821854s" totalImagesPullingTime="953.977304ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 20:14:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 20:14:27.815950606 +0000 UTC m=+799.581340019" watchObservedRunningTime="2026-09-18 20:14:27.81821854 +0000 UTC m=+799.583607938"
Sep 18 20:14:28.790842 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:14:28.790807    2532 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-rwtrr"]
Sep 18 20:14:28.794180 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:14:28.794163    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rwtrr"
Sep 18 20:14:28.796574 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:14:28.796553    2532 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\""
Sep 18 20:14:28.803991 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:14:28.803963    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-rwtrr"]
Sep 18 20:14:28.844754 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:14:28.844714    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e0e19e6d-76ad-4660-881e-e0d71cbc05a2-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-rwtrr\" (UID: \"e0e19e6d-76ad-4660-881e-e0d71cbc05a2\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rwtrr"
Sep 18 20:14:28.844754 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:14:28.844750    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e0e19e6d-76ad-4660-881e-e0d71cbc05a2-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-rwtrr\" (UID: \"e0e19e6d-76ad-4660-881e-e0d71cbc05a2\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rwtrr"
Sep 18 20:14:28.845038 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:14:28.844773    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e0e19e6d-76ad-4660-881e-e0d71cbc05a2-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-rwtrr\" (UID: \"e0e19e6d-76ad-4660-881e-e0d71cbc05a2\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rwtrr"
Sep 18 20:14:28.845038 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:14:28.844815    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-77jkk\" (UniqueName: \"kubernetes.io/projected/e0e19e6d-76ad-4660-881e-e0d71cbc05a2-kube-api-access-77jkk\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-rwtrr\" (UID: \"e0e19e6d-76ad-4660-881e-e0d71cbc05a2\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rwtrr"
Sep 18 20:14:28.845038 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:14:28.844934    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e0e19e6d-76ad-4660-881e-e0d71cbc05a2-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-rwtrr\" (UID: \"e0e19e6d-76ad-4660-881e-e0d71cbc05a2\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rwtrr"
Sep 18 20:14:28.845038 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:14:28.844962    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e0e19e6d-76ad-4660-881e-e0d71cbc05a2-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-rwtrr\" (UID: \"e0e19e6d-76ad-4660-881e-e0d71cbc05a2\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rwtrr"
Sep 18 20:14:28.901069 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:14:28.901033    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:14:28.946345 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:14:28.946308    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e0e19e6d-76ad-4660-881e-e0d71cbc05a2-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-rwtrr\" (UID: \"e0e19e6d-76ad-4660-881e-e0d71cbc05a2\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rwtrr"
Sep 18 20:14:28.946345 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:14:28.946343    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-77jkk\" (UniqueName: \"kubernetes.io/projected/e0e19e6d-76ad-4660-881e-e0d71cbc05a2-kube-api-access-77jkk\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-rwtrr\" (UID: \"e0e19e6d-76ad-4660-881e-e0d71cbc05a2\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rwtrr"
Sep 18 20:14:28.946633 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:14:28.946381    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e0e19e6d-76ad-4660-881e-e0d71cbc05a2-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-rwtrr\" (UID: \"e0e19e6d-76ad-4660-881e-e0d71cbc05a2\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rwtrr"
Sep 18 20:14:28.946633 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:14:28.946400    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e0e19e6d-76ad-4660-881e-e0d71cbc05a2-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-rwtrr\" (UID: \"e0e19e6d-76ad-4660-881e-e0d71cbc05a2\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rwtrr"
Sep 18 20:14:28.946633 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:14:28.946512    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e0e19e6d-76ad-4660-881e-e0d71cbc05a2-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-rwtrr\" (UID: \"e0e19e6d-76ad-4660-881e-e0d71cbc05a2\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rwtrr"
Sep 18 20:14:28.946633 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:14:28.946546    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e0e19e6d-76ad-4660-881e-e0d71cbc05a2-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-rwtrr\" (UID: \"e0e19e6d-76ad-4660-881e-e0d71cbc05a2\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rwtrr"
Sep 18 20:14:28.946883 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:14:28.946862    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e0e19e6d-76ad-4660-881e-e0d71cbc05a2-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-rwtrr\" (UID: \"e0e19e6d-76ad-4660-881e-e0d71cbc05a2\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rwtrr"
Sep 18 20:14:28.946940 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:14:28.946880    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e0e19e6d-76ad-4660-881e-e0d71cbc05a2-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-rwtrr\" (UID: \"e0e19e6d-76ad-4660-881e-e0d71cbc05a2\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rwtrr"
Sep 18 20:14:28.946982 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:14:28.946939    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e0e19e6d-76ad-4660-881e-e0d71cbc05a2-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-rwtrr\" (UID: \"e0e19e6d-76ad-4660-881e-e0d71cbc05a2\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rwtrr"
Sep 18 20:14:28.948713 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:14:28.948693    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e0e19e6d-76ad-4660-881e-e0d71cbc05a2-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-rwtrr\" (UID: \"e0e19e6d-76ad-4660-881e-e0d71cbc05a2\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rwtrr"
Sep 18 20:14:28.949003 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:14:28.948986    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e0e19e6d-76ad-4660-881e-e0d71cbc05a2-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-rwtrr\" (UID: \"e0e19e6d-76ad-4660-881e-e0d71cbc05a2\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rwtrr"
Sep 18 20:14:28.955731 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:14:28.955708    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-77jkk\" (UniqueName: \"kubernetes.io/projected/e0e19e6d-76ad-4660-881e-e0d71cbc05a2-kube-api-access-77jkk\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-rwtrr\" (UID: \"e0e19e6d-76ad-4660-881e-e0d71cbc05a2\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rwtrr"
Sep 18 20:14:29.105242 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:14:29.105144    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rwtrr"
Sep 18 20:14:29.375494 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:14:29.375458    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-rwtrr"]
Sep 18 20:14:29.376818 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:14:29.376792    2532 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode0e19e6d_76ad_4660_881e_e0d71cbc05a2.slice/crio-210e56c7fbd21ee5a7ecf74af7391adcb55bb73a73e91dbeb435e53d34842194 WatchSource:0}: Error finding container 210e56c7fbd21ee5a7ecf74af7391adcb55bb73a73e91dbeb435e53d34842194: Status 404 returned error can't find the container with id 210e56c7fbd21ee5a7ecf74af7391adcb55bb73a73e91dbeb435e53d34842194
Sep 18 20:14:29.812566 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:14:29.812527    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rwtrr" event={"ID":"e0e19e6d-76ad-4660-881e-e0d71cbc05a2","Type":"ContainerStarted","Data":"2d5dfa1faf8387bbfa26eb998f522f63348298e4f4cbeebffe39fda4ff98cfd0"}
Sep 18 20:14:29.812956 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:14:29.812590    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rwtrr" event={"ID":"e0e19e6d-76ad-4660-881e-e0d71cbc05a2","Type":"ContainerStarted","Data":"210e56c7fbd21ee5a7ecf74af7391adcb55bb73a73e91dbeb435e53d34842194"}
Sep 18 20:14:29.831023 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:14:29.830966    2532 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rwtrr" podStartSLOduration=1.647874175 podStartE2EDuration="1.830950503s" totalImagesPullingTime="183.076328ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 20:14:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 20:14:29.828480442 +0000 UTC m=+801.593869870" watchObservedRunningTime="2026-09-18 20:14:29.830950503 +0000 UTC m=+801.596339903"
Sep 18 20:14:34.897127 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:14:34.897096    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-vptdl"]
Sep 18 20:14:36.320723 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:14:36.320687    2532 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fv5rtb"
Sep 18 20:14:36.321154 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:14:36.320749    2532 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fv5rtb"
Sep 18 20:14:36.333144 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:14:36.333121    2532 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fv5rtb"
Sep 18 20:14:36.858764 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:14:36.858735    2532 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fv5rtb"
Sep 18 20:14:38.827997 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:14:38.827934    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:14:39.106030 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:14:39.105942    2532 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rwtrr"
Sep 18 20:14:39.106030 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:14:39.106004    2532 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rwtrr"
Sep 18 20:14:39.118564 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:14:39.118536    2532 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rwtrr"
Sep 18 20:14:39.873218 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:14:39.873186    2532 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rwtrr"
Sep 18 20:14:42.094242 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:14:42.094202    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-vptdl"]
Sep 18 20:15:47.868951 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:15:47.868916    2532 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-68d6f84597-k9t4q"]
Sep 18 20:15:47.872002 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:15:47.871978    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-68d6f84597-k9t4q"
Sep 18 20:15:47.875656 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:15:47.875633    2532 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-c2nff\""
Sep 18 20:15:47.875782 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:15:47.875633    2532 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\""
Sep 18 20:15:47.879354 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:15:47.879331    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-68d6f84597-k9t4q"]
Sep 18 20:15:47.958445 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:15:47.958417    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jdq9t\" (UniqueName: \"kubernetes.io/projected/3d16378d-1758-4be8-b381-f4f337ba2efc-kube-api-access-jdq9t\") pod \"authorino-68d6f84597-k9t4q\" (UID: \"3d16378d-1758-4be8-b381-f4f337ba2efc\") " pod="kuadrant-system/authorino-68d6f84597-k9t4q"
Sep 18 20:15:47.958579 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:15:47.958534    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/3d16378d-1758-4be8-b381-f4f337ba2efc-tls-cert\") pod \"authorino-68d6f84597-k9t4q\" (UID: \"3d16378d-1758-4be8-b381-f4f337ba2efc\") " pod="kuadrant-system/authorino-68d6f84597-k9t4q"
Sep 18 20:15:48.059068 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:15:48.059027    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jdq9t\" (UniqueName: \"kubernetes.io/projected/3d16378d-1758-4be8-b381-f4f337ba2efc-kube-api-access-jdq9t\") pod \"authorino-68d6f84597-k9t4q\" (UID: \"3d16378d-1758-4be8-b381-f4f337ba2efc\") " pod="kuadrant-system/authorino-68d6f84597-k9t4q"
Sep 18 20:15:48.059234 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:15:48.059121    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/3d16378d-1758-4be8-b381-f4f337ba2efc-tls-cert\") pod \"authorino-68d6f84597-k9t4q\" (UID: \"3d16378d-1758-4be8-b381-f4f337ba2efc\") " pod="kuadrant-system/authorino-68d6f84597-k9t4q"
Sep 18 20:15:48.061653 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:15:48.061630    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/3d16378d-1758-4be8-b381-f4f337ba2efc-tls-cert\") pod \"authorino-68d6f84597-k9t4q\" (UID: \"3d16378d-1758-4be8-b381-f4f337ba2efc\") " pod="kuadrant-system/authorino-68d6f84597-k9t4q"
Sep 18 20:15:48.066738 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:15:48.066716    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jdq9t\" (UniqueName: \"kubernetes.io/projected/3d16378d-1758-4be8-b381-f4f337ba2efc-kube-api-access-jdq9t\") pod \"authorino-68d6f84597-k9t4q\" (UID: \"3d16378d-1758-4be8-b381-f4f337ba2efc\") " pod="kuadrant-system/authorino-68d6f84597-k9t4q"
Sep 18 20:15:48.202329 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:15:48.202253    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-68d6f84597-k9t4q"
Sep 18 20:15:48.469421 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:15:48.469272    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-68d6f84597-k9t4q"]
Sep 18 20:15:48.471705 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:15:48.471680    2532 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3d16378d_1758_4be8_b381_f4f337ba2efc.slice/crio-7c27a5846f2aa805ec989cb2a5862961705471a342ff2d8fceb7e7a328dc9407 WatchSource:0}: Error finding container 7c27a5846f2aa805ec989cb2a5862961705471a342ff2d8fceb7e7a328dc9407: Status 404 returned error can't find the container with id 7c27a5846f2aa805ec989cb2a5862961705471a342ff2d8fceb7e7a328dc9407
Sep 18 20:15:49.206727 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:15:49.206697    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-68d6f84597-k9t4q" event={"ID":"3d16378d-1758-4be8-b381-f4f337ba2efc","Type":"ContainerStarted","Data":"7c27a5846f2aa805ec989cb2a5862961705471a342ff2d8fceb7e7a328dc9407"}
Sep 18 20:15:50.282219 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:15:50.282186    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:15:51.221160 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:15:51.221119    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-68d6f84597-k9t4q" event={"ID":"3d16378d-1758-4be8-b381-f4f337ba2efc","Type":"ContainerStarted","Data":"e96dbf0f2c18b7bced2a4fdea5376e6c4226b6703ea4203a2872651f751d5786"}
Sep 18 20:15:51.241452 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:15:51.241396    2532 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-68d6f84597-k9t4q" podStartSLOduration=2.192694653 podStartE2EDuration="4.241380945s" totalImagesPullingTime="2.048686292s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 20:15:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 20:15:51.239364624 +0000 UTC m=+883.004754058" watchObservedRunningTime="2026-09-18 20:15:51.241380945 +0000 UTC m=+883.006770343"
Sep 18 20:15:51.602569 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:15:51.602436    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:15:52.037309 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:15:52.037280    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:15:52.168165 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:15:52.168128    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:15:53.768815 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:15:53.768780    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:16:00.346963 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:16:00.346930    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:16:10.312884 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:16:10.312851    2532 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-m7wdd_32e33d39-7826-499d-bc01-0487b7580e7a/console-operator/1.log"
Sep 18 20:16:10.313959 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:16:10.313938    2532 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-m7wdd_32e33d39-7826-499d-bc01-0487b7580e7a/console-operator/1.log"
Sep 18 20:16:10.321469 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:16:10.321448    2532 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rtw9n_73fbf16b-3549-4a6c-a959-b633237ebd0c/ovn-acl-logging/0.log"
Sep 18 20:16:10.322168 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:16:10.322153    2532 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rtw9n_73fbf16b-3549-4a6c-a959-b633237ebd0c/ovn-acl-logging/0.log"
Sep 18 20:16:13.116859 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:16:13.116827    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:16:14.467151 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:16:14.467122    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:16:33.286891 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:16:33.286859    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:16:33.624445 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:16:33.624347    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:16:34.738004 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:16:34.737971    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:16:34.834411 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:16:34.834379    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:16:35.816877 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:16:35.816848    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:16:35.916339 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:16:35.916305    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:16:50.312792 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:16:50.312761    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:16:50.413432 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:16:50.413402    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:16:50.621984 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:16:50.621884    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:16:50.723456 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:16:50.723423    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:16:50.875337 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:16:50.875233    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:16:50.959983 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:16:50.959950    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:16:51.059705 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:16:51.059674    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:16:51.495743 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:16:51.495713    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:16:51.682277 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:16:51.682245    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:16:51.781814 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:16:51.781732    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:16:52.077090 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:16:52.077009    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:16:52.215953 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:16:52.215922    2532 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-8728-6e045f"]
Sep 18 20:16:52.219291 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:16:52.219273    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-8728-6e045f"
Sep 18 20:16:52.221862 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:16:52.221843    2532 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-pqc57\""
Sep 18 20:16:52.226016 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:16:52.225995    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-8728-6e045f"]
Sep 18 20:16:52.325959 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:16:52.325922    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-njptv\" (UniqueName: \"kubernetes.io/projected/9fc97ce8-0e91-481b-b4fb-150b89f9cadc-kube-api-access-njptv\") pod \"test-curl-8728-6e045f\" (UID: \"9fc97ce8-0e91-481b-b4fb-150b89f9cadc\") " pod="opendatahub/test-curl-8728-6e045f"
Sep 18 20:16:52.426602 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:16:52.426511    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-njptv\" (UniqueName: \"kubernetes.io/projected/9fc97ce8-0e91-481b-b4fb-150b89f9cadc-kube-api-access-njptv\") pod \"test-curl-8728-6e045f\" (UID: \"9fc97ce8-0e91-481b-b4fb-150b89f9cadc\") " pod="opendatahub/test-curl-8728-6e045f"
Sep 18 20:16:52.434282 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:16:52.434248    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-njptv\" (UniqueName: \"kubernetes.io/projected/9fc97ce8-0e91-481b-b4fb-150b89f9cadc-kube-api-access-njptv\") pod \"test-curl-8728-6e045f\" (UID: \"9fc97ce8-0e91-481b-b4fb-150b89f9cadc\") " pod="opendatahub/test-curl-8728-6e045f"
Sep 18 20:16:52.529713 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:16:52.529686    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-8728-6e045f"
Sep 18 20:16:52.796605 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:16:52.796582    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-8728-6e045f"]
Sep 18 20:16:52.797967 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:16:52.797942    2532 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9fc97ce8_0e91_481b_b4fb_150b89f9cadc.slice/crio-c0deeeeb51d4dba05b86fb696ddddb442a4658c51efb98105e8692895a5e1eec WatchSource:0}: Error finding container c0deeeeb51d4dba05b86fb696ddddb442a4658c51efb98105e8692895a5e1eec: Status 404 returned error can't find the container with id c0deeeeb51d4dba05b86fb696ddddb442a4658c51efb98105e8692895a5e1eec
Sep 18 20:16:53.531872 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:16:53.531840    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-8728-6e045f" event={"ID":"9fc97ce8-0e91-481b-b4fb-150b89f9cadc","Type":"ContainerStarted","Data":"c0deeeeb51d4dba05b86fb696ddddb442a4658c51efb98105e8692895a5e1eec"}
Sep 18 20:16:54.154496 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:16:54.154455    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:16:54.263994 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:16:54.263962    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:16:54.360355 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:16:54.360307    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:16:54.558148 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:16:54.558111    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:16:54.647682 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:16:54.647649    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:16:55.447858 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:16:55.447818    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:16:55.511696 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:16:55.511667    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:16:55.553419 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:16:55.553379    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-8728-6e045f" event={"ID":"9fc97ce8-0e91-481b-b4fb-150b89f9cadc","Type":"ContainerStarted","Data":"3c8b3fef9f2353ec4c875172a6ea5f9db00743bdcfae28d13e6a0f696a3f1c4f"}
Sep 18 20:16:55.567974 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:16:55.567931    2532 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-8728-6e045f" podStartSLOduration=1.493315398 podStartE2EDuration="3.567918348s" totalImagesPullingTime="2.07460295s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 20:16:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 20:16:55.567019569 +0000 UTC m=+947.332408969" watchObservedRunningTime="2026-09-18 20:16:55.567918348 +0000 UTC m=+947.333307747"
Sep 18 20:16:55.853939 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:16:55.853908    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:16:55.974788 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:16:55.974761    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:16:56.074780 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:16:56.074747    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:16:56.134052 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:16:56.134021    2532 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-8728-6e045f"]
Sep 18 20:16:56.140755 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:16:56.140726    2532 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-8728-6e045f"]
Sep 18 20:16:56.248055 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:16:56.247979    2532 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-8728-0b2359"]
Sep 18 20:16:56.248338 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:16:56.248326    2532 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9fc97ce8-0e91-481b-b4fb-150b89f9cadc" containerName="test-curl-8728-6e045f"
Sep 18 20:16:56.248386 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:16:56.248340    2532 state_mem.go:107] "Deleted CPUSet assignment" podUID="9fc97ce8-0e91-481b-b4fb-150b89f9cadc" containerName="test-curl-8728-6e045f"
Sep 18 20:16:56.248424 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:16:56.248413    2532 memory_manager.go:368] "RemoveStaleState removing state" podUID="9fc97ce8-0e91-481b-b4fb-150b89f9cadc" containerName="test-curl-8728-6e045f"
Sep 18 20:16:56.251513 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:16:56.251476    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-8728-0b2359"
Sep 18 20:16:56.259615 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:16:56.259593    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-8728-0b2359"]
Sep 18 20:16:56.359846 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:16:56.359810    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-74vn6\" (UniqueName: \"kubernetes.io/projected/7541c380-c55e-4a3b-a3c4-8596464718e8-kube-api-access-74vn6\") pod \"test-curl-8728-0b2359\" (UID: \"7541c380-c55e-4a3b-a3c4-8596464718e8\") " pod="opendatahub/test-curl-8728-0b2359"
Sep 18 20:16:56.410740 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:16:56.410708    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:16:56.461072 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:16:56.461034    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-74vn6\" (UniqueName: \"kubernetes.io/projected/7541c380-c55e-4a3b-a3c4-8596464718e8-kube-api-access-74vn6\") pod \"test-curl-8728-0b2359\" (UID: \"7541c380-c55e-4a3b-a3c4-8596464718e8\") " pod="opendatahub/test-curl-8728-0b2359"
Sep 18 20:16:56.468865 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:16:56.468834    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-74vn6\" (UniqueName: \"kubernetes.io/projected/7541c380-c55e-4a3b-a3c4-8596464718e8-kube-api-access-74vn6\") pod \"test-curl-8728-0b2359\" (UID: \"7541c380-c55e-4a3b-a3c4-8596464718e8\") " pod="opendatahub/test-curl-8728-0b2359"
Sep 18 20:16:56.559557 ip-10-0-130-81 kubenswrapper[2532]: E0918 20:16:56.559435    2532 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-8728-6e045f" container="test-curl-8728-6e045f"
Sep 18 20:16:56.561702 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:16:56.561687    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-8728-0b2359"
Sep 18 20:16:56.831995 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:16:56.831973    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-8728-0b2359"]
Sep 18 20:16:56.834238 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:16:56.834216    2532 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7541c380_c55e_4a3b_a3c4_8596464718e8.slice/crio-4630b99e6310f545cb3a3060921710ace32102d48c35a08b0c595a72d5c5b88d WatchSource:0}: Error finding container 4630b99e6310f545cb3a3060921710ace32102d48c35a08b0c595a72d5c5b88d: Status 404 returned error can't find the container with id 4630b99e6310f545cb3a3060921710ace32102d48c35a08b0c595a72d5c5b88d
Sep 18 20:16:57.158542 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:16:57.158450    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:16:57.258577 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:16:57.258542    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:16:57.517824 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:16:57.517788    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:16:57.566258 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:16:57.566218    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-8728-0b2359" event={"ID":"7541c380-c55e-4a3b-a3c4-8596464718e8","Type":"ContainerStarted","Data":"4c914684cb0e401a9859c0d05214c466336f38c35aecc5fc736ea617ed233bae"}
Sep 18 20:16:57.566459 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:16:57.566271    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-8728-0b2359" event={"ID":"7541c380-c55e-4a3b-a3c4-8596464718e8","Type":"ContainerStarted","Data":"4630b99e6310f545cb3a3060921710ace32102d48c35a08b0c595a72d5c5b88d"}
Sep 18 20:16:57.566459 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:16:57.566327    2532 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-8728-6e045f" podUID="9fc97ce8-0e91-481b-b4fb-150b89f9cadc" containerName="test-curl-8728-6e045f" containerID="cri-o://3c8b3fef9f2353ec4c875172a6ea5f9db00743bdcfae28d13e6a0f696a3f1c4f" gracePeriod=2
Sep 18 20:16:57.580945 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:16:57.580897    2532 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-8728-0b2359" podStartSLOduration=1.5808757629999999 podStartE2EDuration="1.580875763s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 20:16:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 20:16:57.579133761 +0000 UTC m=+949.344523154" watchObservedRunningTime="2026-09-18 20:16:57.580875763 +0000 UTC m=+949.346265164"
Sep 18 20:16:57.679234 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:16:57.679207    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:16:57.773657 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:16:57.773564    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:16:58.125151 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:16:58.125079    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:16:58.173691 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:16:58.173647    2532 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-8728-0b2359"]
Sep 18 20:16:58.176615 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:16:58.176584    2532 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-8728-0b2359"]
Sep 18 20:16:58.191948 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:16:58.191914    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:16:58.347877 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:16:58.347848    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:16:58.571324 ip-10-0-130-81 kubenswrapper[2532]: E0918 20:16:58.571299    2532 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-8728-0b2359" container="test-curl-8728-0b2359"
Sep 18 20:16:58.685299 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:16:58.685266    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:16:58.775749 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:16:58.775713    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:16:59.076805 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:16:59.076775    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:16:59.176562 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:16:59.176534    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:16:59.295496 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:16:59.295449    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:16:59.557527 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:16:59.557475    2532 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-7626fb93-openshift-default-79bbbc8cdb-28q8h"]
Sep 18 20:16:59.557983 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:16:59.557957    2532 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7541c380-c55e-4a3b-a3c4-8596464718e8" containerName="test-curl-8728-0b2359"
Sep 18 20:16:59.557983 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:16:59.557982    2532 state_mem.go:107] "Deleted CPUSet assignment" podUID="7541c380-c55e-4a3b-a3c4-8596464718e8" containerName="test-curl-8728-0b2359"
Sep 18 20:16:59.558188 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:16:59.558066    2532 memory_manager.go:368] "RemoveStaleState removing state" podUID="7541c380-c55e-4a3b-a3c4-8596464718e8" containerName="test-curl-8728-0b2359"
Sep 18 20:16:59.561058 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:16:59.561040    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-7626fb93-openshift-default-79bbbc8cdb-28q8h"
Sep 18 20:16:59.563770 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:16:59.563747    2532 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-7626fb93-openshift-default-dockercfg-bx66q\""
Sep 18 20:16:59.563911 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:16:59.563787    2532 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Sep 18 20:16:59.572125 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:16:59.572101    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-7626fb93-openshift-default-79bbbc8cdb-28q8h"]
Sep 18 20:16:59.577866 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:16:59.577831    2532 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-8728-0b2359" podUID="7541c380-c55e-4a3b-a3c4-8596464718e8" containerName="test-curl-8728-0b2359" containerID="cri-o://4c914684cb0e401a9859c0d05214c466336f38c35aecc5fc736ea617ed233bae" gracePeriod=2
Sep 18 20:16:59.580302 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:16:59.580278    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:16:59.689273 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:16:59.689239    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/25563ed3-db7a-47d0-b2f0-e1efab3a7986-workload-certs\") pod \"e2e-ait-7626fb93-openshift-default-79bbbc8cdb-28q8h\" (UID: \"25563ed3-db7a-47d0-b2f0-e1efab3a7986\") " pod="openshift-ingress/e2e-ait-7626fb93-openshift-default-79bbbc8cdb-28q8h"
Sep 18 20:16:59.689474 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:16:59.689321    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/25563ed3-db7a-47d0-b2f0-e1efab3a7986-istio-token\") pod \"e2e-ait-7626fb93-openshift-default-79bbbc8cdb-28q8h\" (UID: \"25563ed3-db7a-47d0-b2f0-e1efab3a7986\") " pod="openshift-ingress/e2e-ait-7626fb93-openshift-default-79bbbc8cdb-28q8h"
Sep 18 20:16:59.689474 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:16:59.689397    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/25563ed3-db7a-47d0-b2f0-e1efab3a7986-credential-socket\") pod \"e2e-ait-7626fb93-openshift-default-79bbbc8cdb-28q8h\" (UID: \"25563ed3-db7a-47d0-b2f0-e1efab3a7986\") " pod="openshift-ingress/e2e-ait-7626fb93-openshift-default-79bbbc8cdb-28q8h"
Sep 18 20:16:59.689474 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:16:59.689441    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/25563ed3-db7a-47d0-b2f0-e1efab3a7986-istiod-ca-cert\") pod \"e2e-ait-7626fb93-openshift-default-79bbbc8cdb-28q8h\" (UID: \"25563ed3-db7a-47d0-b2f0-e1efab3a7986\") " pod="openshift-ingress/e2e-ait-7626fb93-openshift-default-79bbbc8cdb-28q8h"
Sep 18 20:16:59.689663 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:16:59.689501    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/25563ed3-db7a-47d0-b2f0-e1efab3a7986-istio-podinfo\") pod \"e2e-ait-7626fb93-openshift-default-79bbbc8cdb-28q8h\" (UID: \"25563ed3-db7a-47d0-b2f0-e1efab3a7986\") " pod="openshift-ingress/e2e-ait-7626fb93-openshift-default-79bbbc8cdb-28q8h"
Sep 18 20:16:59.689663 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:16:59.689547    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/25563ed3-db7a-47d0-b2f0-e1efab3a7986-istio-data\") pod \"e2e-ait-7626fb93-openshift-default-79bbbc8cdb-28q8h\" (UID: \"25563ed3-db7a-47d0-b2f0-e1efab3a7986\") " pod="openshift-ingress/e2e-ait-7626fb93-openshift-default-79bbbc8cdb-28q8h"
Sep 18 20:16:59.689663 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:16:59.689613    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/25563ed3-db7a-47d0-b2f0-e1efab3a7986-workload-socket\") pod \"e2e-ait-7626fb93-openshift-default-79bbbc8cdb-28q8h\" (UID: \"25563ed3-db7a-47d0-b2f0-e1efab3a7986\") " pod="openshift-ingress/e2e-ait-7626fb93-openshift-default-79bbbc8cdb-28q8h"
Sep 18 20:16:59.689663 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:16:59.689644    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mrclt\" (UniqueName: \"kubernetes.io/projected/25563ed3-db7a-47d0-b2f0-e1efab3a7986-kube-api-access-mrclt\") pod \"e2e-ait-7626fb93-openshift-default-79bbbc8cdb-28q8h\" (UID: \"25563ed3-db7a-47d0-b2f0-e1efab3a7986\") " pod="openshift-ingress/e2e-ait-7626fb93-openshift-default-79bbbc8cdb-28q8h"
Sep 18 20:16:59.689877 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:16:59.689683    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/25563ed3-db7a-47d0-b2f0-e1efab3a7986-istio-envoy\") pod \"e2e-ait-7626fb93-openshift-default-79bbbc8cdb-28q8h\" (UID: \"25563ed3-db7a-47d0-b2f0-e1efab3a7986\") " pod="openshift-ingress/e2e-ait-7626fb93-openshift-default-79bbbc8cdb-28q8h"
Sep 18 20:16:59.718299 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:16:59.718276    2532 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-8728-6e045f"
Sep 18 20:16:59.790881 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:16:59.790858    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-njptv\" (UniqueName: \"kubernetes.io/projected/9fc97ce8-0e91-481b-b4fb-150b89f9cadc-kube-api-access-njptv\") pod \"9fc97ce8-0e91-481b-b4fb-150b89f9cadc\" (UID: \"9fc97ce8-0e91-481b-b4fb-150b89f9cadc\") "
Sep 18 20:16:59.791044 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:16:59.791026    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/25563ed3-db7a-47d0-b2f0-e1efab3a7986-istio-data\") pod \"e2e-ait-7626fb93-openshift-default-79bbbc8cdb-28q8h\" (UID: \"25563ed3-db7a-47d0-b2f0-e1efab3a7986\") " pod="openshift-ingress/e2e-ait-7626fb93-openshift-default-79bbbc8cdb-28q8h"
Sep 18 20:16:59.791098 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:16:59.791083    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/25563ed3-db7a-47d0-b2f0-e1efab3a7986-workload-socket\") pod \"e2e-ait-7626fb93-openshift-default-79bbbc8cdb-28q8h\" (UID: \"25563ed3-db7a-47d0-b2f0-e1efab3a7986\") " pod="openshift-ingress/e2e-ait-7626fb93-openshift-default-79bbbc8cdb-28q8h"
Sep 18 20:16:59.791145 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:16:59.791120    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mrclt\" (UniqueName: \"kubernetes.io/projected/25563ed3-db7a-47d0-b2f0-e1efab3a7986-kube-api-access-mrclt\") pod \"e2e-ait-7626fb93-openshift-default-79bbbc8cdb-28q8h\" (UID: \"25563ed3-db7a-47d0-b2f0-e1efab3a7986\") " pod="openshift-ingress/e2e-ait-7626fb93-openshift-default-79bbbc8cdb-28q8h"
Sep 18 20:16:59.791191 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:16:59.791150    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/25563ed3-db7a-47d0-b2f0-e1efab3a7986-istio-envoy\") pod \"e2e-ait-7626fb93-openshift-default-79bbbc8cdb-28q8h\" (UID: \"25563ed3-db7a-47d0-b2f0-e1efab3a7986\") " pod="openshift-ingress/e2e-ait-7626fb93-openshift-default-79bbbc8cdb-28q8h"
Sep 18 20:16:59.791318 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:16:59.791291    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/25563ed3-db7a-47d0-b2f0-e1efab3a7986-workload-certs\") pod \"e2e-ait-7626fb93-openshift-default-79bbbc8cdb-28q8h\" (UID: \"25563ed3-db7a-47d0-b2f0-e1efab3a7986\") " pod="openshift-ingress/e2e-ait-7626fb93-openshift-default-79bbbc8cdb-28q8h"
Sep 18 20:16:59.791636 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:16:59.791334    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/25563ed3-db7a-47d0-b2f0-e1efab3a7986-istio-token\") pod \"e2e-ait-7626fb93-openshift-default-79bbbc8cdb-28q8h\" (UID: \"25563ed3-db7a-47d0-b2f0-e1efab3a7986\") " pod="openshift-ingress/e2e-ait-7626fb93-openshift-default-79bbbc8cdb-28q8h"
Sep 18 20:16:59.791636 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:16:59.791386    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/25563ed3-db7a-47d0-b2f0-e1efab3a7986-credential-socket\") pod \"e2e-ait-7626fb93-openshift-default-79bbbc8cdb-28q8h\" (UID: \"25563ed3-db7a-47d0-b2f0-e1efab3a7986\") " pod="openshift-ingress/e2e-ait-7626fb93-openshift-default-79bbbc8cdb-28q8h"
Sep 18 20:16:59.791636 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:16:59.791416    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/25563ed3-db7a-47d0-b2f0-e1efab3a7986-istiod-ca-cert\") pod \"e2e-ait-7626fb93-openshift-default-79bbbc8cdb-28q8h\" (UID: \"25563ed3-db7a-47d0-b2f0-e1efab3a7986\") " pod="openshift-ingress/e2e-ait-7626fb93-openshift-default-79bbbc8cdb-28q8h"
Sep 18 20:16:59.791636 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:16:59.791449    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/25563ed3-db7a-47d0-b2f0-e1efab3a7986-istio-data\") pod \"e2e-ait-7626fb93-openshift-default-79bbbc8cdb-28q8h\" (UID: \"25563ed3-db7a-47d0-b2f0-e1efab3a7986\") " pod="openshift-ingress/e2e-ait-7626fb93-openshift-default-79bbbc8cdb-28q8h"
Sep 18 20:16:59.791636 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:16:59.791462    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/25563ed3-db7a-47d0-b2f0-e1efab3a7986-istio-podinfo\") pod \"e2e-ait-7626fb93-openshift-default-79bbbc8cdb-28q8h\" (UID: \"25563ed3-db7a-47d0-b2f0-e1efab3a7986\") " pod="openshift-ingress/e2e-ait-7626fb93-openshift-default-79bbbc8cdb-28q8h"
Sep 18 20:16:59.791636 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:16:59.791519    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/25563ed3-db7a-47d0-b2f0-e1efab3a7986-workload-socket\") pod \"e2e-ait-7626fb93-openshift-default-79bbbc8cdb-28q8h\" (UID: \"25563ed3-db7a-47d0-b2f0-e1efab3a7986\") " pod="openshift-ingress/e2e-ait-7626fb93-openshift-default-79bbbc8cdb-28q8h"
Sep 18 20:16:59.791968 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:16:59.791736    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/25563ed3-db7a-47d0-b2f0-e1efab3a7986-workload-certs\") pod \"e2e-ait-7626fb93-openshift-default-79bbbc8cdb-28q8h\" (UID: \"25563ed3-db7a-47d0-b2f0-e1efab3a7986\") " pod="openshift-ingress/e2e-ait-7626fb93-openshift-default-79bbbc8cdb-28q8h"
Sep 18 20:16:59.791968 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:16:59.791765    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/25563ed3-db7a-47d0-b2f0-e1efab3a7986-credential-socket\") pod \"e2e-ait-7626fb93-openshift-default-79bbbc8cdb-28q8h\" (UID: \"25563ed3-db7a-47d0-b2f0-e1efab3a7986\") " pod="openshift-ingress/e2e-ait-7626fb93-openshift-default-79bbbc8cdb-28q8h"
Sep 18 20:16:59.792187 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:16:59.792161    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/25563ed3-db7a-47d0-b2f0-e1efab3a7986-istiod-ca-cert\") pod \"e2e-ait-7626fb93-openshift-default-79bbbc8cdb-28q8h\" (UID: \"25563ed3-db7a-47d0-b2f0-e1efab3a7986\") " pod="openshift-ingress/e2e-ait-7626fb93-openshift-default-79bbbc8cdb-28q8h"
Sep 18 20:16:59.793653 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:16:59.793624    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9fc97ce8-0e91-481b-b4fb-150b89f9cadc-kube-api-access-njptv" (OuterVolumeSpecName: "kube-api-access-njptv") pod "9fc97ce8-0e91-481b-b4fb-150b89f9cadc" (UID: "9fc97ce8-0e91-481b-b4fb-150b89f9cadc"). InnerVolumeSpecName "kube-api-access-njptv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 20:16:59.793653 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:16:59.793643    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/25563ed3-db7a-47d0-b2f0-e1efab3a7986-istio-envoy\") pod \"e2e-ait-7626fb93-openshift-default-79bbbc8cdb-28q8h\" (UID: \"25563ed3-db7a-47d0-b2f0-e1efab3a7986\") " pod="openshift-ingress/e2e-ait-7626fb93-openshift-default-79bbbc8cdb-28q8h"
Sep 18 20:16:59.794111 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:16:59.794089    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/25563ed3-db7a-47d0-b2f0-e1efab3a7986-istio-podinfo\") pod \"e2e-ait-7626fb93-openshift-default-79bbbc8cdb-28q8h\" (UID: \"25563ed3-db7a-47d0-b2f0-e1efab3a7986\") " pod="openshift-ingress/e2e-ait-7626fb93-openshift-default-79bbbc8cdb-28q8h"
Sep 18 20:16:59.803654 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:16:59.803624    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/25563ed3-db7a-47d0-b2f0-e1efab3a7986-istio-token\") pod \"e2e-ait-7626fb93-openshift-default-79bbbc8cdb-28q8h\" (UID: \"25563ed3-db7a-47d0-b2f0-e1efab3a7986\") " pod="openshift-ingress/e2e-ait-7626fb93-openshift-default-79bbbc8cdb-28q8h"
Sep 18 20:16:59.803784 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:16:59.803765    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mrclt\" (UniqueName: \"kubernetes.io/projected/25563ed3-db7a-47d0-b2f0-e1efab3a7986-kube-api-access-mrclt\") pod \"e2e-ait-7626fb93-openshift-default-79bbbc8cdb-28q8h\" (UID: \"25563ed3-db7a-47d0-b2f0-e1efab3a7986\") " pod="openshift-ingress/e2e-ait-7626fb93-openshift-default-79bbbc8cdb-28q8h"
Sep 18 20:16:59.872386 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:16:59.872313    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-7626fb93-openshift-default-79bbbc8cdb-28q8h"
Sep 18 20:16:59.892104 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:16:59.892080    2532 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-njptv\" (UniqueName: \"kubernetes.io/projected/9fc97ce8-0e91-481b-b4fb-150b89f9cadc-kube-api-access-njptv\") on node \"ip-10-0-130-81.ec2.internal\" DevicePath \"\""
Sep 18 20:16:59.899000 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:16:59.898971    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:17:00.028703 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:00.028667    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:17:00.148849 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:00.148823    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-7626fb93-openshift-default-79bbbc8cdb-28q8h"]
Sep 18 20:17:00.150501 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:17:00.150452    2532 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod25563ed3_db7a_47d0_b2f0_e1efab3a7986.slice/crio-7c416e91f5be08e335e5157ecd97d55daa5e046822a041c05e43b909ae8b50fc WatchSource:0}: Error finding container 7c416e91f5be08e335e5157ecd97d55daa5e046822a041c05e43b909ae8b50fc: Status 404 returned error can't find the container with id 7c416e91f5be08e335e5157ecd97d55daa5e046822a041c05e43b909ae8b50fc
Sep 18 20:17:00.194910 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:00.194877    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-vptdl"]
Sep 18 20:17:00.306590 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:00.306558    2532 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9fc97ce8-0e91-481b-b4fb-150b89f9cadc" path="/var/lib/kubelet/pods/9fc97ce8-0e91-481b-b4fb-150b89f9cadc/volumes"
Sep 18 20:17:00.585166 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:00.585133    2532 generic.go:356] "Generic (PLEG): container finished" podID="9fc97ce8-0e91-481b-b4fb-150b89f9cadc" containerID="3c8b3fef9f2353ec4c875172a6ea5f9db00743bdcfae28d13e6a0f696a3f1c4f" exitCode=137
Sep 18 20:17:00.585370 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:00.585176    2532 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-8728-6e045f"
Sep 18 20:17:00.585370 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:00.585229    2532 scope.go:117] "RemoveContainer" containerID="3c8b3fef9f2353ec4c875172a6ea5f9db00743bdcfae28d13e6a0f696a3f1c4f"
Sep 18 20:17:00.586706 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:00.586682    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-7626fb93-openshift-default-79bbbc8cdb-28q8h" event={"ID":"25563ed3-db7a-47d0-b2f0-e1efab3a7986","Type":"ContainerStarted","Data":"7c416e91f5be08e335e5157ecd97d55daa5e046822a041c05e43b909ae8b50fc"}
Sep 18 20:17:00.596904 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:00.596872    2532 scope.go:117] "RemoveContainer" containerID="3c8b3fef9f2353ec4c875172a6ea5f9db00743bdcfae28d13e6a0f696a3f1c4f"
Sep 18 20:17:00.597181 ip-10-0-130-81 kubenswrapper[2532]: E0918 20:17:00.597163    2532 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3c8b3fef9f2353ec4c875172a6ea5f9db00743bdcfae28d13e6a0f696a3f1c4f\": container with ID starting with 3c8b3fef9f2353ec4c875172a6ea5f9db00743bdcfae28d13e6a0f696a3f1c4f not found: ID does not exist" containerID="3c8b3fef9f2353ec4c875172a6ea5f9db00743bdcfae28d13e6a0f696a3f1c4f"
Sep 18 20:17:00.597291 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:00.597189    2532 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3c8b3fef9f2353ec4c875172a6ea5f9db00743bdcfae28d13e6a0f696a3f1c4f"} err="failed to get container status \"3c8b3fef9f2353ec4c875172a6ea5f9db00743bdcfae28d13e6a0f696a3f1c4f\": rpc error: code = NotFound desc = could not find container \"3c8b3fef9f2353ec4c875172a6ea5f9db00743bdcfae28d13e6a0f696a3f1c4f\": container with ID starting with 3c8b3fef9f2353ec4c875172a6ea5f9db00743bdcfae28d13e6a0f696a3f1c4f not found: ID does not exist"
Sep 18 20:17:00.812209 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:00.812176    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:17:01.729668 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:01.729645    2532 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-8728-0b2359"
Sep 18 20:17:01.810276 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:01.810247    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-74vn6\" (UniqueName: \"kubernetes.io/projected/7541c380-c55e-4a3b-a3c4-8596464718e8-kube-api-access-74vn6\") pod \"7541c380-c55e-4a3b-a3c4-8596464718e8\" (UID: \"7541c380-c55e-4a3b-a3c4-8596464718e8\") "
Sep 18 20:17:01.812799 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:01.812767    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7541c380-c55e-4a3b-a3c4-8596464718e8-kube-api-access-74vn6" (OuterVolumeSpecName: "kube-api-access-74vn6") pod "7541c380-c55e-4a3b-a3c4-8596464718e8" (UID: "7541c380-c55e-4a3b-a3c4-8596464718e8"). InnerVolumeSpecName "kube-api-access-74vn6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 20:17:01.911390 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:01.911300    2532 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-74vn6\" (UniqueName: \"kubernetes.io/projected/7541c380-c55e-4a3b-a3c4-8596464718e8-kube-api-access-74vn6\") on node \"ip-10-0-130-81.ec2.internal\" DevicePath \"\""
Sep 18 20:17:02.306852 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:02.306820    2532 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7541c380-c55e-4a3b-a3c4-8596464718e8" path="/var/lib/kubelet/pods/7541c380-c55e-4a3b-a3c4-8596464718e8/volumes"
Sep 18 20:17:02.603255 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:02.603160    2532 generic.go:356] "Generic (PLEG): container finished" podID="7541c380-c55e-4a3b-a3c4-8596464718e8" containerID="4c914684cb0e401a9859c0d05214c466336f38c35aecc5fc736ea617ed233bae" exitCode=137
Sep 18 20:17:02.603255 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:02.603205    2532 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-8728-0b2359"
Sep 18 20:17:02.603255 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:02.603233    2532 scope.go:117] "RemoveContainer" containerID="4c914684cb0e401a9859c0d05214c466336f38c35aecc5fc736ea617ed233bae"
Sep 18 20:17:03.084403 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:03.084381    2532 scope.go:117] "RemoveContainer" containerID="4c914684cb0e401a9859c0d05214c466336f38c35aecc5fc736ea617ed233bae"
Sep 18 20:17:03.084780 ip-10-0-130-81 kubenswrapper[2532]: E0918 20:17:03.084760    2532 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4c914684cb0e401a9859c0d05214c466336f38c35aecc5fc736ea617ed233bae\": container with ID starting with 4c914684cb0e401a9859c0d05214c466336f38c35aecc5fc736ea617ed233bae not found: ID does not exist" containerID="4c914684cb0e401a9859c0d05214c466336f38c35aecc5fc736ea617ed233bae"
Sep 18 20:17:03.084835 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:03.084792    2532 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4c914684cb0e401a9859c0d05214c466336f38c35aecc5fc736ea617ed233bae"} err="failed to get container status \"4c914684cb0e401a9859c0d05214c466336f38c35aecc5fc736ea617ed233bae\": rpc error: code = NotFound desc = could not find container \"4c914684cb0e401a9859c0d05214c466336f38c35aecc5fc736ea617ed233bae\": container with ID starting with 4c914684cb0e401a9859c0d05214c466336f38c35aecc5fc736ea617ed233bae not found: ID does not exist"
Sep 18 20:17:03.145900 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:03.145860    2532 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 18 20:17:03.146018 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:03.145948    2532 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 18 20:17:03.146018 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:03.146007    2532 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 18 20:17:03.610836 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:03.610799    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-7626fb93-openshift-default-79bbbc8cdb-28q8h" event={"ID":"25563ed3-db7a-47d0-b2f0-e1efab3a7986","Type":"ContainerStarted","Data":"9ab286c56a0710003f40894b6e156edb25a4e796d9a628a14b9bb7e9dff067c9"}
Sep 18 20:17:03.631058 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:03.631003    2532 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-7626fb93-openshift-default-79bbbc8cdb-28q8h" podStartSLOduration=1.638460449 podStartE2EDuration="4.630986489s" totalImagesPullingTime="2.99252604s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 20:16:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 20:17:03.630389655 +0000 UTC m=+955.395779056" watchObservedRunningTime="2026-09-18 20:17:03.630986489 +0000 UTC m=+955.396375887"
Sep 18 20:17:03.873226 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:03.873125    2532 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-7626fb93-openshift-default-79bbbc8cdb-28q8h"
Sep 18 20:17:04.287116 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:04.287083    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:17:04.878240 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:04.878210    2532 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-7626fb93-openshift-default-79bbbc8cdb-28q8h"
Sep 18 20:17:04.889329 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:04.889288    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:17:04.992632 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:04.992601    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:17:05.092021 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:05.091989    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:17:05.620718 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:05.620687    2532 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-7626fb93-openshift-default-79bbbc8cdb-28q8h"
Sep 18 20:17:05.621820 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:05.621798    2532 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-7626fb93-openshift-default-79bbbc8cdb-28q8h"
Sep 18 20:17:05.922975 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:05.922882    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:17:06.833623 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:06.833596    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:17:07.007681 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:07.007648    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:17:07.125285 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:07.125191    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:17:07.235048 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:07.235013    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:17:07.835752 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:07.835719    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:17:07.893686 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:07.893653    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:17:07.993374 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:07.993339    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:17:08.338449 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:08.338419    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:17:08.438765 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:08.438727    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:17:08.844989 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:08.844953    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:17:08.955510 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:08.955459    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:17:09.216461 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:09.216426    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:17:09.336425 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:09.336385    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:17:09.441370 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:09.441325    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:17:09.545718 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:09.545613    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:17:09.550721 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:09.550691    2532 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-a85f8b-openshift-default-68dd69554d-dkbnd"]
Sep 18 20:17:09.554306 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:09.554287    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-a85f8b-openshift-default-68dd69554d-dkbnd"
Sep 18 20:17:09.557042 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:09.557023    2532 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-a85f8b-openshift-default-dockercfg-dnckz\""
Sep 18 20:17:09.566581 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:09.566550    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-a85f8b-openshift-default-68dd69554d-dkbnd"]
Sep 18 20:17:09.647584 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:09.646975    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:17:09.688745 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:09.688711    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b1db05d0-08eb-47c0-ad79-65727da00166-credential-socket\") pod \"e2e-inf-b-a85f8b-openshift-default-68dd69554d-dkbnd\" (UID: \"b1db05d0-08eb-47c0-ad79-65727da00166\") " pod="openshift-ingress/e2e-inf-b-a85f8b-openshift-default-68dd69554d-dkbnd"
Sep 18 20:17:09.688952 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:09.688756    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b1db05d0-08eb-47c0-ad79-65727da00166-istio-data\") pod \"e2e-inf-b-a85f8b-openshift-default-68dd69554d-dkbnd\" (UID: \"b1db05d0-08eb-47c0-ad79-65727da00166\") " pod="openshift-ingress/e2e-inf-b-a85f8b-openshift-default-68dd69554d-dkbnd"
Sep 18 20:17:09.688952 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:09.688788    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g6dlm\" (UniqueName: \"kubernetes.io/projected/b1db05d0-08eb-47c0-ad79-65727da00166-kube-api-access-g6dlm\") pod \"e2e-inf-b-a85f8b-openshift-default-68dd69554d-dkbnd\" (UID: \"b1db05d0-08eb-47c0-ad79-65727da00166\") " pod="openshift-ingress/e2e-inf-b-a85f8b-openshift-default-68dd69554d-dkbnd"
Sep 18 20:17:09.688952 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:09.688853    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b1db05d0-08eb-47c0-ad79-65727da00166-workload-certs\") pod \"e2e-inf-b-a85f8b-openshift-default-68dd69554d-dkbnd\" (UID: \"b1db05d0-08eb-47c0-ad79-65727da00166\") " pod="openshift-ingress/e2e-inf-b-a85f8b-openshift-default-68dd69554d-dkbnd"
Sep 18 20:17:09.688952 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:09.688882    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b1db05d0-08eb-47c0-ad79-65727da00166-istio-podinfo\") pod \"e2e-inf-b-a85f8b-openshift-default-68dd69554d-dkbnd\" (UID: \"b1db05d0-08eb-47c0-ad79-65727da00166\") " pod="openshift-ingress/e2e-inf-b-a85f8b-openshift-default-68dd69554d-dkbnd"
Sep 18 20:17:09.688952 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:09.688901    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b1db05d0-08eb-47c0-ad79-65727da00166-istiod-ca-cert\") pod \"e2e-inf-b-a85f8b-openshift-default-68dd69554d-dkbnd\" (UID: \"b1db05d0-08eb-47c0-ad79-65727da00166\") " pod="openshift-ingress/e2e-inf-b-a85f8b-openshift-default-68dd69554d-dkbnd"
Sep 18 20:17:09.688952 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:09.688927    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b1db05d0-08eb-47c0-ad79-65727da00166-istio-token\") pod \"e2e-inf-b-a85f8b-openshift-default-68dd69554d-dkbnd\" (UID: \"b1db05d0-08eb-47c0-ad79-65727da00166\") " pod="openshift-ingress/e2e-inf-b-a85f8b-openshift-default-68dd69554d-dkbnd"
Sep 18 20:17:09.688952 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:09.688948    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b1db05d0-08eb-47c0-ad79-65727da00166-workload-socket\") pod \"e2e-inf-b-a85f8b-openshift-default-68dd69554d-dkbnd\" (UID: \"b1db05d0-08eb-47c0-ad79-65727da00166\") " pod="openshift-ingress/e2e-inf-b-a85f8b-openshift-default-68dd69554d-dkbnd"
Sep 18 20:17:09.689313 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:09.689022    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b1db05d0-08eb-47c0-ad79-65727da00166-istio-envoy\") pod \"e2e-inf-b-a85f8b-openshift-default-68dd69554d-dkbnd\" (UID: \"b1db05d0-08eb-47c0-ad79-65727da00166\") " pod="openshift-ingress/e2e-inf-b-a85f8b-openshift-default-68dd69554d-dkbnd"
Sep 18 20:17:09.790448 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:09.790412    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b1db05d0-08eb-47c0-ad79-65727da00166-istio-token\") pod \"e2e-inf-b-a85f8b-openshift-default-68dd69554d-dkbnd\" (UID: \"b1db05d0-08eb-47c0-ad79-65727da00166\") " pod="openshift-ingress/e2e-inf-b-a85f8b-openshift-default-68dd69554d-dkbnd"
Sep 18 20:17:09.790448 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:09.790451    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b1db05d0-08eb-47c0-ad79-65727da00166-workload-socket\") pod \"e2e-inf-b-a85f8b-openshift-default-68dd69554d-dkbnd\" (UID: \"b1db05d0-08eb-47c0-ad79-65727da00166\") " pod="openshift-ingress/e2e-inf-b-a85f8b-openshift-default-68dd69554d-dkbnd"
Sep 18 20:17:09.790715 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:09.790476    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b1db05d0-08eb-47c0-ad79-65727da00166-istio-envoy\") pod \"e2e-inf-b-a85f8b-openshift-default-68dd69554d-dkbnd\" (UID: \"b1db05d0-08eb-47c0-ad79-65727da00166\") " pod="openshift-ingress/e2e-inf-b-a85f8b-openshift-default-68dd69554d-dkbnd"
Sep 18 20:17:09.790715 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:09.790562    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b1db05d0-08eb-47c0-ad79-65727da00166-credential-socket\") pod \"e2e-inf-b-a85f8b-openshift-default-68dd69554d-dkbnd\" (UID: \"b1db05d0-08eb-47c0-ad79-65727da00166\") " pod="openshift-ingress/e2e-inf-b-a85f8b-openshift-default-68dd69554d-dkbnd"
Sep 18 20:17:09.790715 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:09.790697    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b1db05d0-08eb-47c0-ad79-65727da00166-istio-data\") pod \"e2e-inf-b-a85f8b-openshift-default-68dd69554d-dkbnd\" (UID: \"b1db05d0-08eb-47c0-ad79-65727da00166\") " pod="openshift-ingress/e2e-inf-b-a85f8b-openshift-default-68dd69554d-dkbnd"
Sep 18 20:17:09.790898 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:09.790751    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g6dlm\" (UniqueName: \"kubernetes.io/projected/b1db05d0-08eb-47c0-ad79-65727da00166-kube-api-access-g6dlm\") pod \"e2e-inf-b-a85f8b-openshift-default-68dd69554d-dkbnd\" (UID: \"b1db05d0-08eb-47c0-ad79-65727da00166\") " pod="openshift-ingress/e2e-inf-b-a85f8b-openshift-default-68dd69554d-dkbnd"
Sep 18 20:17:09.790898 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:09.790804    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b1db05d0-08eb-47c0-ad79-65727da00166-workload-certs\") pod \"e2e-inf-b-a85f8b-openshift-default-68dd69554d-dkbnd\" (UID: \"b1db05d0-08eb-47c0-ad79-65727da00166\") " pod="openshift-ingress/e2e-inf-b-a85f8b-openshift-default-68dd69554d-dkbnd"
Sep 18 20:17:09.790898 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:09.790844    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b1db05d0-08eb-47c0-ad79-65727da00166-istio-podinfo\") pod \"e2e-inf-b-a85f8b-openshift-default-68dd69554d-dkbnd\" (UID: \"b1db05d0-08eb-47c0-ad79-65727da00166\") " pod="openshift-ingress/e2e-inf-b-a85f8b-openshift-default-68dd69554d-dkbnd"
Sep 18 20:17:09.790898 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:09.790868    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b1db05d0-08eb-47c0-ad79-65727da00166-workload-socket\") pod \"e2e-inf-b-a85f8b-openshift-default-68dd69554d-dkbnd\" (UID: \"b1db05d0-08eb-47c0-ad79-65727da00166\") " pod="openshift-ingress/e2e-inf-b-a85f8b-openshift-default-68dd69554d-dkbnd"
Sep 18 20:17:09.791121 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:09.790940    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b1db05d0-08eb-47c0-ad79-65727da00166-credential-socket\") pod \"e2e-inf-b-a85f8b-openshift-default-68dd69554d-dkbnd\" (UID: \"b1db05d0-08eb-47c0-ad79-65727da00166\") " pod="openshift-ingress/e2e-inf-b-a85f8b-openshift-default-68dd69554d-dkbnd"
Sep 18 20:17:09.791121 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:09.790874    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b1db05d0-08eb-47c0-ad79-65727da00166-istiod-ca-cert\") pod \"e2e-inf-b-a85f8b-openshift-default-68dd69554d-dkbnd\" (UID: \"b1db05d0-08eb-47c0-ad79-65727da00166\") " pod="openshift-ingress/e2e-inf-b-a85f8b-openshift-default-68dd69554d-dkbnd"
Sep 18 20:17:09.791121 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:09.791073    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b1db05d0-08eb-47c0-ad79-65727da00166-istio-data\") pod \"e2e-inf-b-a85f8b-openshift-default-68dd69554d-dkbnd\" (UID: \"b1db05d0-08eb-47c0-ad79-65727da00166\") " pod="openshift-ingress/e2e-inf-b-a85f8b-openshift-default-68dd69554d-dkbnd"
Sep 18 20:17:09.791288 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:09.791187    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b1db05d0-08eb-47c0-ad79-65727da00166-workload-certs\") pod \"e2e-inf-b-a85f8b-openshift-default-68dd69554d-dkbnd\" (UID: \"b1db05d0-08eb-47c0-ad79-65727da00166\") " pod="openshift-ingress/e2e-inf-b-a85f8b-openshift-default-68dd69554d-dkbnd"
Sep 18 20:17:09.791564 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:09.791544    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b1db05d0-08eb-47c0-ad79-65727da00166-istiod-ca-cert\") pod \"e2e-inf-b-a85f8b-openshift-default-68dd69554d-dkbnd\" (UID: \"b1db05d0-08eb-47c0-ad79-65727da00166\") " pod="openshift-ingress/e2e-inf-b-a85f8b-openshift-default-68dd69554d-dkbnd"
Sep 18 20:17:09.793253 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:09.793222    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b1db05d0-08eb-47c0-ad79-65727da00166-istio-envoy\") pod \"e2e-inf-b-a85f8b-openshift-default-68dd69554d-dkbnd\" (UID: \"b1db05d0-08eb-47c0-ad79-65727da00166\") " pod="openshift-ingress/e2e-inf-b-a85f8b-openshift-default-68dd69554d-dkbnd"
Sep 18 20:17:09.793375 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:09.793358    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b1db05d0-08eb-47c0-ad79-65727da00166-istio-podinfo\") pod \"e2e-inf-b-a85f8b-openshift-default-68dd69554d-dkbnd\" (UID: \"b1db05d0-08eb-47c0-ad79-65727da00166\") " pod="openshift-ingress/e2e-inf-b-a85f8b-openshift-default-68dd69554d-dkbnd"
Sep 18 20:17:09.798776 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:09.798720    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b1db05d0-08eb-47c0-ad79-65727da00166-istio-token\") pod \"e2e-inf-b-a85f8b-openshift-default-68dd69554d-dkbnd\" (UID: \"b1db05d0-08eb-47c0-ad79-65727da00166\") " pod="openshift-ingress/e2e-inf-b-a85f8b-openshift-default-68dd69554d-dkbnd"
Sep 18 20:17:09.798964 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:09.798944    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g6dlm\" (UniqueName: \"kubernetes.io/projected/b1db05d0-08eb-47c0-ad79-65727da00166-kube-api-access-g6dlm\") pod \"e2e-inf-b-a85f8b-openshift-default-68dd69554d-dkbnd\" (UID: \"b1db05d0-08eb-47c0-ad79-65727da00166\") " pod="openshift-ingress/e2e-inf-b-a85f8b-openshift-default-68dd69554d-dkbnd"
Sep 18 20:17:09.869312 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:09.869282    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-a85f8b-openshift-default-68dd69554d-dkbnd"
Sep 18 20:17:09.925264 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:09.924855    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:17:10.152560 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:10.152526    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-a85f8b-openshift-default-68dd69554d-dkbnd"]
Sep 18 20:17:10.152911 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:17:10.152887    2532 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb1db05d0_08eb_47c0_ad79_65727da00166.slice/crio-8ae439f122df595798f6e1394818af119a89e6c16d48d842a0d5a407e997611d WatchSource:0}: Error finding container 8ae439f122df595798f6e1394818af119a89e6c16d48d842a0d5a407e997611d: Status 404 returned error can't find the container with id 8ae439f122df595798f6e1394818af119a89e6c16d48d842a0d5a407e997611d
Sep 18 20:17:10.156403 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:10.156372    2532 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 18 20:17:10.156475 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:10.156437    2532 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 18 20:17:10.156475 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:10.156471    2532 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 18 20:17:10.323545 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:10.323517    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:17:10.631681 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:10.631650    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:17:10.660379 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:10.660339    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-a85f8b-openshift-default-68dd69554d-dkbnd" event={"ID":"b1db05d0-08eb-47c0-ad79-65727da00166","Type":"ContainerStarted","Data":"a3762b002234f42683cf694e1366356522186f2c6a113a3464bd79ad83b50397"}
Sep 18 20:17:10.660582 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:10.660396    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-a85f8b-openshift-default-68dd69554d-dkbnd" event={"ID":"b1db05d0-08eb-47c0-ad79-65727da00166","Type":"ContainerStarted","Data":"8ae439f122df595798f6e1394818af119a89e6c16d48d842a0d5a407e997611d"}
Sep 18 20:17:10.681748 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:10.681701    2532 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-a85f8b-openshift-default-68dd69554d-dkbnd" podStartSLOduration=1.681685111 podStartE2EDuration="1.681685111s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 20:17:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 20:17:10.679646508 +0000 UTC m=+962.445035908" watchObservedRunningTime="2026-09-18 20:17:10.681685111 +0000 UTC m=+962.447074510"
Sep 18 20:17:10.869849 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:10.869817    2532 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-a85f8b-openshift-default-68dd69554d-dkbnd"
Sep 18 20:17:11.513611 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:11.513573    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:17:11.875432 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:11.875363    2532 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-a85f8b-openshift-default-68dd69554d-dkbnd"
Sep 18 20:17:12.205840 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:12.205815    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:17:12.550652 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:12.550559    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:17:12.639082 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:12.639044    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:17:12.670162 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:12.670133    2532 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-a85f8b-openshift-default-68dd69554d-dkbnd"
Sep 18 20:17:12.671050 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:12.671033    2532 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-a85f8b-openshift-default-68dd69554d-dkbnd"
Sep 18 20:17:13.123339 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:13.123304    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:17:13.702782 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:13.702744    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:17:13.968704 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:13.968673    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:17:14.079371 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:14.079337    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:17:14.168945 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:14.168916    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:17:14.299406 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:14.299322    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:17:14.541221 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:14.541177    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:17:15.144864 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:15.144837    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:17:16.034087 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:16.034052    2532 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-a85f8b-75d49bbcb8-wj8lj"]
Sep 18 20:17:16.038657 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:16.038640    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a85f8b-75d49bbcb8-wj8lj"
Sep 18 20:17:16.041538 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:16.041508    2532 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-b-a85f8b-serving-cert\""
Sep 18 20:17:16.042619 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:16.042600    2532 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Sep 18 20:17:16.042843 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:16.042827    2532 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-fhqv4\""
Sep 18 20:17:16.042843 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:16.042836    2532 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Sep 18 20:17:16.042972 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:16.042830    2532 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 18 20:17:16.053667 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:16.053630    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-a85f8b-75d49bbcb8-wj8lj"]
Sep 18 20:17:16.054038 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:16.054017    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b953a796-0402-46c7-bef7-696262fcc8fc-metrics-certs\") pod \"maas-api-e2e-inf-b-a85f8b-75d49bbcb8-wj8lj\" (UID: \"b953a796-0402-46c7-bef7-696262fcc8fc\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a85f8b-75d49bbcb8-wj8lj"
Sep 18 20:17:16.054162 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:16.054052    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b953a796-0402-46c7-bef7-696262fcc8fc-maas-api-tls\") pod \"maas-api-e2e-inf-b-a85f8b-75d49bbcb8-wj8lj\" (UID: \"b953a796-0402-46c7-bef7-696262fcc8fc\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a85f8b-75d49bbcb8-wj8lj"
Sep 18 20:17:16.054162 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:16.054113    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b953a796-0402-46c7-bef7-696262fcc8fc-openshift-service-ca\") pod \"maas-api-e2e-inf-b-a85f8b-75d49bbcb8-wj8lj\" (UID: \"b953a796-0402-46c7-bef7-696262fcc8fc\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a85f8b-75d49bbcb8-wj8lj"
Sep 18 20:17:16.054281 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:16.054160    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jq2bq\" (UniqueName: \"kubernetes.io/projected/b953a796-0402-46c7-bef7-696262fcc8fc-kube-api-access-jq2bq\") pod \"maas-api-e2e-inf-b-a85f8b-75d49bbcb8-wj8lj\" (UID: \"b953a796-0402-46c7-bef7-696262fcc8fc\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a85f8b-75d49bbcb8-wj8lj"
Sep 18 20:17:16.098965 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:16.098918    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:17:16.154843 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:16.154803    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b953a796-0402-46c7-bef7-696262fcc8fc-metrics-certs\") pod \"maas-api-e2e-inf-b-a85f8b-75d49bbcb8-wj8lj\" (UID: \"b953a796-0402-46c7-bef7-696262fcc8fc\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a85f8b-75d49bbcb8-wj8lj"
Sep 18 20:17:16.155042 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:16.154851    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b953a796-0402-46c7-bef7-696262fcc8fc-maas-api-tls\") pod \"maas-api-e2e-inf-b-a85f8b-75d49bbcb8-wj8lj\" (UID: \"b953a796-0402-46c7-bef7-696262fcc8fc\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a85f8b-75d49bbcb8-wj8lj"
Sep 18 20:17:16.155042 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:16.154891    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b953a796-0402-46c7-bef7-696262fcc8fc-openshift-service-ca\") pod \"maas-api-e2e-inf-b-a85f8b-75d49bbcb8-wj8lj\" (UID: \"b953a796-0402-46c7-bef7-696262fcc8fc\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a85f8b-75d49bbcb8-wj8lj"
Sep 18 20:17:16.155042 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:16.154916    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jq2bq\" (UniqueName: \"kubernetes.io/projected/b953a796-0402-46c7-bef7-696262fcc8fc-kube-api-access-jq2bq\") pod \"maas-api-e2e-inf-b-a85f8b-75d49bbcb8-wj8lj\" (UID: \"b953a796-0402-46c7-bef7-696262fcc8fc\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a85f8b-75d49bbcb8-wj8lj"
Sep 18 20:17:16.155042 ip-10-0-130-81 kubenswrapper[2532]: E0918 20:17:16.155014    2532 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-b-a85f8b-serving-cert: secret "maas-api-e2e-inf-b-a85f8b-serving-cert" not found
Sep 18 20:17:16.155271 ip-10-0-130-81 kubenswrapper[2532]: E0918 20:17:16.155094    2532 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b953a796-0402-46c7-bef7-696262fcc8fc-maas-api-tls podName:b953a796-0402-46c7-bef7-696262fcc8fc nodeName:}" failed. No retries permitted until 2026-09-18 20:17:16.655072574 +0000 UTC m=+968.420461966 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/b953a796-0402-46c7-bef7-696262fcc8fc-maas-api-tls") pod "maas-api-e2e-inf-b-a85f8b-75d49bbcb8-wj8lj" (UID: "b953a796-0402-46c7-bef7-696262fcc8fc") : secret "maas-api-e2e-inf-b-a85f8b-serving-cert" not found
Sep 18 20:17:16.155723 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:16.155706    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b953a796-0402-46c7-bef7-696262fcc8fc-openshift-service-ca\") pod \"maas-api-e2e-inf-b-a85f8b-75d49bbcb8-wj8lj\" (UID: \"b953a796-0402-46c7-bef7-696262fcc8fc\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a85f8b-75d49bbcb8-wj8lj"
Sep 18 20:17:16.157346 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:16.157325    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b953a796-0402-46c7-bef7-696262fcc8fc-metrics-certs\") pod \"maas-api-e2e-inf-b-a85f8b-75d49bbcb8-wj8lj\" (UID: \"b953a796-0402-46c7-bef7-696262fcc8fc\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a85f8b-75d49bbcb8-wj8lj"
Sep 18 20:17:16.163099 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:16.163079    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jq2bq\" (UniqueName: \"kubernetes.io/projected/b953a796-0402-46c7-bef7-696262fcc8fc-kube-api-access-jq2bq\") pod \"maas-api-e2e-inf-b-a85f8b-75d49bbcb8-wj8lj\" (UID: \"b953a796-0402-46c7-bef7-696262fcc8fc\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a85f8b-75d49bbcb8-wj8lj"
Sep 18 20:17:16.200263 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:16.200235    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:17:16.303626 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:16.303536    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:17:16.543422 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:16.543396    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:17:16.646709 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:16.646627    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:17:16.659041 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:16.659010    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b953a796-0402-46c7-bef7-696262fcc8fc-maas-api-tls\") pod \"maas-api-e2e-inf-b-a85f8b-75d49bbcb8-wj8lj\" (UID: \"b953a796-0402-46c7-bef7-696262fcc8fc\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a85f8b-75d49bbcb8-wj8lj"
Sep 18 20:17:16.661479 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:16.661458    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b953a796-0402-46c7-bef7-696262fcc8fc-maas-api-tls\") pod \"maas-api-e2e-inf-b-a85f8b-75d49bbcb8-wj8lj\" (UID: \"b953a796-0402-46c7-bef7-696262fcc8fc\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a85f8b-75d49bbcb8-wj8lj"
Sep 18 20:17:16.923586 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:16.923505    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:17:16.949529 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:16.949481    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a85f8b-75d49bbcb8-wj8lj"
Sep 18 20:17:17.228256 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:17.228231    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-a85f8b-75d49bbcb8-wj8lj"]
Sep 18 20:17:17.229668 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:17:17.229643    2532 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb953a796_0402_46c7_bef7_696262fcc8fc.slice/crio-f11db38c9a1d29b1204b2caa2a46c8aba60b88a1a1964ad748046430bb061ef5 WatchSource:0}: Error finding container f11db38c9a1d29b1204b2caa2a46c8aba60b88a1a1964ad748046430bb061ef5: Status 404 returned error can't find the container with id f11db38c9a1d29b1204b2caa2a46c8aba60b88a1a1964ad748046430bb061ef5
Sep 18 20:17:17.515639 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:17.515563    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-vptdl"]
Sep 18 20:17:17.639430 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:17.639396    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:17:17.702884 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:17.702803    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a85f8b-75d49bbcb8-wj8lj" event={"ID":"b953a796-0402-46c7-bef7-696262fcc8fc","Type":"ContainerStarted","Data":"f11db38c9a1d29b1204b2caa2a46c8aba60b88a1a1964ad748046430bb061ef5"}
Sep 18 20:17:18.444747 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:18.444703    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:17:19.191664 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:19.191619    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:17:20.173583 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:20.173473    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:17:20.235764 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:20.235726    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:17:20.735347 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:20.735310    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a85f8b-75d49bbcb8-wj8lj" event={"ID":"b953a796-0402-46c7-bef7-696262fcc8fc","Type":"ContainerStarted","Data":"3868648a51b2df5ffc6d41f522eaab621010533b181cdb69678a62b361e70c88"}
Sep 18 20:17:20.735584 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:20.735380    2532 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a85f8b-75d49bbcb8-wj8lj"
Sep 18 20:17:20.752949 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:20.752899    2532 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a85f8b-75d49bbcb8-wj8lj" podStartSLOduration=2.251628827 podStartE2EDuration="4.752885085s" totalImagesPullingTime="2.501256258s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 20:17:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 20:17:20.751564552 +0000 UTC m=+972.516953952" watchObservedRunningTime="2026-09-18 20:17:20.752885085 +0000 UTC m=+972.518274553"
Sep 18 20:17:26.746992 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:26.746961    2532 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a85f8b-75d49bbcb8-wj8lj"
Sep 18 20:17:27.773440 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:27.773408    2532 generic.go:356] "Generic (PLEG): container finished" podID="25563ed3-db7a-47d0-b2f0-e1efab3a7986" containerID="9ab286c56a0710003f40894b6e156edb25a4e796d9a628a14b9bb7e9dff067c9" exitCode=0
Sep 18 20:17:27.773839 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:27.773503    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-7626fb93-openshift-default-79bbbc8cdb-28q8h" event={"ID":"25563ed3-db7a-47d0-b2f0-e1efab3a7986","Type":"ContainerDied","Data":"9ab286c56a0710003f40894b6e156edb25a4e796d9a628a14b9bb7e9dff067c9"}
Sep 18 20:17:27.773906 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:27.773891    2532 scope.go:117] "RemoveContainer" containerID="9ab286c56a0710003f40894b6e156edb25a4e796d9a628a14b9bb7e9dff067c9"
Sep 18 20:17:27.775037 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:27.775008    2532 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 18 20:17:27.775100 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:27.775083    2532 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 18 20:17:27.775139 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:27.775117    2532 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 18 20:17:28.719500 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:28.719411    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:17:28.780425 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:28.780386    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-7626fb93-openshift-default-79bbbc8cdb-28q8h" event={"ID":"25563ed3-db7a-47d0-b2f0-e1efab3a7986","Type":"ContainerStarted","Data":"b9ea7929ffbb461f6398db41fd0da0a96175ec85e1a437a988b4cd39e5941283"}
Sep 18 20:17:28.872807 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:28.872774    2532 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-7626fb93-openshift-default-79bbbc8cdb-28q8h"
Sep 18 20:17:28.872981 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:28.872877    2532 patch_prober.go:28] interesting pod/e2e-ait-7626fb93-openshift-default-79bbbc8cdb-28q8h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.43:15021/healthz/ready\": dial tcp 10.133.0.43:15021: connect: connection refused" start-of-body=
Sep 18 20:17:28.872981 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:28.872910    2532 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-7626fb93-openshift-default-79bbbc8cdb-28q8h" podUID="25563ed3-db7a-47d0-b2f0-e1efab3a7986" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.43:15021/healthz/ready\": dial tcp 10.133.0.43:15021: connect: connection refused"
Sep 18 20:17:29.873087 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:29.873051    2532 patch_prober.go:28] interesting pod/e2e-ait-7626fb93-openshift-default-79bbbc8cdb-28q8h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.43:15021/healthz/ready\": dial tcp 10.133.0.43:15021: connect: connection refused" start-of-body=
Sep 18 20:17:29.873507 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:29.873113    2532 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-7626fb93-openshift-default-79bbbc8cdb-28q8h" podUID="25563ed3-db7a-47d0-b2f0-e1efab3a7986" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.43:15021/healthz/ready\": dial tcp 10.133.0.43:15021: connect: connection refused"
Sep 18 20:17:30.119271 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:30.119230    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:17:30.872709 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:30.872673    2532 patch_prober.go:28] interesting pod/e2e-ait-7626fb93-openshift-default-79bbbc8cdb-28q8h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.43:15021/healthz/ready\": dial tcp 10.133.0.43:15021: connect: connection refused" start-of-body=
Sep 18 20:17:30.872925 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:30.872744    2532 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-7626fb93-openshift-default-79bbbc8cdb-28q8h" podUID="25563ed3-db7a-47d0-b2f0-e1efab3a7986" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.43:15021/healthz/ready\": dial tcp 10.133.0.43:15021: connect: connection refused"
Sep 18 20:17:30.893157 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:30.893118    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:17:31.301602 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:31.301565    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-vptdl"]
Sep 18 20:17:31.527506 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:31.527418    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:17:31.872672 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:31.872631    2532 patch_prober.go:28] interesting pod/e2e-ait-7626fb93-openshift-default-79bbbc8cdb-28q8h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.43:15021/healthz/ready\": dial tcp 10.133.0.43:15021: connect: connection refused" start-of-body=
Sep 18 20:17:31.872862 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:31.872706    2532 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-7626fb93-openshift-default-79bbbc8cdb-28q8h" podUID="25563ed3-db7a-47d0-b2f0-e1efab3a7986" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.43:15021/healthz/ready\": dial tcp 10.133.0.43:15021: connect: connection refused"
Sep 18 20:17:32.148833 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:32.148718    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:17:32.872859 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:32.872813    2532 patch_prober.go:28] interesting pod/e2e-ait-7626fb93-openshift-default-79bbbc8cdb-28q8h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.43:15021/healthz/ready\": dial tcp 10.133.0.43:15021: connect: connection refused" start-of-body=
Sep 18 20:17:32.873049 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:32.872877    2532 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-7626fb93-openshift-default-79bbbc8cdb-28q8h" podUID="25563ed3-db7a-47d0-b2f0-e1efab3a7986" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.43:15021/healthz/ready\": dial tcp 10.133.0.43:15021: connect: connection refused"
Sep 18 20:17:32.919925 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:32.919884    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:17:33.614526 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:33.614245    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:17:33.873199 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:33.873107    2532 patch_prober.go:28] interesting pod/e2e-ait-7626fb93-openshift-default-79bbbc8cdb-28q8h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.43:15021/healthz/ready\": dial tcp 10.133.0.43:15021: connect: connection refused" start-of-body=
Sep 18 20:17:33.873199 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:33.873170    2532 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-7626fb93-openshift-default-79bbbc8cdb-28q8h" podUID="25563ed3-db7a-47d0-b2f0-e1efab3a7986" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.43:15021/healthz/ready\": dial tcp 10.133.0.43:15021: connect: connection refused"
Sep 18 20:17:34.247808 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:34.247778    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:17:34.804271 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:34.804236    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:17:34.876975 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:34.876943    2532 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-7626fb93-openshift-default-79bbbc8cdb-28q8h"
Sep 18 20:17:34.877400 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:34.877374    2532 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-7626fb93-openshift-default-79bbbc8cdb-28q8h"
Sep 18 20:17:34.878084 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:34.878062    2532 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-7626fb93-openshift-default-79bbbc8cdb-28q8h"
Sep 18 20:17:35.609337 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:35.609302    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:17:35.819902 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:35.819817    2532 generic.go:356] "Generic (PLEG): container finished" podID="b1db05d0-08eb-47c0-ad79-65727da00166" containerID="a3762b002234f42683cf694e1366356522186f2c6a113a3464bd79ad83b50397" exitCode=0
Sep 18 20:17:35.819902 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:35.819892    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-a85f8b-openshift-default-68dd69554d-dkbnd" event={"ID":"b1db05d0-08eb-47c0-ad79-65727da00166","Type":"ContainerDied","Data":"a3762b002234f42683cf694e1366356522186f2c6a113a3464bd79ad83b50397"}
Sep 18 20:17:35.820382 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:35.820367    2532 scope.go:117] "RemoveContainer" containerID="a3762b002234f42683cf694e1366356522186f2c6a113a3464bd79ad83b50397"
Sep 18 20:17:35.821404 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:35.821375    2532 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 18 20:17:35.821526 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:35.821470    2532 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 18 20:17:35.821613 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:35.821594    2532 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 18 20:17:36.400925 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:36.400892    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:17:36.830780 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:36.830742    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-a85f8b-openshift-default-68dd69554d-dkbnd" event={"ID":"b1db05d0-08eb-47c0-ad79-65727da00166","Type":"ContainerStarted","Data":"0a2c5d67da837e175e585a1734b3f5c8a27d506c9e00170a7d1cb0454739476f"}
Sep 18 20:17:36.870092 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:36.870063    2532 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-a85f8b-openshift-default-68dd69554d-dkbnd"
Sep 18 20:17:36.870246 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:36.870232    2532 patch_prober.go:28] interesting pod/e2e-inf-b-a85f8b-openshift-default-68dd69554d-dkbnd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused" start-of-body=
Sep 18 20:17:36.870289 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:36.870269    2532 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-a85f8b-openshift-default-68dd69554d-dkbnd" podUID="b1db05d0-08eb-47c0-ad79-65727da00166" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused"
Sep 18 20:17:37.110791 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:37.110644    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:17:37.446962 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:37.446886    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:17:37.671715 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:37.671678    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:17:37.766192 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:37.766159    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:17:37.865754 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:37.865721    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:17:37.869759 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:37.869734    2532 patch_prober.go:28] interesting pod/e2e-inf-b-a85f8b-openshift-default-68dd69554d-dkbnd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused" start-of-body=
Sep 18 20:17:37.869881 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:37.869781    2532 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-a85f8b-openshift-default-68dd69554d-dkbnd" podUID="b1db05d0-08eb-47c0-ad79-65727da00166" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused"
Sep 18 20:17:37.925955 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:37.925914    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:17:38.036866 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:38.036708    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:17:38.037965 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:38.037937    2532 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-7626fb93-openshift-default-79bbbc8cdb-28q8h"]
Sep 18 20:17:38.038281 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:38.038250    2532 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-7626fb93-openshift-default-79bbbc8cdb-28q8h" podUID="25563ed3-db7a-47d0-b2f0-e1efab3a7986" containerName="istio-proxy" containerID="cri-o://b9ea7929ffbb461f6398db41fd0da0a96175ec85e1a437a988b4cd39e5941283" gracePeriod=30
Sep 18 20:17:38.511974 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:38.511932    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:17:38.589010 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:38.588977    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:17:38.714160 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:38.714123    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:17:38.823091 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:38.822922    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:17:38.870507 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:38.870453    2532 patch_prober.go:28] interesting pod/e2e-inf-b-a85f8b-openshift-default-68dd69554d-dkbnd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused" start-of-body=
Sep 18 20:17:38.870894 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:38.870528    2532 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-a85f8b-openshift-default-68dd69554d-dkbnd" podUID="b1db05d0-08eb-47c0-ad79-65727da00166" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused"
Sep 18 20:17:38.986838 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:38.986799    2532 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-migrate-6d00b9e3-gw-openshift-default-c4d74979b-ncsh6"]
Sep 18 20:17:38.989562 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:38.989537    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-6d00b9e3-gw-openshift-default-c4d74979b-ncsh6"
Sep 18 20:17:38.992208 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:38.992189    2532 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-migrate-6d00b9e3-gw-openshift-default-dockercfg-gh44l\""
Sep 18 20:17:39.012220 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:39.012191    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-6d00b9e3-gw-openshift-default-c4d74979b-ncsh6"]
Sep 18 20:17:39.025559 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:39.025527    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:17:39.082264 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:39.082165    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7e11376c-93ad-497f-aab6-cc0b000ff5f7-istio-data\") pod \"e2e-migrate-6d00b9e3-gw-openshift-default-c4d74979b-ncsh6\" (UID: \"7e11376c-93ad-497f-aab6-cc0b000ff5f7\") " pod="openshift-ingress/e2e-migrate-6d00b9e3-gw-openshift-default-c4d74979b-ncsh6"
Sep 18 20:17:39.082264 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:39.082225    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7e11376c-93ad-497f-aab6-cc0b000ff5f7-istio-envoy\") pod \"e2e-migrate-6d00b9e3-gw-openshift-default-c4d74979b-ncsh6\" (UID: \"7e11376c-93ad-497f-aab6-cc0b000ff5f7\") " pod="openshift-ingress/e2e-migrate-6d00b9e3-gw-openshift-default-c4d74979b-ncsh6"
Sep 18 20:17:39.082264 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:39.082257    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7e11376c-93ad-497f-aab6-cc0b000ff5f7-istiod-ca-cert\") pod \"e2e-migrate-6d00b9e3-gw-openshift-default-c4d74979b-ncsh6\" (UID: \"7e11376c-93ad-497f-aab6-cc0b000ff5f7\") " pod="openshift-ingress/e2e-migrate-6d00b9e3-gw-openshift-default-c4d74979b-ncsh6"
Sep 18 20:17:39.082560 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:39.082287    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7e11376c-93ad-497f-aab6-cc0b000ff5f7-istio-podinfo\") pod \"e2e-migrate-6d00b9e3-gw-openshift-default-c4d74979b-ncsh6\" (UID: \"7e11376c-93ad-497f-aab6-cc0b000ff5f7\") " pod="openshift-ingress/e2e-migrate-6d00b9e3-gw-openshift-default-c4d74979b-ncsh6"
Sep 18 20:17:39.082560 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:39.082371    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7e11376c-93ad-497f-aab6-cc0b000ff5f7-istio-token\") pod \"e2e-migrate-6d00b9e3-gw-openshift-default-c4d74979b-ncsh6\" (UID: \"7e11376c-93ad-497f-aab6-cc0b000ff5f7\") " pod="openshift-ingress/e2e-migrate-6d00b9e3-gw-openshift-default-c4d74979b-ncsh6"
Sep 18 20:17:39.082560 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:39.082450    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7e11376c-93ad-497f-aab6-cc0b000ff5f7-credential-socket\") pod \"e2e-migrate-6d00b9e3-gw-openshift-default-c4d74979b-ncsh6\" (UID: \"7e11376c-93ad-497f-aab6-cc0b000ff5f7\") " pod="openshift-ingress/e2e-migrate-6d00b9e3-gw-openshift-default-c4d74979b-ncsh6"
Sep 18 20:17:39.082560 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:39.082533    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7e11376c-93ad-497f-aab6-cc0b000ff5f7-workload-certs\") pod \"e2e-migrate-6d00b9e3-gw-openshift-default-c4d74979b-ncsh6\" (UID: \"7e11376c-93ad-497f-aab6-cc0b000ff5f7\") " pod="openshift-ingress/e2e-migrate-6d00b9e3-gw-openshift-default-c4d74979b-ncsh6"
Sep 18 20:17:39.082560 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:39.082553    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7fbdm\" (UniqueName: \"kubernetes.io/projected/7e11376c-93ad-497f-aab6-cc0b000ff5f7-kube-api-access-7fbdm\") pod \"e2e-migrate-6d00b9e3-gw-openshift-default-c4d74979b-ncsh6\" (UID: \"7e11376c-93ad-497f-aab6-cc0b000ff5f7\") " pod="openshift-ingress/e2e-migrate-6d00b9e3-gw-openshift-default-c4d74979b-ncsh6"
Sep 18 20:17:39.082816 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:39.082578    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7e11376c-93ad-497f-aab6-cc0b000ff5f7-workload-socket\") pod \"e2e-migrate-6d00b9e3-gw-openshift-default-c4d74979b-ncsh6\" (UID: \"7e11376c-93ad-497f-aab6-cc0b000ff5f7\") " pod="openshift-ingress/e2e-migrate-6d00b9e3-gw-openshift-default-c4d74979b-ncsh6"
Sep 18 20:17:39.183546 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:39.183509    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7e11376c-93ad-497f-aab6-cc0b000ff5f7-credential-socket\") pod \"e2e-migrate-6d00b9e3-gw-openshift-default-c4d74979b-ncsh6\" (UID: \"7e11376c-93ad-497f-aab6-cc0b000ff5f7\") " pod="openshift-ingress/e2e-migrate-6d00b9e3-gw-openshift-default-c4d74979b-ncsh6"
Sep 18 20:17:39.183739 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:39.183568    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7e11376c-93ad-497f-aab6-cc0b000ff5f7-workload-certs\") pod \"e2e-migrate-6d00b9e3-gw-openshift-default-c4d74979b-ncsh6\" (UID: \"7e11376c-93ad-497f-aab6-cc0b000ff5f7\") " pod="openshift-ingress/e2e-migrate-6d00b9e3-gw-openshift-default-c4d74979b-ncsh6"
Sep 18 20:17:39.183739 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:39.183589    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7fbdm\" (UniqueName: \"kubernetes.io/projected/7e11376c-93ad-497f-aab6-cc0b000ff5f7-kube-api-access-7fbdm\") pod \"e2e-migrate-6d00b9e3-gw-openshift-default-c4d74979b-ncsh6\" (UID: \"7e11376c-93ad-497f-aab6-cc0b000ff5f7\") " pod="openshift-ingress/e2e-migrate-6d00b9e3-gw-openshift-default-c4d74979b-ncsh6"
Sep 18 20:17:39.183739 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:39.183623    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7e11376c-93ad-497f-aab6-cc0b000ff5f7-workload-socket\") pod \"e2e-migrate-6d00b9e3-gw-openshift-default-c4d74979b-ncsh6\" (UID: \"7e11376c-93ad-497f-aab6-cc0b000ff5f7\") " pod="openshift-ingress/e2e-migrate-6d00b9e3-gw-openshift-default-c4d74979b-ncsh6"
Sep 18 20:17:39.183739 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:39.183676    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7e11376c-93ad-497f-aab6-cc0b000ff5f7-istio-data\") pod \"e2e-migrate-6d00b9e3-gw-openshift-default-c4d74979b-ncsh6\" (UID: \"7e11376c-93ad-497f-aab6-cc0b000ff5f7\") " pod="openshift-ingress/e2e-migrate-6d00b9e3-gw-openshift-default-c4d74979b-ncsh6"
Sep 18 20:17:39.183739 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:39.183708    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7e11376c-93ad-497f-aab6-cc0b000ff5f7-istio-envoy\") pod \"e2e-migrate-6d00b9e3-gw-openshift-default-c4d74979b-ncsh6\" (UID: \"7e11376c-93ad-497f-aab6-cc0b000ff5f7\") " pod="openshift-ingress/e2e-migrate-6d00b9e3-gw-openshift-default-c4d74979b-ncsh6"
Sep 18 20:17:39.183739 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:39.183731    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7e11376c-93ad-497f-aab6-cc0b000ff5f7-istiod-ca-cert\") pod \"e2e-migrate-6d00b9e3-gw-openshift-default-c4d74979b-ncsh6\" (UID: \"7e11376c-93ad-497f-aab6-cc0b000ff5f7\") " pod="openshift-ingress/e2e-migrate-6d00b9e3-gw-openshift-default-c4d74979b-ncsh6"
Sep 18 20:17:39.184047 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:39.183760    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7e11376c-93ad-497f-aab6-cc0b000ff5f7-istio-podinfo\") pod \"e2e-migrate-6d00b9e3-gw-openshift-default-c4d74979b-ncsh6\" (UID: \"7e11376c-93ad-497f-aab6-cc0b000ff5f7\") " pod="openshift-ingress/e2e-migrate-6d00b9e3-gw-openshift-default-c4d74979b-ncsh6"
Sep 18 20:17:39.184047 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:39.183797    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7e11376c-93ad-497f-aab6-cc0b000ff5f7-istio-token\") pod \"e2e-migrate-6d00b9e3-gw-openshift-default-c4d74979b-ncsh6\" (UID: \"7e11376c-93ad-497f-aab6-cc0b000ff5f7\") " pod="openshift-ingress/e2e-migrate-6d00b9e3-gw-openshift-default-c4d74979b-ncsh6"
Sep 18 20:17:39.184047 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:39.183995    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7e11376c-93ad-497f-aab6-cc0b000ff5f7-credential-socket\") pod \"e2e-migrate-6d00b9e3-gw-openshift-default-c4d74979b-ncsh6\" (UID: \"7e11376c-93ad-497f-aab6-cc0b000ff5f7\") " pod="openshift-ingress/e2e-migrate-6d00b9e3-gw-openshift-default-c4d74979b-ncsh6"
Sep 18 20:17:39.184047 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:39.184014    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7e11376c-93ad-497f-aab6-cc0b000ff5f7-workload-socket\") pod \"e2e-migrate-6d00b9e3-gw-openshift-default-c4d74979b-ncsh6\" (UID: \"7e11376c-93ad-497f-aab6-cc0b000ff5f7\") " pod="openshift-ingress/e2e-migrate-6d00b9e3-gw-openshift-default-c4d74979b-ncsh6"
Sep 18 20:17:39.184321 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:39.184298    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7e11376c-93ad-497f-aab6-cc0b000ff5f7-istio-data\") pod \"e2e-migrate-6d00b9e3-gw-openshift-default-c4d74979b-ncsh6\" (UID: \"7e11376c-93ad-497f-aab6-cc0b000ff5f7\") " pod="openshift-ingress/e2e-migrate-6d00b9e3-gw-openshift-default-c4d74979b-ncsh6"
Sep 18 20:17:39.184505 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:39.184464    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7e11376c-93ad-497f-aab6-cc0b000ff5f7-workload-certs\") pod \"e2e-migrate-6d00b9e3-gw-openshift-default-c4d74979b-ncsh6\" (UID: \"7e11376c-93ad-497f-aab6-cc0b000ff5f7\") " pod="openshift-ingress/e2e-migrate-6d00b9e3-gw-openshift-default-c4d74979b-ncsh6"
Sep 18 20:17:39.184763 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:39.184746    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7e11376c-93ad-497f-aab6-cc0b000ff5f7-istiod-ca-cert\") pod \"e2e-migrate-6d00b9e3-gw-openshift-default-c4d74979b-ncsh6\" (UID: \"7e11376c-93ad-497f-aab6-cc0b000ff5f7\") " pod="openshift-ingress/e2e-migrate-6d00b9e3-gw-openshift-default-c4d74979b-ncsh6"
Sep 18 20:17:39.186813 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:39.186785    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7e11376c-93ad-497f-aab6-cc0b000ff5f7-istio-envoy\") pod \"e2e-migrate-6d00b9e3-gw-openshift-default-c4d74979b-ncsh6\" (UID: \"7e11376c-93ad-497f-aab6-cc0b000ff5f7\") " pod="openshift-ingress/e2e-migrate-6d00b9e3-gw-openshift-default-c4d74979b-ncsh6"
Sep 18 20:17:39.187014 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:39.186995    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7e11376c-93ad-497f-aab6-cc0b000ff5f7-istio-podinfo\") pod \"e2e-migrate-6d00b9e3-gw-openshift-default-c4d74979b-ncsh6\" (UID: \"7e11376c-93ad-497f-aab6-cc0b000ff5f7\") " pod="openshift-ingress/e2e-migrate-6d00b9e3-gw-openshift-default-c4d74979b-ncsh6"
Sep 18 20:17:39.192299 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:39.192273    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7e11376c-93ad-497f-aab6-cc0b000ff5f7-istio-token\") pod \"e2e-migrate-6d00b9e3-gw-openshift-default-c4d74979b-ncsh6\" (UID: \"7e11376c-93ad-497f-aab6-cc0b000ff5f7\") " pod="openshift-ingress/e2e-migrate-6d00b9e3-gw-openshift-default-c4d74979b-ncsh6"
Sep 18 20:17:39.192374 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:39.192336    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7fbdm\" (UniqueName: \"kubernetes.io/projected/7e11376c-93ad-497f-aab6-cc0b000ff5f7-kube-api-access-7fbdm\") pod \"e2e-migrate-6d00b9e3-gw-openshift-default-c4d74979b-ncsh6\" (UID: \"7e11376c-93ad-497f-aab6-cc0b000ff5f7\") " pod="openshift-ingress/e2e-migrate-6d00b9e3-gw-openshift-default-c4d74979b-ncsh6"
Sep 18 20:17:39.227117 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:39.227083    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:17:39.302155 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:39.302120    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-6d00b9e3-gw-openshift-default-c4d74979b-ncsh6"
Sep 18 20:17:39.579284 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:39.579243    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:17:39.597014 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:39.596989    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-6d00b9e3-gw-openshift-default-c4d74979b-ncsh6"]
Sep 18 20:17:39.598062 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:17:39.598038    2532 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7e11376c_93ad_497f_aab6_cc0b000ff5f7.slice/crio-72133f2d6bafa70cf544f312ac3036f4ae64e4e969f6727d9d5551e1f8b5b305 WatchSource:0}: Error finding container 72133f2d6bafa70cf544f312ac3036f4ae64e4e969f6727d9d5551e1f8b5b305: Status 404 returned error can't find the container with id 72133f2d6bafa70cf544f312ac3036f4ae64e4e969f6727d9d5551e1f8b5b305
Sep 18 20:17:39.600400 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:39.600370    2532 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 18 20:17:39.600466 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:39.600435    2532 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 18 20:17:39.600525 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:39.600463    2532 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 18 20:17:39.849827 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:39.849722    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-6d00b9e3-gw-openshift-default-c4d74979b-ncsh6" event={"ID":"7e11376c-93ad-497f-aab6-cc0b000ff5f7","Type":"ContainerStarted","Data":"36352a42a8d4788405cb430d541cba1534eff6607182ac7fa078903c11ed0969"}
Sep 18 20:17:39.849827 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:39.849791    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-6d00b9e3-gw-openshift-default-c4d74979b-ncsh6" event={"ID":"7e11376c-93ad-497f-aab6-cc0b000ff5f7","Type":"ContainerStarted","Data":"72133f2d6bafa70cf544f312ac3036f4ae64e4e969f6727d9d5551e1f8b5b305"}
Sep 18 20:17:39.870319 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:39.870287    2532 patch_prober.go:28] interesting pod/e2e-inf-b-a85f8b-openshift-default-68dd69554d-dkbnd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused" start-of-body=
Sep 18 20:17:39.870456 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:39.870346    2532 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-a85f8b-openshift-default-68dd69554d-dkbnd" podUID="b1db05d0-08eb-47c0-ad79-65727da00166" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused"
Sep 18 20:17:39.896935 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:39.896886    2532 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-migrate-6d00b9e3-gw-openshift-default-c4d74979b-ncsh6" podStartSLOduration=1.8968703900000001 podStartE2EDuration="1.89687039s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 20:17:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 20:17:39.896627682 +0000 UTC m=+991.662017082" watchObservedRunningTime="2026-09-18 20:17:39.89687039 +0000 UTC m=+991.662259789"
Sep 18 20:17:40.309478 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:40.309440    2532 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-6d00b9e3-gw-openshift-default-c4d74979b-ncsh6"
Sep 18 20:17:40.312259 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:40.312157    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:17:40.313053 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:40.312977    2532 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-6d00b9e3-gw-openshift-default-c4d74979b-ncsh6"
Sep 18 20:17:40.854580 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:40.854551    2532 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-6d00b9e3-gw-openshift-default-c4d74979b-ncsh6"
Sep 18 20:17:40.856079 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:40.856055    2532 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-6d00b9e3-gw-openshift-default-c4d74979b-ncsh6"
Sep 18 20:17:40.870549 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:40.870518    2532 patch_prober.go:28] interesting pod/e2e-inf-b-a85f8b-openshift-default-68dd69554d-dkbnd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused" start-of-body=
Sep 18 20:17:40.870654 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:40.870574    2532 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-a85f8b-openshift-default-68dd69554d-dkbnd" podUID="b1db05d0-08eb-47c0-ad79-65727da00166" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused"
Sep 18 20:17:41.004349 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:41.004314    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:17:41.353600 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:41.353563    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:17:41.453802 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:41.453767    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:17:41.795004 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:41.794973    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-vptdl"]
Sep 18 20:17:41.821907 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:41.821868    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:17:41.874475 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:41.874447    2532 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-a85f8b-openshift-default-68dd69554d-dkbnd"
Sep 18 20:17:41.874767 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:41.874751    2532 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-a85f8b-openshift-default-68dd69554d-dkbnd"
Sep 18 20:17:41.875507 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:41.875473    2532 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-a85f8b-openshift-default-68dd69554d-dkbnd"
Sep 18 20:17:42.519202 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:42.519147    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:17:42.619559 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:42.619525    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:17:43.042155 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:43.042127    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:17:43.136687 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:43.136654    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:17:43.235616 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:43.235582    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:17:43.307672 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:43.307649    2532 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-7626fb93-openshift-default-79bbbc8cdb-28q8h"
Sep 18 20:17:43.426475 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:43.426443    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/25563ed3-db7a-47d0-b2f0-e1efab3a7986-credential-socket\") pod \"25563ed3-db7a-47d0-b2f0-e1efab3a7986\" (UID: \"25563ed3-db7a-47d0-b2f0-e1efab3a7986\") "
Sep 18 20:17:43.426696 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:43.426529    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/25563ed3-db7a-47d0-b2f0-e1efab3a7986-istio-data\") pod \"25563ed3-db7a-47d0-b2f0-e1efab3a7986\" (UID: \"25563ed3-db7a-47d0-b2f0-e1efab3a7986\") "
Sep 18 20:17:43.426696 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:43.426566    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/25563ed3-db7a-47d0-b2f0-e1efab3a7986-workload-certs\") pod \"25563ed3-db7a-47d0-b2f0-e1efab3a7986\" (UID: \"25563ed3-db7a-47d0-b2f0-e1efab3a7986\") "
Sep 18 20:17:43.426696 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:43.426582    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/25563ed3-db7a-47d0-b2f0-e1efab3a7986-istiod-ca-cert\") pod \"25563ed3-db7a-47d0-b2f0-e1efab3a7986\" (UID: \"25563ed3-db7a-47d0-b2f0-e1efab3a7986\") "
Sep 18 20:17:43.426696 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:43.426601    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/25563ed3-db7a-47d0-b2f0-e1efab3a7986-istio-envoy\") pod \"25563ed3-db7a-47d0-b2f0-e1efab3a7986\" (UID: \"25563ed3-db7a-47d0-b2f0-e1efab3a7986\") "
Sep 18 20:17:43.426696 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:43.426620    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/25563ed3-db7a-47d0-b2f0-e1efab3a7986-workload-socket\") pod \"25563ed3-db7a-47d0-b2f0-e1efab3a7986\" (UID: \"25563ed3-db7a-47d0-b2f0-e1efab3a7986\") "
Sep 18 20:17:43.426696 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:43.426653    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/25563ed3-db7a-47d0-b2f0-e1efab3a7986-istio-token\") pod \"25563ed3-db7a-47d0-b2f0-e1efab3a7986\" (UID: \"25563ed3-db7a-47d0-b2f0-e1efab3a7986\") "
Sep 18 20:17:43.426696 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:43.426684    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mrclt\" (UniqueName: \"kubernetes.io/projected/25563ed3-db7a-47d0-b2f0-e1efab3a7986-kube-api-access-mrclt\") pod \"25563ed3-db7a-47d0-b2f0-e1efab3a7986\" (UID: \"25563ed3-db7a-47d0-b2f0-e1efab3a7986\") "
Sep 18 20:17:43.427059 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:43.426749    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/25563ed3-db7a-47d0-b2f0-e1efab3a7986-istio-podinfo\") pod \"25563ed3-db7a-47d0-b2f0-e1efab3a7986\" (UID: \"25563ed3-db7a-47d0-b2f0-e1efab3a7986\") "
Sep 18 20:17:43.427059 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:43.426775    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/25563ed3-db7a-47d0-b2f0-e1efab3a7986-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "25563ed3-db7a-47d0-b2f0-e1efab3a7986" (UID: "25563ed3-db7a-47d0-b2f0-e1efab3a7986"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 20:17:43.427059 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:43.426887    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/25563ed3-db7a-47d0-b2f0-e1efab3a7986-istio-data" (OuterVolumeSpecName: "istio-data") pod "25563ed3-db7a-47d0-b2f0-e1efab3a7986" (UID: "25563ed3-db7a-47d0-b2f0-e1efab3a7986"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 20:17:43.427059 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:43.426896    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/25563ed3-db7a-47d0-b2f0-e1efab3a7986-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "25563ed3-db7a-47d0-b2f0-e1efab3a7986" (UID: "25563ed3-db7a-47d0-b2f0-e1efab3a7986"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 20:17:43.427059 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:43.427016    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/25563ed3-db7a-47d0-b2f0-e1efab3a7986-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "25563ed3-db7a-47d0-b2f0-e1efab3a7986" (UID: "25563ed3-db7a-47d0-b2f0-e1efab3a7986"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 20:17:43.427322 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:43.427078    2532 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/25563ed3-db7a-47d0-b2f0-e1efab3a7986-credential-socket\") on node \"ip-10-0-130-81.ec2.internal\" DevicePath \"\""
Sep 18 20:17:43.427322 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:43.427098    2532 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/25563ed3-db7a-47d0-b2f0-e1efab3a7986-istio-data\") on node \"ip-10-0-130-81.ec2.internal\" DevicePath \"\""
Sep 18 20:17:43.427322 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:43.427112    2532 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/25563ed3-db7a-47d0-b2f0-e1efab3a7986-workload-certs\") on node \"ip-10-0-130-81.ec2.internal\" DevicePath \"\""
Sep 18 20:17:43.427322 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:43.427274    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/25563ed3-db7a-47d0-b2f0-e1efab3a7986-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "25563ed3-db7a-47d0-b2f0-e1efab3a7986" (UID: "25563ed3-db7a-47d0-b2f0-e1efab3a7986"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 20:17:43.428957 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:43.428932    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/25563ed3-db7a-47d0-b2f0-e1efab3a7986-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "25563ed3-db7a-47d0-b2f0-e1efab3a7986" (UID: "25563ed3-db7a-47d0-b2f0-e1efab3a7986"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 20:17:43.428957 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:43.428935    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/25563ed3-db7a-47d0-b2f0-e1efab3a7986-istio-token" (OuterVolumeSpecName: "istio-token") pod "25563ed3-db7a-47d0-b2f0-e1efab3a7986" (UID: "25563ed3-db7a-47d0-b2f0-e1efab3a7986"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 20:17:43.429309 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:43.429291    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/25563ed3-db7a-47d0-b2f0-e1efab3a7986-kube-api-access-mrclt" (OuterVolumeSpecName: "kube-api-access-mrclt") pod "25563ed3-db7a-47d0-b2f0-e1efab3a7986" (UID: "25563ed3-db7a-47d0-b2f0-e1efab3a7986"). InnerVolumeSpecName "kube-api-access-mrclt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 20:17:43.429309 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:43.429296    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/25563ed3-db7a-47d0-b2f0-e1efab3a7986-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "25563ed3-db7a-47d0-b2f0-e1efab3a7986" (UID: "25563ed3-db7a-47d0-b2f0-e1efab3a7986"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 18 20:17:43.527974 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:43.527937    2532 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/25563ed3-db7a-47d0-b2f0-e1efab3a7986-istiod-ca-cert\") on node \"ip-10-0-130-81.ec2.internal\" DevicePath \"\""
Sep 18 20:17:43.527974 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:43.527966    2532 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/25563ed3-db7a-47d0-b2f0-e1efab3a7986-istio-envoy\") on node \"ip-10-0-130-81.ec2.internal\" DevicePath \"\""
Sep 18 20:17:43.527974 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:43.527978    2532 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/25563ed3-db7a-47d0-b2f0-e1efab3a7986-workload-socket\") on node \"ip-10-0-130-81.ec2.internal\" DevicePath \"\""
Sep 18 20:17:43.528421 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:43.527991    2532 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/25563ed3-db7a-47d0-b2f0-e1efab3a7986-istio-token\") on node \"ip-10-0-130-81.ec2.internal\" DevicePath \"\""
Sep 18 20:17:43.528421 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:43.528006    2532 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mrclt\" (UniqueName: \"kubernetes.io/projected/25563ed3-db7a-47d0-b2f0-e1efab3a7986-kube-api-access-mrclt\") on node \"ip-10-0-130-81.ec2.internal\" DevicePath \"\""
Sep 18 20:17:43.528421 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:43.528017    2532 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/25563ed3-db7a-47d0-b2f0-e1efab3a7986-istio-podinfo\") on node \"ip-10-0-130-81.ec2.internal\" DevicePath \"\""
Sep 18 20:17:43.625128 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:43.625042    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:17:43.871997 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:43.871961    2532 generic.go:356] "Generic (PLEG): container finished" podID="25563ed3-db7a-47d0-b2f0-e1efab3a7986" containerID="b9ea7929ffbb461f6398db41fd0da0a96175ec85e1a437a988b4cd39e5941283" exitCode=0
Sep 18 20:17:43.872157 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:43.872005    2532 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-7626fb93-openshift-default-79bbbc8cdb-28q8h"
Sep 18 20:17:43.872157 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:43.872047    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-7626fb93-openshift-default-79bbbc8cdb-28q8h" event={"ID":"25563ed3-db7a-47d0-b2f0-e1efab3a7986","Type":"ContainerDied","Data":"b9ea7929ffbb461f6398db41fd0da0a96175ec85e1a437a988b4cd39e5941283"}
Sep 18 20:17:43.872157 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:43.872096    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-7626fb93-openshift-default-79bbbc8cdb-28q8h" event={"ID":"25563ed3-db7a-47d0-b2f0-e1efab3a7986","Type":"ContainerDied","Data":"7c416e91f5be08e335e5157ecd97d55daa5e046822a041c05e43b909ae8b50fc"}
Sep 18 20:17:43.872157 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:43.872123    2532 scope.go:117] "RemoveContainer" containerID="b9ea7929ffbb461f6398db41fd0da0a96175ec85e1a437a988b4cd39e5941283"
Sep 18 20:17:43.881600 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:43.881584    2532 scope.go:117] "RemoveContainer" containerID="9ab286c56a0710003f40894b6e156edb25a4e796d9a628a14b9bb7e9dff067c9"
Sep 18 20:17:43.889554 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:43.889531    2532 scope.go:117] "RemoveContainer" containerID="b9ea7929ffbb461f6398db41fd0da0a96175ec85e1a437a988b4cd39e5941283"
Sep 18 20:17:43.889809 ip-10-0-130-81 kubenswrapper[2532]: E0918 20:17:43.889791    2532 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b9ea7929ffbb461f6398db41fd0da0a96175ec85e1a437a988b4cd39e5941283\": container with ID starting with b9ea7929ffbb461f6398db41fd0da0a96175ec85e1a437a988b4cd39e5941283 not found: ID does not exist" containerID="b9ea7929ffbb461f6398db41fd0da0a96175ec85e1a437a988b4cd39e5941283"
Sep 18 20:17:43.889856 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:43.889818    2532 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b9ea7929ffbb461f6398db41fd0da0a96175ec85e1a437a988b4cd39e5941283"} err="failed to get container status \"b9ea7929ffbb461f6398db41fd0da0a96175ec85e1a437a988b4cd39e5941283\": rpc error: code = NotFound desc = could not find container \"b9ea7929ffbb461f6398db41fd0da0a96175ec85e1a437a988b4cd39e5941283\": container with ID starting with b9ea7929ffbb461f6398db41fd0da0a96175ec85e1a437a988b4cd39e5941283 not found: ID does not exist"
Sep 18 20:17:43.889856 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:43.889834    2532 scope.go:117] "RemoveContainer" containerID="9ab286c56a0710003f40894b6e156edb25a4e796d9a628a14b9bb7e9dff067c9"
Sep 18 20:17:43.890082 ip-10-0-130-81 kubenswrapper[2532]: E0918 20:17:43.890064    2532 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9ab286c56a0710003f40894b6e156edb25a4e796d9a628a14b9bb7e9dff067c9\": container with ID starting with 9ab286c56a0710003f40894b6e156edb25a4e796d9a628a14b9bb7e9dff067c9 not found: ID does not exist" containerID="9ab286c56a0710003f40894b6e156edb25a4e796d9a628a14b9bb7e9dff067c9"
Sep 18 20:17:43.890122 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:43.890088    2532 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9ab286c56a0710003f40894b6e156edb25a4e796d9a628a14b9bb7e9dff067c9"} err="failed to get container status \"9ab286c56a0710003f40894b6e156edb25a4e796d9a628a14b9bb7e9dff067c9\": rpc error: code = NotFound desc = could not find container \"9ab286c56a0710003f40894b6e156edb25a4e796d9a628a14b9bb7e9dff067c9\": container with ID starting with 9ab286c56a0710003f40894b6e156edb25a4e796d9a628a14b9bb7e9dff067c9 not found: ID does not exist"
Sep 18 20:17:43.894250 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:43.894229    2532 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-7626fb93-openshift-default-79bbbc8cdb-28q8h"]
Sep 18 20:17:43.898398 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:43.898374    2532 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-7626fb93-openshift-default-79bbbc8cdb-28q8h"]
Sep 18 20:17:44.306088 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:44.306057    2532 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="25563ed3-db7a-47d0-b2f0-e1efab3a7986" path="/var/lib/kubelet/pods/25563ed3-db7a-47d0-b2f0-e1efab3a7986/volumes"
Sep 18 20:17:44.360162 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:44.360131    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:17:45.202009 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:45.201972    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:17:45.819672 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:45.819638    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:17:51.187765 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:51.187727    2532 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w3-b88002/simulator-w3-b88002-kserve-7f78b4fbdd-w7dzh"]
Sep 18 20:17:51.188203 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:51.188067    2532 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="25563ed3-db7a-47d0-b2f0-e1efab3a7986" containerName="istio-proxy"
Sep 18 20:17:51.188203 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:51.188077    2532 state_mem.go:107] "Deleted CPUSet assignment" podUID="25563ed3-db7a-47d0-b2f0-e1efab3a7986" containerName="istio-proxy"
Sep 18 20:17:51.188203 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:51.188085    2532 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="25563ed3-db7a-47d0-b2f0-e1efab3a7986" containerName="istio-proxy"
Sep 18 20:17:51.188203 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:51.188091    2532 state_mem.go:107] "Deleted CPUSet assignment" podUID="25563ed3-db7a-47d0-b2f0-e1efab3a7986" containerName="istio-proxy"
Sep 18 20:17:51.188203 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:51.188159    2532 memory_manager.go:368] "RemoveStaleState removing state" podUID="25563ed3-db7a-47d0-b2f0-e1efab3a7986" containerName="istio-proxy"
Sep 18 20:17:51.192989 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:51.192962    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-b88002/simulator-w3-b88002-kserve-7f78b4fbdd-w7dzh"
Sep 18 20:17:51.197975 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:51.197940    2532 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w3-b88002\"/\"openshift-service-ca.crt\""
Sep 18 20:17:51.199264 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:51.198802    2532 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w3-b88002\"/\"default-dockercfg-hdqwp\""
Sep 18 20:17:51.199383 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:51.199347    2532 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w3-b88002\"/\"simulator-w3-b88002-kserve-self-signed-certs\""
Sep 18 20:17:51.200240 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:51.200151    2532 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w3-b88002\"/\"kube-root-ca.crt\""
Sep 18 20:17:51.211466 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:51.211429    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w3-b88002/simulator-w3-b88002-kserve-7f78b4fbdd-w7dzh"]
Sep 18 20:17:51.285836 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:51.285776    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:17:51.312510 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:51.307968    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/268e7f4d-7c4e-4f88-8fcf-8d076415de22-dshm\") pod \"simulator-w3-b88002-kserve-7f78b4fbdd-w7dzh\" (UID: \"268e7f4d-7c4e-4f88-8fcf-8d076415de22\") " pod="e2e-models-e2e-worker-w3-b88002/simulator-w3-b88002-kserve-7f78b4fbdd-w7dzh"
Sep 18 20:17:51.312510 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:51.308027    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/268e7f4d-7c4e-4f88-8fcf-8d076415de22-tmp-dir\") pod \"simulator-w3-b88002-kserve-7f78b4fbdd-w7dzh\" (UID: \"268e7f4d-7c4e-4f88-8fcf-8d076415de22\") " pod="e2e-models-e2e-worker-w3-b88002/simulator-w3-b88002-kserve-7f78b4fbdd-w7dzh"
Sep 18 20:17:51.312510 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:51.308095    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/268e7f4d-7c4e-4f88-8fcf-8d076415de22-model-cache\") pod \"simulator-w3-b88002-kserve-7f78b4fbdd-w7dzh\" (UID: \"268e7f4d-7c4e-4f88-8fcf-8d076415de22\") " pod="e2e-models-e2e-worker-w3-b88002/simulator-w3-b88002-kserve-7f78b4fbdd-w7dzh"
Sep 18 20:17:51.312510 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:51.308122    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/268e7f4d-7c4e-4f88-8fcf-8d076415de22-tls-certs\") pod \"simulator-w3-b88002-kserve-7f78b4fbdd-w7dzh\" (UID: \"268e7f4d-7c4e-4f88-8fcf-8d076415de22\") " pod="e2e-models-e2e-worker-w3-b88002/simulator-w3-b88002-kserve-7f78b4fbdd-w7dzh"
Sep 18 20:17:51.312510 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:51.308167    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/268e7f4d-7c4e-4f88-8fcf-8d076415de22-home\") pod \"simulator-w3-b88002-kserve-7f78b4fbdd-w7dzh\" (UID: \"268e7f4d-7c4e-4f88-8fcf-8d076415de22\") " pod="e2e-models-e2e-worker-w3-b88002/simulator-w3-b88002-kserve-7f78b4fbdd-w7dzh"
Sep 18 20:17:51.312510 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:51.308202    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pjjm7\" (UniqueName: \"kubernetes.io/projected/268e7f4d-7c4e-4f88-8fcf-8d076415de22-kube-api-access-pjjm7\") pod \"simulator-w3-b88002-kserve-7f78b4fbdd-w7dzh\" (UID: \"268e7f4d-7c4e-4f88-8fcf-8d076415de22\") " pod="e2e-models-e2e-worker-w3-b88002/simulator-w3-b88002-kserve-7f78b4fbdd-w7dzh"
Sep 18 20:17:51.409550 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:51.409504    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/268e7f4d-7c4e-4f88-8fcf-8d076415de22-home\") pod \"simulator-w3-b88002-kserve-7f78b4fbdd-w7dzh\" (UID: \"268e7f4d-7c4e-4f88-8fcf-8d076415de22\") " pod="e2e-models-e2e-worker-w3-b88002/simulator-w3-b88002-kserve-7f78b4fbdd-w7dzh"
Sep 18 20:17:51.409755 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:51.409571    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pjjm7\" (UniqueName: \"kubernetes.io/projected/268e7f4d-7c4e-4f88-8fcf-8d076415de22-kube-api-access-pjjm7\") pod \"simulator-w3-b88002-kserve-7f78b4fbdd-w7dzh\" (UID: \"268e7f4d-7c4e-4f88-8fcf-8d076415de22\") " pod="e2e-models-e2e-worker-w3-b88002/simulator-w3-b88002-kserve-7f78b4fbdd-w7dzh"
Sep 18 20:17:51.409755 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:51.409650    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/268e7f4d-7c4e-4f88-8fcf-8d076415de22-dshm\") pod \"simulator-w3-b88002-kserve-7f78b4fbdd-w7dzh\" (UID: \"268e7f4d-7c4e-4f88-8fcf-8d076415de22\") " pod="e2e-models-e2e-worker-w3-b88002/simulator-w3-b88002-kserve-7f78b4fbdd-w7dzh"
Sep 18 20:17:51.409755 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:51.409678    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/268e7f4d-7c4e-4f88-8fcf-8d076415de22-tmp-dir\") pod \"simulator-w3-b88002-kserve-7f78b4fbdd-w7dzh\" (UID: \"268e7f4d-7c4e-4f88-8fcf-8d076415de22\") " pod="e2e-models-e2e-worker-w3-b88002/simulator-w3-b88002-kserve-7f78b4fbdd-w7dzh"
Sep 18 20:17:51.409755 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:51.409745    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/268e7f4d-7c4e-4f88-8fcf-8d076415de22-model-cache\") pod \"simulator-w3-b88002-kserve-7f78b4fbdd-w7dzh\" (UID: \"268e7f4d-7c4e-4f88-8fcf-8d076415de22\") " pod="e2e-models-e2e-worker-w3-b88002/simulator-w3-b88002-kserve-7f78b4fbdd-w7dzh"
Sep 18 20:17:51.409984 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:51.409772    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/268e7f4d-7c4e-4f88-8fcf-8d076415de22-tls-certs\") pod \"simulator-w3-b88002-kserve-7f78b4fbdd-w7dzh\" (UID: \"268e7f4d-7c4e-4f88-8fcf-8d076415de22\") " pod="e2e-models-e2e-worker-w3-b88002/simulator-w3-b88002-kserve-7f78b4fbdd-w7dzh"
Sep 18 20:17:51.410949 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:51.410919    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/268e7f4d-7c4e-4f88-8fcf-8d076415de22-home\") pod \"simulator-w3-b88002-kserve-7f78b4fbdd-w7dzh\" (UID: \"268e7f4d-7c4e-4f88-8fcf-8d076415de22\") " pod="e2e-models-e2e-worker-w3-b88002/simulator-w3-b88002-kserve-7f78b4fbdd-w7dzh"
Sep 18 20:17:51.411112 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:51.411092    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/268e7f4d-7c4e-4f88-8fcf-8d076415de22-tmp-dir\") pod \"simulator-w3-b88002-kserve-7f78b4fbdd-w7dzh\" (UID: \"268e7f4d-7c4e-4f88-8fcf-8d076415de22\") " pod="e2e-models-e2e-worker-w3-b88002/simulator-w3-b88002-kserve-7f78b4fbdd-w7dzh"
Sep 18 20:17:51.411372 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:51.411351    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/268e7f4d-7c4e-4f88-8fcf-8d076415de22-model-cache\") pod \"simulator-w3-b88002-kserve-7f78b4fbdd-w7dzh\" (UID: \"268e7f4d-7c4e-4f88-8fcf-8d076415de22\") " pod="e2e-models-e2e-worker-w3-b88002/simulator-w3-b88002-kserve-7f78b4fbdd-w7dzh"
Sep 18 20:17:51.413902 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:51.413855    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/268e7f4d-7c4e-4f88-8fcf-8d076415de22-dshm\") pod \"simulator-w3-b88002-kserve-7f78b4fbdd-w7dzh\" (UID: \"268e7f4d-7c4e-4f88-8fcf-8d076415de22\") " pod="e2e-models-e2e-worker-w3-b88002/simulator-w3-b88002-kserve-7f78b4fbdd-w7dzh"
Sep 18 20:17:51.415993 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:51.415948    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/268e7f4d-7c4e-4f88-8fcf-8d076415de22-tls-certs\") pod \"simulator-w3-b88002-kserve-7f78b4fbdd-w7dzh\" (UID: \"268e7f4d-7c4e-4f88-8fcf-8d076415de22\") " pod="e2e-models-e2e-worker-w3-b88002/simulator-w3-b88002-kserve-7f78b4fbdd-w7dzh"
Sep 18 20:17:51.422254 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:51.422178    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pjjm7\" (UniqueName: \"kubernetes.io/projected/268e7f4d-7c4e-4f88-8fcf-8d076415de22-kube-api-access-pjjm7\") pod \"simulator-w3-b88002-kserve-7f78b4fbdd-w7dzh\" (UID: \"268e7f4d-7c4e-4f88-8fcf-8d076415de22\") " pod="e2e-models-e2e-worker-w3-b88002/simulator-w3-b88002-kserve-7f78b4fbdd-w7dzh"
Sep 18 20:17:51.512194 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:51.512156    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-b88002/simulator-w3-b88002-kserve-7f78b4fbdd-w7dzh"
Sep 18 20:17:51.788025 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:51.788002    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w3-b88002/simulator-w3-b88002-kserve-7f78b4fbdd-w7dzh"]
Sep 18 20:17:51.789474 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:17:51.789444    2532 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod268e7f4d_7c4e_4f88_8fcf_8d076415de22.slice/crio-752eb3b1ff8f71c515428d181f7d0fdc6660c8ffe162568b2e315bfcdde7c553 WatchSource:0}: Error finding container 752eb3b1ff8f71c515428d181f7d0fdc6660c8ffe162568b2e315bfcdde7c553: Status 404 returned error can't find the container with id 752eb3b1ff8f71c515428d181f7d0fdc6660c8ffe162568b2e315bfcdde7c553
Sep 18 20:17:51.931050 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:51.931017    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-b88002/simulator-w3-b88002-kserve-7f78b4fbdd-w7dzh" event={"ID":"268e7f4d-7c4e-4f88-8fcf-8d076415de22","Type":"ContainerStarted","Data":"0e8721fcbb421d919d6aecebb2b8d49404cc85b0e285866fa7df82c967fe9633"}
Sep 18 20:17:51.931217 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:51.931060    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-b88002/simulator-w3-b88002-kserve-7f78b4fbdd-w7dzh" event={"ID":"268e7f4d-7c4e-4f88-8fcf-8d076415de22","Type":"ContainerStarted","Data":"752eb3b1ff8f71c515428d181f7d0fdc6660c8ffe162568b2e315bfcdde7c553"}
Sep 18 20:17:51.951278 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:51.951229    2532 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-models-e2e-worker-w3-b88002/simulator-w3-b88002-kserve-7f78b4fbdd-w7dzh" podStartSLOduration=0.951213427 podStartE2EDuration="951.213427ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 20:17:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 20:17:51.949481514 +0000 UTC m=+1003.714870913" watchObservedRunningTime="2026-09-18 20:17:51.951213427 +0000 UTC m=+1003.716602826"
Sep 18 20:17:57.198144 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:57.198108    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-vptdl"]
Sep 18 20:17:58.982955 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:58.982924    2532 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w0-116778/simulator-w0-116778-kserve-8f94f4bdc-4wbtx"]
Sep 18 20:17:58.983356 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:58.983343    2532 memory_manager.go:368] "RemoveStaleState removing state" podUID="25563ed3-db7a-47d0-b2f0-e1efab3a7986" containerName="istio-proxy"
Sep 18 20:17:58.985417 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:58.985401    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-116778/simulator-w0-116778-kserve-8f94f4bdc-4wbtx"
Sep 18 20:17:58.989341 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:58.989308    2532 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-116778\"/\"simulator-w0-116778-kserve-self-signed-certs\""
Sep 18 20:17:58.989341 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:58.989324    2532 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w0-116778\"/\"openshift-service-ca.crt\""
Sep 18 20:17:58.989663 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:58.989648    2532 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w0-116778\"/\"kube-root-ca.crt\""
Sep 18 20:17:58.989726 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:58.989682    2532 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-116778\"/\"default-dockercfg-phwfg\""
Sep 18 20:17:58.996591 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:58.996568    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-116778/simulator-w0-116778-kserve-8f94f4bdc-4wbtx"]
Sep 18 20:17:59.010600 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:59.010565    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:17:59.097456 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:59.097410    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d839bc6f-e7f5-496d-9e64-eacee63a95fd-dshm\") pod \"simulator-w0-116778-kserve-8f94f4bdc-4wbtx\" (UID: \"d839bc6f-e7f5-496d-9e64-eacee63a95fd\") " pod="e2e-models-e2e-worker-w0-116778/simulator-w0-116778-kserve-8f94f4bdc-4wbtx"
Sep 18 20:17:59.097456 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:59.097457    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d839bc6f-e7f5-496d-9e64-eacee63a95fd-model-cache\") pod \"simulator-w0-116778-kserve-8f94f4bdc-4wbtx\" (UID: \"d839bc6f-e7f5-496d-9e64-eacee63a95fd\") " pod="e2e-models-e2e-worker-w0-116778/simulator-w0-116778-kserve-8f94f4bdc-4wbtx"
Sep 18 20:17:59.097702 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:59.097616    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lxjxv\" (UniqueName: \"kubernetes.io/projected/d839bc6f-e7f5-496d-9e64-eacee63a95fd-kube-api-access-lxjxv\") pod \"simulator-w0-116778-kserve-8f94f4bdc-4wbtx\" (UID: \"d839bc6f-e7f5-496d-9e64-eacee63a95fd\") " pod="e2e-models-e2e-worker-w0-116778/simulator-w0-116778-kserve-8f94f4bdc-4wbtx"
Sep 18 20:17:59.097702 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:59.097676    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d839bc6f-e7f5-496d-9e64-eacee63a95fd-home\") pod \"simulator-w0-116778-kserve-8f94f4bdc-4wbtx\" (UID: \"d839bc6f-e7f5-496d-9e64-eacee63a95fd\") " pod="e2e-models-e2e-worker-w0-116778/simulator-w0-116778-kserve-8f94f4bdc-4wbtx"
Sep 18 20:17:59.097779 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:59.097702    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d839bc6f-e7f5-496d-9e64-eacee63a95fd-tls-certs\") pod \"simulator-w0-116778-kserve-8f94f4bdc-4wbtx\" (UID: \"d839bc6f-e7f5-496d-9e64-eacee63a95fd\") " pod="e2e-models-e2e-worker-w0-116778/simulator-w0-116778-kserve-8f94f4bdc-4wbtx"
Sep 18 20:17:59.097779 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:59.097721    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d839bc6f-e7f5-496d-9e64-eacee63a95fd-tmp-dir\") pod \"simulator-w0-116778-kserve-8f94f4bdc-4wbtx\" (UID: \"d839bc6f-e7f5-496d-9e64-eacee63a95fd\") " pod="e2e-models-e2e-worker-w0-116778/simulator-w0-116778-kserve-8f94f4bdc-4wbtx"
Sep 18 20:17:59.198645 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:59.198615    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d839bc6f-e7f5-496d-9e64-eacee63a95fd-dshm\") pod \"simulator-w0-116778-kserve-8f94f4bdc-4wbtx\" (UID: \"d839bc6f-e7f5-496d-9e64-eacee63a95fd\") " pod="e2e-models-e2e-worker-w0-116778/simulator-w0-116778-kserve-8f94f4bdc-4wbtx"
Sep 18 20:17:59.198645 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:59.198646    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d839bc6f-e7f5-496d-9e64-eacee63a95fd-model-cache\") pod \"simulator-w0-116778-kserve-8f94f4bdc-4wbtx\" (UID: \"d839bc6f-e7f5-496d-9e64-eacee63a95fd\") " pod="e2e-models-e2e-worker-w0-116778/simulator-w0-116778-kserve-8f94f4bdc-4wbtx"
Sep 18 20:17:59.198891 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:59.198708    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lxjxv\" (UniqueName: \"kubernetes.io/projected/d839bc6f-e7f5-496d-9e64-eacee63a95fd-kube-api-access-lxjxv\") pod \"simulator-w0-116778-kserve-8f94f4bdc-4wbtx\" (UID: \"d839bc6f-e7f5-496d-9e64-eacee63a95fd\") " pod="e2e-models-e2e-worker-w0-116778/simulator-w0-116778-kserve-8f94f4bdc-4wbtx"
Sep 18 20:17:59.198891 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:59.198743    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d839bc6f-e7f5-496d-9e64-eacee63a95fd-home\") pod \"simulator-w0-116778-kserve-8f94f4bdc-4wbtx\" (UID: \"d839bc6f-e7f5-496d-9e64-eacee63a95fd\") " pod="e2e-models-e2e-worker-w0-116778/simulator-w0-116778-kserve-8f94f4bdc-4wbtx"
Sep 18 20:17:59.198891 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:59.198759    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d839bc6f-e7f5-496d-9e64-eacee63a95fd-tls-certs\") pod \"simulator-w0-116778-kserve-8f94f4bdc-4wbtx\" (UID: \"d839bc6f-e7f5-496d-9e64-eacee63a95fd\") " pod="e2e-models-e2e-worker-w0-116778/simulator-w0-116778-kserve-8f94f4bdc-4wbtx"
Sep 18 20:17:59.198891 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:59.198780    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d839bc6f-e7f5-496d-9e64-eacee63a95fd-tmp-dir\") pod \"simulator-w0-116778-kserve-8f94f4bdc-4wbtx\" (UID: \"d839bc6f-e7f5-496d-9e64-eacee63a95fd\") " pod="e2e-models-e2e-worker-w0-116778/simulator-w0-116778-kserve-8f94f4bdc-4wbtx"
Sep 18 20:17:59.199188 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:59.199152    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d839bc6f-e7f5-496d-9e64-eacee63a95fd-model-cache\") pod \"simulator-w0-116778-kserve-8f94f4bdc-4wbtx\" (UID: \"d839bc6f-e7f5-496d-9e64-eacee63a95fd\") " pod="e2e-models-e2e-worker-w0-116778/simulator-w0-116778-kserve-8f94f4bdc-4wbtx"
Sep 18 20:17:59.199188 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:59.199176    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d839bc6f-e7f5-496d-9e64-eacee63a95fd-tmp-dir\") pod \"simulator-w0-116778-kserve-8f94f4bdc-4wbtx\" (UID: \"d839bc6f-e7f5-496d-9e64-eacee63a95fd\") " pod="e2e-models-e2e-worker-w0-116778/simulator-w0-116778-kserve-8f94f4bdc-4wbtx"
Sep 18 20:17:59.199379 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:59.199196    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d839bc6f-e7f5-496d-9e64-eacee63a95fd-home\") pod \"simulator-w0-116778-kserve-8f94f4bdc-4wbtx\" (UID: \"d839bc6f-e7f5-496d-9e64-eacee63a95fd\") " pod="e2e-models-e2e-worker-w0-116778/simulator-w0-116778-kserve-8f94f4bdc-4wbtx"
Sep 18 20:17:59.200947 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:59.200924    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d839bc6f-e7f5-496d-9e64-eacee63a95fd-dshm\") pod \"simulator-w0-116778-kserve-8f94f4bdc-4wbtx\" (UID: \"d839bc6f-e7f5-496d-9e64-eacee63a95fd\") " pod="e2e-models-e2e-worker-w0-116778/simulator-w0-116778-kserve-8f94f4bdc-4wbtx"
Sep 18 20:17:59.201359 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:59.201339    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d839bc6f-e7f5-496d-9e64-eacee63a95fd-tls-certs\") pod \"simulator-w0-116778-kserve-8f94f4bdc-4wbtx\" (UID: \"d839bc6f-e7f5-496d-9e64-eacee63a95fd\") " pod="e2e-models-e2e-worker-w0-116778/simulator-w0-116778-kserve-8f94f4bdc-4wbtx"
Sep 18 20:17:59.207847 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:59.207824    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lxjxv\" (UniqueName: \"kubernetes.io/projected/d839bc6f-e7f5-496d-9e64-eacee63a95fd-kube-api-access-lxjxv\") pod \"simulator-w0-116778-kserve-8f94f4bdc-4wbtx\" (UID: \"d839bc6f-e7f5-496d-9e64-eacee63a95fd\") " pod="e2e-models-e2e-worker-w0-116778/simulator-w0-116778-kserve-8f94f4bdc-4wbtx"
Sep 18 20:17:59.295935 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:59.295839    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-116778/simulator-w0-116778-kserve-8f94f4bdc-4wbtx"
Sep 18 20:17:59.582744 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:59.582716    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-116778/simulator-w0-116778-kserve-8f94f4bdc-4wbtx"]
Sep 18 20:17:59.584844 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:17:59.584820    2532 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd839bc6f_e7f5_496d_9e64_eacee63a95fd.slice/crio-8a4b4e8eac984c72c2ba948b489707433aef556e31d67e74df2d29200aefc04f WatchSource:0}: Error finding container 8a4b4e8eac984c72c2ba948b489707433aef556e31d67e74df2d29200aefc04f: Status 404 returned error can't find the container with id 8a4b4e8eac984c72c2ba948b489707433aef556e31d67e74df2d29200aefc04f
Sep 18 20:17:59.610279 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:59.610251    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:17:59.669056 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:59.669030    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:17:59.980863 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:59.980835    2532 generic.go:356] "Generic (PLEG): container finished" podID="7e11376c-93ad-497f-aab6-cc0b000ff5f7" containerID="36352a42a8d4788405cb430d541cba1534eff6607182ac7fa078903c11ed0969" exitCode=0
Sep 18 20:17:59.981069 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:59.980914    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-6d00b9e3-gw-openshift-default-c4d74979b-ncsh6" event={"ID":"7e11376c-93ad-497f-aab6-cc0b000ff5f7","Type":"ContainerDied","Data":"36352a42a8d4788405cb430d541cba1534eff6607182ac7fa078903c11ed0969"}
Sep 18 20:17:59.981390 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:59.981372    2532 scope.go:117] "RemoveContainer" containerID="36352a42a8d4788405cb430d541cba1534eff6607182ac7fa078903c11ed0969"
Sep 18 20:17:59.982581 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:59.982552    2532 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 18 20:17:59.982694 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:59.982634    2532 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 18 20:17:59.982694 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:59.982656    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-116778/simulator-w0-116778-kserve-8f94f4bdc-4wbtx" event={"ID":"d839bc6f-e7f5-496d-9e64-eacee63a95fd","Type":"ContainerStarted","Data":"0eb85f11fccbb6c2ed97ecd27fdf6120fdd7c602d3eec60dae71e112ff48d0c0"}
Sep 18 20:17:59.982694 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:59.982679    2532 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 18 20:17:59.982861 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:17:59.982684    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-116778/simulator-w0-116778-kserve-8f94f4bdc-4wbtx" event={"ID":"d839bc6f-e7f5-496d-9e64-eacee63a95fd","Type":"ContainerStarted","Data":"8a4b4e8eac984c72c2ba948b489707433aef556e31d67e74df2d29200aefc04f"}
Sep 18 20:18:00.021167 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:18:00.021107    2532 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-models-e2e-worker-w0-116778/simulator-w0-116778-kserve-8f94f4bdc-4wbtx" podStartSLOduration=2.02108379 podStartE2EDuration="2.02108379s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 20:17:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 20:18:00.017132351 +0000 UTC m=+1011.782521750" watchObservedRunningTime="2026-09-18 20:18:00.02108379 +0000 UTC m=+1011.786473191"
Sep 18 20:18:00.989831 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:18:00.989795    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-6d00b9e3-gw-openshift-default-c4d74979b-ncsh6" event={"ID":"7e11376c-93ad-497f-aab6-cc0b000ff5f7","Type":"ContainerStarted","Data":"20d2571486e94dcdcc985578718a1f73684619ff000b80013cc6def41699b6ee"}
Sep 18 20:18:01.302388 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:18:01.302290    2532 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-6d00b9e3-gw-openshift-default-c4d74979b-ncsh6"
Sep 18 20:18:01.302872 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:18:01.302585    2532 patch_prober.go:28] interesting pod/e2e-migrate-6d00b9e3-gw-openshift-default-c4d74979b-ncsh6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused" start-of-body=
Sep 18 20:18:01.302872 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:18:01.302633    2532 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-6d00b9e3-gw-openshift-default-c4d74979b-ncsh6" podUID="7e11376c-93ad-497f-aab6-cc0b000ff5f7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused"
Sep 18 20:18:01.513765 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:18:01.513718    2532 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w3-b88002/simulator-w3-b88002-kserve-7f78b4fbdd-w7dzh"
Sep 18 20:18:01.514503 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:18:01.514460    2532 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-b88002/simulator-w3-b88002-kserve-7f78b4fbdd-w7dzh"
Sep 18 20:18:01.539013 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:18:01.538979    2532 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w3-b88002/simulator-w3-b88002-kserve-7f78b4fbdd-w7dzh"
Sep 18 20:18:02.032346 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:18:02.032319    2532 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-b88002/simulator-w3-b88002-kserve-7f78b4fbdd-w7dzh"
Sep 18 20:18:02.305463 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:18:02.304266    2532 patch_prober.go:28] interesting pod/e2e-migrate-6d00b9e3-gw-openshift-default-c4d74979b-ncsh6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused" start-of-body=
Sep 18 20:18:02.305463 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:18:02.304350    2532 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-6d00b9e3-gw-openshift-default-c4d74979b-ncsh6" podUID="7e11376c-93ad-497f-aab6-cc0b000ff5f7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused"
Sep 18 20:18:03.303615 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:18:03.303572    2532 patch_prober.go:28] interesting pod/e2e-migrate-6d00b9e3-gw-openshift-default-c4d74979b-ncsh6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused" start-of-body=
Sep 18 20:18:03.303923 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:18:03.303893    2532 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-6d00b9e3-gw-openshift-default-c4d74979b-ncsh6" podUID="7e11376c-93ad-497f-aab6-cc0b000ff5f7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused"
Sep 18 20:18:04.306152 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:18:04.306102    2532 patch_prober.go:28] interesting pod/e2e-migrate-6d00b9e3-gw-openshift-default-c4d74979b-ncsh6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused" start-of-body=
Sep 18 20:18:04.306775 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:18:04.306743    2532 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-6d00b9e3-gw-openshift-default-c4d74979b-ncsh6" podUID="7e11376c-93ad-497f-aab6-cc0b000ff5f7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused"
Sep 18 20:18:04.417564 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:18:04.417529    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:18:05.191740 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:18:05.191702    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:18:05.303715 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:18:05.303671    2532 patch_prober.go:28] interesting pod/e2e-migrate-6d00b9e3-gw-openshift-default-c4d74979b-ncsh6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused" start-of-body=
Sep 18 20:18:05.303932 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:18:05.303742    2532 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-6d00b9e3-gw-openshift-default-c4d74979b-ncsh6" podUID="7e11376c-93ad-497f-aab6-cc0b000ff5f7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused"
Sep 18 20:18:05.718330 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:18:05.718287    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:18:06.307620 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:18:06.305943    2532 patch_prober.go:28] interesting pod/e2e-migrate-6d00b9e3-gw-openshift-default-c4d74979b-ncsh6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused" start-of-body=
Sep 18 20:18:06.307620 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:18:06.306011    2532 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-6d00b9e3-gw-openshift-default-c4d74979b-ncsh6" podUID="7e11376c-93ad-497f-aab6-cc0b000ff5f7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused"
Sep 18 20:18:06.484309 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:18:06.484265    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:18:07.279062 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:18:07.279027    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:18:07.306805 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:18:07.306778    2532 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-6d00b9e3-gw-openshift-default-c4d74979b-ncsh6"
Sep 18 20:18:07.307137 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:18:07.307118    2532 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-6d00b9e3-gw-openshift-default-c4d74979b-ncsh6"
Sep 18 20:18:07.307764 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:18:07.307744    2532 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-6d00b9e3-gw-openshift-default-c4d74979b-ncsh6"
Sep 18 20:18:07.934928 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:18:07.934893    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:18:08.621030 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:18:08.620964    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:18:09.296332 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:18:09.296293    2532 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w0-116778/simulator-w0-116778-kserve-8f94f4bdc-4wbtx"
Sep 18 20:18:09.296549 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:18:09.296347    2532 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-116778/simulator-w0-116778-kserve-8f94f4bdc-4wbtx"
Sep 18 20:18:09.309183 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:18:09.309159    2532 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w0-116778/simulator-w0-116778-kserve-8f94f4bdc-4wbtx"
Sep 18 20:18:09.417099 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:18:09.417058    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:18:10.049153 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:18:10.049109    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:18:10.060727 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:18:10.060708    2532 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-116778/simulator-w0-116778-kserve-8f94f4bdc-4wbtx"
Sep 18 20:18:10.754573 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:18:10.754540    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:18:11.598667 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:18:11.598630    2532 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-a-e2dea3/test-model-e2dea3-kserve-84f88488c7-n2v8d"]
Sep 18 20:18:11.602267 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:18:11.602247    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-e2dea3/test-model-e2dea3-kserve-84f88488c7-n2v8d"
Sep 18 20:18:11.605005 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:18:11.604976    2532 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-e2dea3\"/\"openshift-service-ca.crt\""
Sep 18 20:18:11.606298 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:18:11.606259    2532 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-e2dea3\"/\"test-model-e2dea3-kserve-self-signed-certs\""
Sep 18 20:18:11.606523 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:18:11.606276    2532 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-e2dea3\"/\"kube-root-ca.crt\""
Sep 18 20:18:11.606687 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:18:11.606342    2532 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-e2dea3\"/\"default-dockercfg-d5msd\""
Sep 18 20:18:11.609424 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:18:11.609401    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-e2dea3/test-model-e2dea3-kserve-84f88488c7-n2v8d"]
Sep 18 20:18:11.622294 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:18:11.622264    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:18:11.722905 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:18:11.722873    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a8ac32f5-6438-4dfa-91ea-12e67ff6b4bf-model-cache\") pod \"test-model-e2dea3-kserve-84f88488c7-n2v8d\" (UID: \"a8ac32f5-6438-4dfa-91ea-12e67ff6b4bf\") " pod="ai-tenant-e2e-inf-a-e2dea3/test-model-e2dea3-kserve-84f88488c7-n2v8d"
Sep 18 20:18:11.723081 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:18:11.722913    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a8ac32f5-6438-4dfa-91ea-12e67ff6b4bf-tmp-dir\") pod \"test-model-e2dea3-kserve-84f88488c7-n2v8d\" (UID: \"a8ac32f5-6438-4dfa-91ea-12e67ff6b4bf\") " pod="ai-tenant-e2e-inf-a-e2dea3/test-model-e2dea3-kserve-84f88488c7-n2v8d"
Sep 18 20:18:11.723081 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:18:11.722939    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a8ac32f5-6438-4dfa-91ea-12e67ff6b4bf-tls-certs\") pod \"test-model-e2dea3-kserve-84f88488c7-n2v8d\" (UID: \"a8ac32f5-6438-4dfa-91ea-12e67ff6b4bf\") " pod="ai-tenant-e2e-inf-a-e2dea3/test-model-e2dea3-kserve-84f88488c7-n2v8d"
Sep 18 20:18:11.723081 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:18:11.722966    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mtndb\" (UniqueName: \"kubernetes.io/projected/a8ac32f5-6438-4dfa-91ea-12e67ff6b4bf-kube-api-access-mtndb\") pod \"test-model-e2dea3-kserve-84f88488c7-n2v8d\" (UID: \"a8ac32f5-6438-4dfa-91ea-12e67ff6b4bf\") " pod="ai-tenant-e2e-inf-a-e2dea3/test-model-e2dea3-kserve-84f88488c7-n2v8d"
Sep 18 20:18:11.723081 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:18:11.722994    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a8ac32f5-6438-4dfa-91ea-12e67ff6b4bf-home\") pod \"test-model-e2dea3-kserve-84f88488c7-n2v8d\" (UID: \"a8ac32f5-6438-4dfa-91ea-12e67ff6b4bf\") " pod="ai-tenant-e2e-inf-a-e2dea3/test-model-e2dea3-kserve-84f88488c7-n2v8d"
Sep 18 20:18:11.723081 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:18:11.723023    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a8ac32f5-6438-4dfa-91ea-12e67ff6b4bf-dshm\") pod \"test-model-e2dea3-kserve-84f88488c7-n2v8d\" (UID: \"a8ac32f5-6438-4dfa-91ea-12e67ff6b4bf\") " pod="ai-tenant-e2e-inf-a-e2dea3/test-model-e2dea3-kserve-84f88488c7-n2v8d"
Sep 18 20:18:11.824139 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:18:11.824105    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a8ac32f5-6438-4dfa-91ea-12e67ff6b4bf-dshm\") pod \"test-model-e2dea3-kserve-84f88488c7-n2v8d\" (UID: \"a8ac32f5-6438-4dfa-91ea-12e67ff6b4bf\") " pod="ai-tenant-e2e-inf-a-e2dea3/test-model-e2dea3-kserve-84f88488c7-n2v8d"
Sep 18 20:18:11.824353 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:18:11.824210    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a8ac32f5-6438-4dfa-91ea-12e67ff6b4bf-model-cache\") pod \"test-model-e2dea3-kserve-84f88488c7-n2v8d\" (UID: \"a8ac32f5-6438-4dfa-91ea-12e67ff6b4bf\") " pod="ai-tenant-e2e-inf-a-e2dea3/test-model-e2dea3-kserve-84f88488c7-n2v8d"
Sep 18 20:18:11.824353 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:18:11.824253    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a8ac32f5-6438-4dfa-91ea-12e67ff6b4bf-tmp-dir\") pod \"test-model-e2dea3-kserve-84f88488c7-n2v8d\" (UID: \"a8ac32f5-6438-4dfa-91ea-12e67ff6b4bf\") " pod="ai-tenant-e2e-inf-a-e2dea3/test-model-e2dea3-kserve-84f88488c7-n2v8d"
Sep 18 20:18:11.824353 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:18:11.824296    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a8ac32f5-6438-4dfa-91ea-12e67ff6b4bf-tls-certs\") pod \"test-model-e2dea3-kserve-84f88488c7-n2v8d\" (UID: \"a8ac32f5-6438-4dfa-91ea-12e67ff6b4bf\") " pod="ai-tenant-e2e-inf-a-e2dea3/test-model-e2dea3-kserve-84f88488c7-n2v8d"
Sep 18 20:18:11.824353 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:18:11.824319    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mtndb\" (UniqueName: \"kubernetes.io/projected/a8ac32f5-6438-4dfa-91ea-12e67ff6b4bf-kube-api-access-mtndb\") pod \"test-model-e2dea3-kserve-84f88488c7-n2v8d\" (UID: \"a8ac32f5-6438-4dfa-91ea-12e67ff6b4bf\") " pod="ai-tenant-e2e-inf-a-e2dea3/test-model-e2dea3-kserve-84f88488c7-n2v8d"
Sep 18 20:18:11.824610 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:18:11.824363    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a8ac32f5-6438-4dfa-91ea-12e67ff6b4bf-home\") pod \"test-model-e2dea3-kserve-84f88488c7-n2v8d\" (UID: \"a8ac32f5-6438-4dfa-91ea-12e67ff6b4bf\") " pod="ai-tenant-e2e-inf-a-e2dea3/test-model-e2dea3-kserve-84f88488c7-n2v8d"
Sep 18 20:18:11.824728 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:18:11.824688    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a8ac32f5-6438-4dfa-91ea-12e67ff6b4bf-model-cache\") pod \"test-model-e2dea3-kserve-84f88488c7-n2v8d\" (UID: \"a8ac32f5-6438-4dfa-91ea-12e67ff6b4bf\") " pod="ai-tenant-e2e-inf-a-e2dea3/test-model-e2dea3-kserve-84f88488c7-n2v8d"
Sep 18 20:18:11.824728 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:18:11.824712    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a8ac32f5-6438-4dfa-91ea-12e67ff6b4bf-tmp-dir\") pod \"test-model-e2dea3-kserve-84f88488c7-n2v8d\" (UID: \"a8ac32f5-6438-4dfa-91ea-12e67ff6b4bf\") " pod="ai-tenant-e2e-inf-a-e2dea3/test-model-e2dea3-kserve-84f88488c7-n2v8d"
Sep 18 20:18:11.824887 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:18:11.824780    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a8ac32f5-6438-4dfa-91ea-12e67ff6b4bf-home\") pod \"test-model-e2dea3-kserve-84f88488c7-n2v8d\" (UID: \"a8ac32f5-6438-4dfa-91ea-12e67ff6b4bf\") " pod="ai-tenant-e2e-inf-a-e2dea3/test-model-e2dea3-kserve-84f88488c7-n2v8d"
Sep 18 20:18:11.826549 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:18:11.826525    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a8ac32f5-6438-4dfa-91ea-12e67ff6b4bf-dshm\") pod \"test-model-e2dea3-kserve-84f88488c7-n2v8d\" (UID: \"a8ac32f5-6438-4dfa-91ea-12e67ff6b4bf\") " pod="ai-tenant-e2e-inf-a-e2dea3/test-model-e2dea3-kserve-84f88488c7-n2v8d"
Sep 18 20:18:11.826786 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:18:11.826771    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a8ac32f5-6438-4dfa-91ea-12e67ff6b4bf-tls-certs\") pod \"test-model-e2dea3-kserve-84f88488c7-n2v8d\" (UID: \"a8ac32f5-6438-4dfa-91ea-12e67ff6b4bf\") " pod="ai-tenant-e2e-inf-a-e2dea3/test-model-e2dea3-kserve-84f88488c7-n2v8d"
Sep 18 20:18:11.833348 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:18:11.833330    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mtndb\" (UniqueName: \"kubernetes.io/projected/a8ac32f5-6438-4dfa-91ea-12e67ff6b4bf-kube-api-access-mtndb\") pod \"test-model-e2dea3-kserve-84f88488c7-n2v8d\" (UID: \"a8ac32f5-6438-4dfa-91ea-12e67ff6b4bf\") " pod="ai-tenant-e2e-inf-a-e2dea3/test-model-e2dea3-kserve-84f88488c7-n2v8d"
Sep 18 20:18:11.913416 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:18:11.913327    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-e2dea3/test-model-e2dea3-kserve-84f88488c7-n2v8d"
Sep 18 20:18:12.184677 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:18:12.184650    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-e2dea3/test-model-e2dea3-kserve-84f88488c7-n2v8d"]
Sep 18 20:18:12.186068 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:18:12.186042    2532 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda8ac32f5_6438_4dfa_91ea_12e67ff6b4bf.slice/crio-5c0bc438407dde4ac234830e6744d451e8eab8aa238147943469e8021c8cbd8e WatchSource:0}: Error finding container 5c0bc438407dde4ac234830e6744d451e8eab8aa238147943469e8021c8cbd8e: Status 404 returned error can't find the container with id 5c0bc438407dde4ac234830e6744d451e8eab8aa238147943469e8021c8cbd8e
Sep 18 20:18:12.611523 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:18:12.611467    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:18:13.066392 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:18:13.066358    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-e2dea3/test-model-e2dea3-kserve-84f88488c7-n2v8d" event={"ID":"a8ac32f5-6438-4dfa-91ea-12e67ff6b4bf","Type":"ContainerStarted","Data":"2d5735a568996d7a605ef4c995967f7509253a15094fad3ba907494061406369"}
Sep 18 20:18:13.066588 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:18:13.066398    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-e2dea3/test-model-e2dea3-kserve-84f88488c7-n2v8d" event={"ID":"a8ac32f5-6438-4dfa-91ea-12e67ff6b4bf","Type":"ContainerStarted","Data":"5c0bc438407dde4ac234830e6744d451e8eab8aa238147943469e8021c8cbd8e"}
Sep 18 20:18:13.083673 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:18:13.083615    2532 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-a-e2dea3/test-model-e2dea3-kserve-84f88488c7-n2v8d" podStartSLOduration=2.08359593 podStartE2EDuration="2.08359593s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 20:18:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 20:18:13.083371721 +0000 UTC m=+1024.848761129" watchObservedRunningTime="2026-09-18 20:18:13.08359593 +0000 UTC m=+1024.848985346"
Sep 18 20:18:13.195609 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:18:13.195579    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:18:13.295752 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:18:13.295718    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:18:13.398326 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:18:13.398232    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:18:13.473839 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:18:13.473801    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:18:13.622343 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:18:13.622305    2532 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-6d00b9e3-gw-openshift-default-c4d74979b-ncsh6"]
Sep 18 20:18:13.622818 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:18:13.622588    2532 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-migrate-6d00b9e3-gw-openshift-default-c4d74979b-ncsh6" podUID="7e11376c-93ad-497f-aab6-cc0b000ff5f7" containerName="istio-proxy" containerID="cri-o://20d2571486e94dcdcc985578718a1f73684619ff000b80013cc6def41699b6ee" gracePeriod=30
Sep 18 20:18:13.668891 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:18:13.668803    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:18:13.670461 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:18:13.670443    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-vptdl"]
Sep 18 20:18:13.905122 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:18:13.905087    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:18:14.056512 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:18:14.056460    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:18:14.384290 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:18:14.384168    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:18:14.570812 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:18:14.570757    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:18:14.670832 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:18:14.670734    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:18:15.199424 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:18:15.199371    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:18:16.010065 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:18:16.010028    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:18:16.446969 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:18:16.446880    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:18:16.628838 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:18:16.628808    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:18:17.204331 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:18:17.204283    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:18:17.297713 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:18:17.297681    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:18:17.529757 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:18:17.529723    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:18:18.367113 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:18:18.367062    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:18:18.875248 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:18:18.875224    2532 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-6d00b9e3-gw-openshift-default-c4d74979b-ncsh6"
Sep 18 20:18:18.995280 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:18:18.995246    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7e11376c-93ad-497f-aab6-cc0b000ff5f7-istio-envoy\") pod \"7e11376c-93ad-497f-aab6-cc0b000ff5f7\" (UID: \"7e11376c-93ad-497f-aab6-cc0b000ff5f7\") "
Sep 18 20:18:18.995280 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:18:18.995283    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7e11376c-93ad-497f-aab6-cc0b000ff5f7-istiod-ca-cert\") pod \"7e11376c-93ad-497f-aab6-cc0b000ff5f7\" (UID: \"7e11376c-93ad-497f-aab6-cc0b000ff5f7\") "
Sep 18 20:18:18.995582 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:18:18.995349    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7e11376c-93ad-497f-aab6-cc0b000ff5f7-istio-token\") pod \"7e11376c-93ad-497f-aab6-cc0b000ff5f7\" (UID: \"7e11376c-93ad-497f-aab6-cc0b000ff5f7\") "
Sep 18 20:18:18.995582 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:18:18.995374    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7e11376c-93ad-497f-aab6-cc0b000ff5f7-workload-certs\") pod \"7e11376c-93ad-497f-aab6-cc0b000ff5f7\" (UID: \"7e11376c-93ad-497f-aab6-cc0b000ff5f7\") "
Sep 18 20:18:18.995582 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:18:18.995396    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7fbdm\" (UniqueName: \"kubernetes.io/projected/7e11376c-93ad-497f-aab6-cc0b000ff5f7-kube-api-access-7fbdm\") pod \"7e11376c-93ad-497f-aab6-cc0b000ff5f7\" (UID: \"7e11376c-93ad-497f-aab6-cc0b000ff5f7\") "
Sep 18 20:18:18.995582 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:18:18.995448    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7e11376c-93ad-497f-aab6-cc0b000ff5f7-istio-data\") pod \"7e11376c-93ad-497f-aab6-cc0b000ff5f7\" (UID: \"7e11376c-93ad-497f-aab6-cc0b000ff5f7\") "
Sep 18 20:18:18.995582 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:18:18.995478    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7e11376c-93ad-497f-aab6-cc0b000ff5f7-credential-socket\") pod \"7e11376c-93ad-497f-aab6-cc0b000ff5f7\" (UID: \"7e11376c-93ad-497f-aab6-cc0b000ff5f7\") "
Sep 18 20:18:18.995582 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:18:18.995539    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7e11376c-93ad-497f-aab6-cc0b000ff5f7-istio-podinfo\") pod \"7e11376c-93ad-497f-aab6-cc0b000ff5f7\" (UID: \"7e11376c-93ad-497f-aab6-cc0b000ff5f7\") "
Sep 18 20:18:18.995582 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:18:18.995581    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7e11376c-93ad-497f-aab6-cc0b000ff5f7-workload-socket\") pod \"7e11376c-93ad-497f-aab6-cc0b000ff5f7\" (UID: \"7e11376c-93ad-497f-aab6-cc0b000ff5f7\") "
Sep 18 20:18:18.995935 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:18:18.995720    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7e11376c-93ad-497f-aab6-cc0b000ff5f7-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "7e11376c-93ad-497f-aab6-cc0b000ff5f7" (UID: "7e11376c-93ad-497f-aab6-cc0b000ff5f7"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 20:18:18.995935 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:18:18.995805    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7e11376c-93ad-497f-aab6-cc0b000ff5f7-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "7e11376c-93ad-497f-aab6-cc0b000ff5f7" (UID: "7e11376c-93ad-497f-aab6-cc0b000ff5f7"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 20:18:18.995935 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:18:18.995798    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7e11376c-93ad-497f-aab6-cc0b000ff5f7-istio-data" (OuterVolumeSpecName: "istio-data") pod "7e11376c-93ad-497f-aab6-cc0b000ff5f7" (UID: "7e11376c-93ad-497f-aab6-cc0b000ff5f7"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 20:18:18.995935 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:18:18.995862    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7e11376c-93ad-497f-aab6-cc0b000ff5f7-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "7e11376c-93ad-497f-aab6-cc0b000ff5f7" (UID: "7e11376c-93ad-497f-aab6-cc0b000ff5f7"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 20:18:18.996104 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:18:18.995943    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7e11376c-93ad-497f-aab6-cc0b000ff5f7-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "7e11376c-93ad-497f-aab6-cc0b000ff5f7" (UID: "7e11376c-93ad-497f-aab6-cc0b000ff5f7"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 20:18:18.996104 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:18:18.995970    2532 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7e11376c-93ad-497f-aab6-cc0b000ff5f7-workload-certs\") on node \"ip-10-0-130-81.ec2.internal\" DevicePath \"\""
Sep 18 20:18:18.996104 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:18:18.995989    2532 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7e11376c-93ad-497f-aab6-cc0b000ff5f7-istio-data\") on node \"ip-10-0-130-81.ec2.internal\" DevicePath \"\""
Sep 18 20:18:18.996104 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:18:18.996005    2532 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7e11376c-93ad-497f-aab6-cc0b000ff5f7-credential-socket\") on node \"ip-10-0-130-81.ec2.internal\" DevicePath \"\""
Sep 18 20:18:18.996104 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:18:18.996018    2532 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7e11376c-93ad-497f-aab6-cc0b000ff5f7-istiod-ca-cert\") on node \"ip-10-0-130-81.ec2.internal\" DevicePath \"\""
Sep 18 20:18:18.997773 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:18:18.997752    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7e11376c-93ad-497f-aab6-cc0b000ff5f7-istio-token" (OuterVolumeSpecName: "istio-token") pod "7e11376c-93ad-497f-aab6-cc0b000ff5f7" (UID: "7e11376c-93ad-497f-aab6-cc0b000ff5f7"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 20:18:18.997874 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:18:18.997784    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/7e11376c-93ad-497f-aab6-cc0b000ff5f7-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "7e11376c-93ad-497f-aab6-cc0b000ff5f7" (UID: "7e11376c-93ad-497f-aab6-cc0b000ff5f7"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 18 20:18:18.998043 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:18:18.998015    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7e11376c-93ad-497f-aab6-cc0b000ff5f7-kube-api-access-7fbdm" (OuterVolumeSpecName: "kube-api-access-7fbdm") pod "7e11376c-93ad-497f-aab6-cc0b000ff5f7" (UID: "7e11376c-93ad-497f-aab6-cc0b000ff5f7"). InnerVolumeSpecName "kube-api-access-7fbdm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 20:18:18.998136 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:18:18.998035    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7e11376c-93ad-497f-aab6-cc0b000ff5f7-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "7e11376c-93ad-497f-aab6-cc0b000ff5f7" (UID: "7e11376c-93ad-497f-aab6-cc0b000ff5f7"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 20:18:19.092775 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:18:19.092745    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:18:19.097228 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:18:19.097198    2532 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7e11376c-93ad-497f-aab6-cc0b000ff5f7-istio-podinfo\") on node \"ip-10-0-130-81.ec2.internal\" DevicePath \"\""
Sep 18 20:18:19.097228 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:18:19.097225    2532 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7e11376c-93ad-497f-aab6-cc0b000ff5f7-workload-socket\") on node \"ip-10-0-130-81.ec2.internal\" DevicePath \"\""
Sep 18 20:18:19.097228 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:18:19.097234    2532 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7e11376c-93ad-497f-aab6-cc0b000ff5f7-istio-envoy\") on node \"ip-10-0-130-81.ec2.internal\" DevicePath \"\""
Sep 18 20:18:19.097430 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:18:19.097243    2532 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7e11376c-93ad-497f-aab6-cc0b000ff5f7-istio-token\") on node \"ip-10-0-130-81.ec2.internal\" DevicePath \"\""
Sep 18 20:18:19.097430 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:18:19.097251    2532 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7fbdm\" (UniqueName: \"kubernetes.io/projected/7e11376c-93ad-497f-aab6-cc0b000ff5f7-kube-api-access-7fbdm\") on node \"ip-10-0-130-81.ec2.internal\" DevicePath \"\""
Sep 18 20:18:19.099701 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:18:19.099679    2532 generic.go:356] "Generic (PLEG): container finished" podID="7e11376c-93ad-497f-aab6-cc0b000ff5f7" containerID="20d2571486e94dcdcc985578718a1f73684619ff000b80013cc6def41699b6ee" exitCode=0
Sep 18 20:18:19.099787 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:18:19.099729    2532 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-6d00b9e3-gw-openshift-default-c4d74979b-ncsh6"
Sep 18 20:18:19.099787 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:18:19.099772    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-6d00b9e3-gw-openshift-default-c4d74979b-ncsh6" event={"ID":"7e11376c-93ad-497f-aab6-cc0b000ff5f7","Type":"ContainerDied","Data":"20d2571486e94dcdcc985578718a1f73684619ff000b80013cc6def41699b6ee"}
Sep 18 20:18:19.099871 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:18:19.099830    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-6d00b9e3-gw-openshift-default-c4d74979b-ncsh6" event={"ID":"7e11376c-93ad-497f-aab6-cc0b000ff5f7","Type":"ContainerDied","Data":"72133f2d6bafa70cf544f312ac3036f4ae64e4e969f6727d9d5551e1f8b5b305"}
Sep 18 20:18:19.099871 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:18:19.099858    2532 scope.go:117] "RemoveContainer" containerID="20d2571486e94dcdcc985578718a1f73684619ff000b80013cc6def41699b6ee"
Sep 18 20:18:19.109651 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:18:19.109629    2532 scope.go:117] "RemoveContainer" containerID="36352a42a8d4788405cb430d541cba1534eff6607182ac7fa078903c11ed0969"
Sep 18 20:18:19.117756 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:18:19.117735    2532 scope.go:117] "RemoveContainer" containerID="20d2571486e94dcdcc985578718a1f73684619ff000b80013cc6def41699b6ee"
Sep 18 20:18:19.118009 ip-10-0-130-81 kubenswrapper[2532]: E0918 20:18:19.117990    2532 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"20d2571486e94dcdcc985578718a1f73684619ff000b80013cc6def41699b6ee\": container with ID starting with 20d2571486e94dcdcc985578718a1f73684619ff000b80013cc6def41699b6ee not found: ID does not exist" containerID="20d2571486e94dcdcc985578718a1f73684619ff000b80013cc6def41699b6ee"
Sep 18 20:18:19.118086 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:18:19.118016    2532 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"20d2571486e94dcdcc985578718a1f73684619ff000b80013cc6def41699b6ee"} err="failed to get container status \"20d2571486e94dcdcc985578718a1f73684619ff000b80013cc6def41699b6ee\": rpc error: code = NotFound desc = could not find container \"20d2571486e94dcdcc985578718a1f73684619ff000b80013cc6def41699b6ee\": container with ID starting with 20d2571486e94dcdcc985578718a1f73684619ff000b80013cc6def41699b6ee not found: ID does not exist"
Sep 18 20:18:19.118086 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:18:19.118032    2532 scope.go:117] "RemoveContainer" containerID="36352a42a8d4788405cb430d541cba1534eff6607182ac7fa078903c11ed0969"
Sep 18 20:18:19.118287 ip-10-0-130-81 kubenswrapper[2532]: E0918 20:18:19.118269    2532 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"36352a42a8d4788405cb430d541cba1534eff6607182ac7fa078903c11ed0969\": container with ID starting with 36352a42a8d4788405cb430d541cba1534eff6607182ac7fa078903c11ed0969 not found: ID does not exist" containerID="36352a42a8d4788405cb430d541cba1534eff6607182ac7fa078903c11ed0969"
Sep 18 20:18:19.118326 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:18:19.118289    2532 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"36352a42a8d4788405cb430d541cba1534eff6607182ac7fa078903c11ed0969"} err="failed to get container status \"36352a42a8d4788405cb430d541cba1534eff6607182ac7fa078903c11ed0969\": rpc error: code = NotFound desc = could not find container \"36352a42a8d4788405cb430d541cba1534eff6607182ac7fa078903c11ed0969\": container with ID starting with 36352a42a8d4788405cb430d541cba1534eff6607182ac7fa078903c11ed0969 not found: ID does not exist"
Sep 18 20:18:19.123908 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:18:19.123886    2532 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-6d00b9e3-gw-openshift-default-c4d74979b-ncsh6"]
Sep 18 20:18:19.128120 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:18:19.128095    2532 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-migrate-6d00b9e3-gw-openshift-default-c4d74979b-ncsh6"]
Sep 18 20:18:19.932816 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:18:19.932774    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:18:20.306374 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:18:20.306333    2532 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7e11376c-93ad-497f-aab6-cc0b000ff5f7" path="/var/lib/kubelet/pods/7e11376c-93ad-497f-aab6-cc0b000ff5f7/volumes"
Sep 18 20:18:20.593716 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:18:20.593640    2532 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-f88d7657-65d5465c95-fkgz5"]
Sep 18 20:18:20.594172 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:18:20.594154    2532 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7e11376c-93ad-497f-aab6-cc0b000ff5f7" containerName="istio-proxy"
Sep 18 20:18:20.594218 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:18:20.594177    2532 state_mem.go:107] "Deleted CPUSet assignment" podUID="7e11376c-93ad-497f-aab6-cc0b000ff5f7" containerName="istio-proxy"
Sep 18 20:18:20.594262 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:18:20.594236    2532 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7e11376c-93ad-497f-aab6-cc0b000ff5f7" containerName="istio-proxy"
Sep 18 20:18:20.594262 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:18:20.594245    2532 state_mem.go:107] "Deleted CPUSet assignment" podUID="7e11376c-93ad-497f-aab6-cc0b000ff5f7" containerName="istio-proxy"
Sep 18 20:18:20.594355 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:18:20.594343    2532 memory_manager.go:368] "RemoveStaleState removing state" podUID="7e11376c-93ad-497f-aab6-cc0b000ff5f7" containerName="istio-proxy"
Sep 18 20:18:20.594391 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:18:20.594359    2532 memory_manager.go:368] "RemoveStaleState removing state" podUID="7e11376c-93ad-497f-aab6-cc0b000ff5f7" containerName="istio-proxy"
Sep 18 20:18:20.598655 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:18:20.598626    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f88d7657-65d5465c95-fkgz5"
Sep 18 20:18:20.601316 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:18:20.601290    2532 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-f88d7657-serving-cert\""
Sep 18 20:18:20.609750 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:18:20.609716    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-f88d7657-65d5465c95-fkgz5"]
Sep 18 20:18:20.626942 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:18:20.626907    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:18:20.686672 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:18:20.686632    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:18:20.711897 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:18:20.711862    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/de2f9f42-d157-4b92-b15e-471d5598cc83-metrics-certs\") pod \"maas-api-e2e-ait-f88d7657-65d5465c95-fkgz5\" (UID: \"de2f9f42-d157-4b92-b15e-471d5598cc83\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f88d7657-65d5465c95-fkgz5"
Sep 18 20:18:20.711897 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:18:20.711900    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/de2f9f42-d157-4b92-b15e-471d5598cc83-openshift-service-ca\") pod \"maas-api-e2e-ait-f88d7657-65d5465c95-fkgz5\" (UID: \"de2f9f42-d157-4b92-b15e-471d5598cc83\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f88d7657-65d5465c95-fkgz5"
Sep 18 20:18:20.712175 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:18:20.711933    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/de2f9f42-d157-4b92-b15e-471d5598cc83-maas-api-tls\") pod \"maas-api-e2e-ait-f88d7657-65d5465c95-fkgz5\" (UID: \"de2f9f42-d157-4b92-b15e-471d5598cc83\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f88d7657-65d5465c95-fkgz5"
Sep 18 20:18:20.712233 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:18:20.712181    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bh5zz\" (UniqueName: \"kubernetes.io/projected/de2f9f42-d157-4b92-b15e-471d5598cc83-kube-api-access-bh5zz\") pod \"maas-api-e2e-ait-f88d7657-65d5465c95-fkgz5\" (UID: \"de2f9f42-d157-4b92-b15e-471d5598cc83\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f88d7657-65d5465c95-fkgz5"
Sep 18 20:18:20.791236 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:18:20.791204    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:18:20.812765 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:18:20.812733    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/de2f9f42-d157-4b92-b15e-471d5598cc83-metrics-certs\") pod \"maas-api-e2e-ait-f88d7657-65d5465c95-fkgz5\" (UID: \"de2f9f42-d157-4b92-b15e-471d5598cc83\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f88d7657-65d5465c95-fkgz5"
Sep 18 20:18:20.812765 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:18:20.812768    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/de2f9f42-d157-4b92-b15e-471d5598cc83-openshift-service-ca\") pod \"maas-api-e2e-ait-f88d7657-65d5465c95-fkgz5\" (UID: \"de2f9f42-d157-4b92-b15e-471d5598cc83\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f88d7657-65d5465c95-fkgz5"
Sep 18 20:18:20.813037 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:18:20.812792    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/de2f9f42-d157-4b92-b15e-471d5598cc83-maas-api-tls\") pod \"maas-api-e2e-ait-f88d7657-65d5465c95-fkgz5\" (UID: \"de2f9f42-d157-4b92-b15e-471d5598cc83\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f88d7657-65d5465c95-fkgz5"
Sep 18 20:18:20.813037 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:18:20.812880    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bh5zz\" (UniqueName: \"kubernetes.io/projected/de2f9f42-d157-4b92-b15e-471d5598cc83-kube-api-access-bh5zz\") pod \"maas-api-e2e-ait-f88d7657-65d5465c95-fkgz5\" (UID: \"de2f9f42-d157-4b92-b15e-471d5598cc83\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f88d7657-65d5465c95-fkgz5"
Sep 18 20:18:20.813460 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:18:20.813435    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/de2f9f42-d157-4b92-b15e-471d5598cc83-openshift-service-ca\") pod \"maas-api-e2e-ait-f88d7657-65d5465c95-fkgz5\" (UID: \"de2f9f42-d157-4b92-b15e-471d5598cc83\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f88d7657-65d5465c95-fkgz5"
Sep 18 20:18:20.815460 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:18:20.815434    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/de2f9f42-d157-4b92-b15e-471d5598cc83-metrics-certs\") pod \"maas-api-e2e-ait-f88d7657-65d5465c95-fkgz5\" (UID: \"de2f9f42-d157-4b92-b15e-471d5598cc83\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f88d7657-65d5465c95-fkgz5"
Sep 18 20:18:20.815622 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:18:20.815433    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/de2f9f42-d157-4b92-b15e-471d5598cc83-maas-api-tls\") pod \"maas-api-e2e-ait-f88d7657-65d5465c95-fkgz5\" (UID: \"de2f9f42-d157-4b92-b15e-471d5598cc83\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f88d7657-65d5465c95-fkgz5"
Sep 18 20:18:20.822093 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:18:20.822058    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bh5zz\" (UniqueName: \"kubernetes.io/projected/de2f9f42-d157-4b92-b15e-471d5598cc83-kube-api-access-bh5zz\") pod \"maas-api-e2e-ait-f88d7657-65d5465c95-fkgz5\" (UID: \"de2f9f42-d157-4b92-b15e-471d5598cc83\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f88d7657-65d5465c95-fkgz5"
Sep 18 20:18:20.913725 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:18:20.913633    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f88d7657-65d5465c95-fkgz5"
Sep 18 20:18:21.162085 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:18:21.162058    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:18:21.197883 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:18:21.197656    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-f88d7657-65d5465c95-fkgz5"]
Sep 18 20:18:21.201258 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:18:21.201221    2532 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podde2f9f42_d157_4b92_b15e_471d5598cc83.slice/crio-ddfa105382f209bd348e10dd31c66408ee8777dcf5090eb3f37dae81eea12b5a WatchSource:0}: Error finding container ddfa105382f209bd348e10dd31c66408ee8777dcf5090eb3f37dae81eea12b5a: Status 404 returned error can't find the container with id ddfa105382f209bd348e10dd31c66408ee8777dcf5090eb3f37dae81eea12b5a
Sep 18 20:18:21.265179 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:18:21.265131    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:18:21.913871 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:18:21.913776    2532 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-a-e2dea3/test-model-e2dea3-kserve-84f88488c7-n2v8d"
Sep 18 20:18:21.913871 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:18:21.913854    2532 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-a-e2dea3/test-model-e2dea3-kserve-84f88488c7-n2v8d"
Sep 18 20:18:21.926747 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:18:21.926726    2532 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-a-e2dea3/test-model-e2dea3-kserve-84f88488c7-n2v8d"
Sep 18 20:18:22.118658 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:18:22.118622    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f88d7657-65d5465c95-fkgz5" event={"ID":"de2f9f42-d157-4b92-b15e-471d5598cc83","Type":"ContainerStarted","Data":"5d2de6433ebeea48b15fa713dedfd9bb9522676b7b388b210d7ddff644d0bd82"}
Sep 18 20:18:22.118831 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:18:22.118676    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f88d7657-65d5465c95-fkgz5" event={"ID":"de2f9f42-d157-4b92-b15e-471d5598cc83","Type":"ContainerStarted","Data":"ddfa105382f209bd348e10dd31c66408ee8777dcf5090eb3f37dae81eea12b5a"}
Sep 18 20:18:22.118831 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:18:22.118750    2532 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f88d7657-65d5465c95-fkgz5"
Sep 18 20:18:22.130143 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:18:22.130115    2532 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-a-e2dea3/test-model-e2dea3-kserve-84f88488c7-n2v8d"
Sep 18 20:18:22.135896 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:18:22.135842    2532 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f88d7657-65d5465c95-fkgz5" podStartSLOduration=1.754997898 podStartE2EDuration="2.135823191s" totalImagesPullingTime="380.825293ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 20:18:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 20:18:22.135673179 +0000 UTC m=+1033.901062579" watchObservedRunningTime="2026-09-18 20:18:22.135823191 +0000 UTC m=+1033.901212593"
Sep 18 20:18:28.129897 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:18:28.129867    2532 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f88d7657-65d5465c95-fkgz5"
Sep 18 20:18:29.673476 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:18:29.673438    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:18:30.901028 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:18:30.900989    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-vptdl"]
Sep 18 20:18:42.628057 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:18:42.628020    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:18:43.308722 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:18:43.308689    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:18:44.029176 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:18:44.029122    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:18:44.523279 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:18:44.523245    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:18:44.622368 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:18:44.622338    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:18:44.722228 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:18:44.722193    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:18:45.423289 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:18:45.423257    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:18:46.036704 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:18:46.036654    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:18:49.008068 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:18:49.008026    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:18:49.700637 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:18:49.700593    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:18:50.247214 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:18:50.247177    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:18:50.860369 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:18:50.860334    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:18:51.629968 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:18:51.629941    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:18:52.275065 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:18:52.275035    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:18:58.985916 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:18:58.985881    2532 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-f88d7657-65d5465c95-fkgz5"]
Sep 18 20:18:58.986387 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:18:58.986151    2532 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f88d7657-65d5465c95-fkgz5" podUID="de2f9f42-d157-4b92-b15e-471d5598cc83" containerName="maas-api" containerID="cri-o://5d2de6433ebeea48b15fa713dedfd9bb9522676b7b388b210d7ddff644d0bd82" gracePeriod=30
Sep 18 20:18:59.016439 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:18:59.016398    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:18:59.079334 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:18:59.079300    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:18:59.145251 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:18:59.145218    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:18:59.232008 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:18:59.231986    2532 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f88d7657-65d5465c95-fkgz5"
Sep 18 20:18:59.236678 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:18:59.236616    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:18:59.328933 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:18:59.328896    2532 generic.go:356] "Generic (PLEG): container finished" podID="de2f9f42-d157-4b92-b15e-471d5598cc83" containerID="5d2de6433ebeea48b15fa713dedfd9bb9522676b7b388b210d7ddff644d0bd82" exitCode=0
Sep 18 20:18:59.328933 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:18:59.328942    2532 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f88d7657-65d5465c95-fkgz5"
Sep 18 20:18:59.329217 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:18:59.328969    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f88d7657-65d5465c95-fkgz5" event={"ID":"de2f9f42-d157-4b92-b15e-471d5598cc83","Type":"ContainerDied","Data":"5d2de6433ebeea48b15fa713dedfd9bb9522676b7b388b210d7ddff644d0bd82"}
Sep 18 20:18:59.329217 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:18:59.329027    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f88d7657-65d5465c95-fkgz5" event={"ID":"de2f9f42-d157-4b92-b15e-471d5598cc83","Type":"ContainerDied","Data":"ddfa105382f209bd348e10dd31c66408ee8777dcf5090eb3f37dae81eea12b5a"}
Sep 18 20:18:59.329217 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:18:59.329049    2532 scope.go:117] "RemoveContainer" containerID="5d2de6433ebeea48b15fa713dedfd9bb9522676b7b388b210d7ddff644d0bd82"
Sep 18 20:18:59.338358 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:18:59.338341    2532 scope.go:117] "RemoveContainer" containerID="5d2de6433ebeea48b15fa713dedfd9bb9522676b7b388b210d7ddff644d0bd82"
Sep 18 20:18:59.338699 ip-10-0-130-81 kubenswrapper[2532]: E0918 20:18:59.338677    2532 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5d2de6433ebeea48b15fa713dedfd9bb9522676b7b388b210d7ddff644d0bd82\": container with ID starting with 5d2de6433ebeea48b15fa713dedfd9bb9522676b7b388b210d7ddff644d0bd82 not found: ID does not exist" containerID="5d2de6433ebeea48b15fa713dedfd9bb9522676b7b388b210d7ddff644d0bd82"
Sep 18 20:18:59.338765 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:18:59.338714    2532 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5d2de6433ebeea48b15fa713dedfd9bb9522676b7b388b210d7ddff644d0bd82"} err="failed to get container status \"5d2de6433ebeea48b15fa713dedfd9bb9522676b7b388b210d7ddff644d0bd82\": rpc error: code = NotFound desc = could not find container \"5d2de6433ebeea48b15fa713dedfd9bb9522676b7b388b210d7ddff644d0bd82\": container with ID starting with 5d2de6433ebeea48b15fa713dedfd9bb9522676b7b388b210d7ddff644d0bd82 not found: ID does not exist"
Sep 18 20:18:59.387063 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:18:59.387027    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/de2f9f42-d157-4b92-b15e-471d5598cc83-maas-api-tls\") pod \"de2f9f42-d157-4b92-b15e-471d5598cc83\" (UID: \"de2f9f42-d157-4b92-b15e-471d5598cc83\") "
Sep 18 20:18:59.387227 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:18:59.387130    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/de2f9f42-d157-4b92-b15e-471d5598cc83-openshift-service-ca\") pod \"de2f9f42-d157-4b92-b15e-471d5598cc83\" (UID: \"de2f9f42-d157-4b92-b15e-471d5598cc83\") "
Sep 18 20:18:59.387227 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:18:59.387149    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bh5zz\" (UniqueName: \"kubernetes.io/projected/de2f9f42-d157-4b92-b15e-471d5598cc83-kube-api-access-bh5zz\") pod \"de2f9f42-d157-4b92-b15e-471d5598cc83\" (UID: \"de2f9f42-d157-4b92-b15e-471d5598cc83\") "
Sep 18 20:18:59.387343 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:18:59.387321    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/de2f9f42-d157-4b92-b15e-471d5598cc83-metrics-certs\") pod \"de2f9f42-d157-4b92-b15e-471d5598cc83\" (UID: \"de2f9f42-d157-4b92-b15e-471d5598cc83\") "
Sep 18 20:18:59.387530 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:18:59.387506    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/de2f9f42-d157-4b92-b15e-471d5598cc83-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "de2f9f42-d157-4b92-b15e-471d5598cc83" (UID: "de2f9f42-d157-4b92-b15e-471d5598cc83"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 20:18:59.387710 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:18:59.387694    2532 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/de2f9f42-d157-4b92-b15e-471d5598cc83-openshift-service-ca\") on node \"ip-10-0-130-81.ec2.internal\" DevicePath \"\""
Sep 18 20:18:59.389321 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:18:59.389293    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/de2f9f42-d157-4b92-b15e-471d5598cc83-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "de2f9f42-d157-4b92-b15e-471d5598cc83" (UID: "de2f9f42-d157-4b92-b15e-471d5598cc83"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 20:18:59.389426 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:18:59.389367    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/de2f9f42-d157-4b92-b15e-471d5598cc83-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "de2f9f42-d157-4b92-b15e-471d5598cc83" (UID: "de2f9f42-d157-4b92-b15e-471d5598cc83"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 20:18:59.389426 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:18:59.389403    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/de2f9f42-d157-4b92-b15e-471d5598cc83-kube-api-access-bh5zz" (OuterVolumeSpecName: "kube-api-access-bh5zz") pod "de2f9f42-d157-4b92-b15e-471d5598cc83" (UID: "de2f9f42-d157-4b92-b15e-471d5598cc83"). InnerVolumeSpecName "kube-api-access-bh5zz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 20:18:59.488950 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:18:59.488863    2532 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/de2f9f42-d157-4b92-b15e-471d5598cc83-maas-api-tls\") on node \"ip-10-0-130-81.ec2.internal\" DevicePath \"\""
Sep 18 20:18:59.488950 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:18:59.488894    2532 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bh5zz\" (UniqueName: \"kubernetes.io/projected/de2f9f42-d157-4b92-b15e-471d5598cc83-kube-api-access-bh5zz\") on node \"ip-10-0-130-81.ec2.internal\" DevicePath \"\""
Sep 18 20:18:59.488950 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:18:59.488904    2532 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/de2f9f42-d157-4b92-b15e-471d5598cc83-metrics-certs\") on node \"ip-10-0-130-81.ec2.internal\" DevicePath \"\""
Sep 18 20:18:59.603063 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:18:59.603024    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:18:59.656357 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:18:59.656324    2532 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-f88d7657-65d5465c95-fkgz5"]
Sep 18 20:18:59.660112 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:18:59.660091    2532 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-f88d7657-65d5465c95-fkgz5"]
Sep 18 20:19:00.306994 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:19:00.306964    2532 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="de2f9f42-d157-4b92-b15e-471d5598cc83" path="/var/lib/kubelet/pods/de2f9f42-d157-4b92-b15e-471d5598cc83/volumes"
Sep 18 20:19:00.328035 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:19:00.328001    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:19:00.430601 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:19:00.430566    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:19:00.811459 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:19:00.811421    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:19:00.990690 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:19:00.990655    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:19:01.105981 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:19:01.105909    2532 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-derive-6e56ebc0-openshift-default-76d6dff56d-qwbdn"]
Sep 18 20:19:01.106298 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:19:01.106280    2532 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="de2f9f42-d157-4b92-b15e-471d5598cc83" containerName="maas-api"
Sep 18 20:19:01.106298 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:19:01.106296    2532 state_mem.go:107] "Deleted CPUSet assignment" podUID="de2f9f42-d157-4b92-b15e-471d5598cc83" containerName="maas-api"
Sep 18 20:19:01.106470 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:19:01.106372    2532 memory_manager.go:368] "RemoveStaleState removing state" podUID="de2f9f42-d157-4b92-b15e-471d5598cc83" containerName="maas-api"
Sep 18 20:19:01.108831 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:19:01.108814    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-6e56ebc0-openshift-default-76d6dff56d-qwbdn"
Sep 18 20:19:01.111569 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:19:01.111540    2532 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-derive-6e56ebc0-openshift-default-dockercfg-fm9bn\""
Sep 18 20:19:01.122798 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:19:01.122777    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-6e56ebc0-openshift-default-76d6dff56d-qwbdn"]
Sep 18 20:19:01.130791 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:19:01.130765    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:19:01.202620 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:19:01.202591    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/da42bbbc-b3fb-48dd-841e-e193c95ba582-istiod-ca-cert\") pod \"e2e-derive-6e56ebc0-openshift-default-76d6dff56d-qwbdn\" (UID: \"da42bbbc-b3fb-48dd-841e-e193c95ba582\") " pod="openshift-ingress/e2e-derive-6e56ebc0-openshift-default-76d6dff56d-qwbdn"
Sep 18 20:19:01.202813 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:19:01.202627    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/da42bbbc-b3fb-48dd-841e-e193c95ba582-istio-podinfo\") pod \"e2e-derive-6e56ebc0-openshift-default-76d6dff56d-qwbdn\" (UID: \"da42bbbc-b3fb-48dd-841e-e193c95ba582\") " pod="openshift-ingress/e2e-derive-6e56ebc0-openshift-default-76d6dff56d-qwbdn"
Sep 18 20:19:01.202813 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:19:01.202646    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8j2jp\" (UniqueName: \"kubernetes.io/projected/da42bbbc-b3fb-48dd-841e-e193c95ba582-kube-api-access-8j2jp\") pod \"e2e-derive-6e56ebc0-openshift-default-76d6dff56d-qwbdn\" (UID: \"da42bbbc-b3fb-48dd-841e-e193c95ba582\") " pod="openshift-ingress/e2e-derive-6e56ebc0-openshift-default-76d6dff56d-qwbdn"
Sep 18 20:19:01.202813 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:19:01.202677    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/da42bbbc-b3fb-48dd-841e-e193c95ba582-istio-data\") pod \"e2e-derive-6e56ebc0-openshift-default-76d6dff56d-qwbdn\" (UID: \"da42bbbc-b3fb-48dd-841e-e193c95ba582\") " pod="openshift-ingress/e2e-derive-6e56ebc0-openshift-default-76d6dff56d-qwbdn"
Sep 18 20:19:01.202813 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:19:01.202749    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/da42bbbc-b3fb-48dd-841e-e193c95ba582-istio-envoy\") pod \"e2e-derive-6e56ebc0-openshift-default-76d6dff56d-qwbdn\" (UID: \"da42bbbc-b3fb-48dd-841e-e193c95ba582\") " pod="openshift-ingress/e2e-derive-6e56ebc0-openshift-default-76d6dff56d-qwbdn"
Sep 18 20:19:01.203046 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:19:01.202838    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/da42bbbc-b3fb-48dd-841e-e193c95ba582-workload-socket\") pod \"e2e-derive-6e56ebc0-openshift-default-76d6dff56d-qwbdn\" (UID: \"da42bbbc-b3fb-48dd-841e-e193c95ba582\") " pod="openshift-ingress/e2e-derive-6e56ebc0-openshift-default-76d6dff56d-qwbdn"
Sep 18 20:19:01.203046 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:19:01.202873    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/da42bbbc-b3fb-48dd-841e-e193c95ba582-workload-certs\") pod \"e2e-derive-6e56ebc0-openshift-default-76d6dff56d-qwbdn\" (UID: \"da42bbbc-b3fb-48dd-841e-e193c95ba582\") " pod="openshift-ingress/e2e-derive-6e56ebc0-openshift-default-76d6dff56d-qwbdn"
Sep 18 20:19:01.203046 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:19:01.202906    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/da42bbbc-b3fb-48dd-841e-e193c95ba582-credential-socket\") pod \"e2e-derive-6e56ebc0-openshift-default-76d6dff56d-qwbdn\" (UID: \"da42bbbc-b3fb-48dd-841e-e193c95ba582\") " pod="openshift-ingress/e2e-derive-6e56ebc0-openshift-default-76d6dff56d-qwbdn"
Sep 18 20:19:01.203046 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:19:01.202999    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/da42bbbc-b3fb-48dd-841e-e193c95ba582-istio-token\") pod \"e2e-derive-6e56ebc0-openshift-default-76d6dff56d-qwbdn\" (UID: \"da42bbbc-b3fb-48dd-841e-e193c95ba582\") " pod="openshift-ingress/e2e-derive-6e56ebc0-openshift-default-76d6dff56d-qwbdn"
Sep 18 20:19:01.304507 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:19:01.304456    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/da42bbbc-b3fb-48dd-841e-e193c95ba582-istio-podinfo\") pod \"e2e-derive-6e56ebc0-openshift-default-76d6dff56d-qwbdn\" (UID: \"da42bbbc-b3fb-48dd-841e-e193c95ba582\") " pod="openshift-ingress/e2e-derive-6e56ebc0-openshift-default-76d6dff56d-qwbdn"
Sep 18 20:19:01.304654 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:19:01.304519    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8j2jp\" (UniqueName: \"kubernetes.io/projected/da42bbbc-b3fb-48dd-841e-e193c95ba582-kube-api-access-8j2jp\") pod \"e2e-derive-6e56ebc0-openshift-default-76d6dff56d-qwbdn\" (UID: \"da42bbbc-b3fb-48dd-841e-e193c95ba582\") " pod="openshift-ingress/e2e-derive-6e56ebc0-openshift-default-76d6dff56d-qwbdn"
Sep 18 20:19:01.304654 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:19:01.304574    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/da42bbbc-b3fb-48dd-841e-e193c95ba582-istio-data\") pod \"e2e-derive-6e56ebc0-openshift-default-76d6dff56d-qwbdn\" (UID: \"da42bbbc-b3fb-48dd-841e-e193c95ba582\") " pod="openshift-ingress/e2e-derive-6e56ebc0-openshift-default-76d6dff56d-qwbdn"
Sep 18 20:19:01.304654 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:19:01.304602    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/da42bbbc-b3fb-48dd-841e-e193c95ba582-istio-envoy\") pod \"e2e-derive-6e56ebc0-openshift-default-76d6dff56d-qwbdn\" (UID: \"da42bbbc-b3fb-48dd-841e-e193c95ba582\") " pod="openshift-ingress/e2e-derive-6e56ebc0-openshift-default-76d6dff56d-qwbdn"
Sep 18 20:19:01.304654 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:19:01.304637    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/da42bbbc-b3fb-48dd-841e-e193c95ba582-workload-socket\") pod \"e2e-derive-6e56ebc0-openshift-default-76d6dff56d-qwbdn\" (UID: \"da42bbbc-b3fb-48dd-841e-e193c95ba582\") " pod="openshift-ingress/e2e-derive-6e56ebc0-openshift-default-76d6dff56d-qwbdn"
Sep 18 20:19:01.304886 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:19:01.304665    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/da42bbbc-b3fb-48dd-841e-e193c95ba582-workload-certs\") pod \"e2e-derive-6e56ebc0-openshift-default-76d6dff56d-qwbdn\" (UID: \"da42bbbc-b3fb-48dd-841e-e193c95ba582\") " pod="openshift-ingress/e2e-derive-6e56ebc0-openshift-default-76d6dff56d-qwbdn"
Sep 18 20:19:01.304886 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:19:01.304695    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/da42bbbc-b3fb-48dd-841e-e193c95ba582-credential-socket\") pod \"e2e-derive-6e56ebc0-openshift-default-76d6dff56d-qwbdn\" (UID: \"da42bbbc-b3fb-48dd-841e-e193c95ba582\") " pod="openshift-ingress/e2e-derive-6e56ebc0-openshift-default-76d6dff56d-qwbdn"
Sep 18 20:19:01.304886 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:19:01.304776    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/da42bbbc-b3fb-48dd-841e-e193c95ba582-istio-token\") pod \"e2e-derive-6e56ebc0-openshift-default-76d6dff56d-qwbdn\" (UID: \"da42bbbc-b3fb-48dd-841e-e193c95ba582\") " pod="openshift-ingress/e2e-derive-6e56ebc0-openshift-default-76d6dff56d-qwbdn"
Sep 18 20:19:01.304886 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:19:01.304853    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/da42bbbc-b3fb-48dd-841e-e193c95ba582-istiod-ca-cert\") pod \"e2e-derive-6e56ebc0-openshift-default-76d6dff56d-qwbdn\" (UID: \"da42bbbc-b3fb-48dd-841e-e193c95ba582\") " pod="openshift-ingress/e2e-derive-6e56ebc0-openshift-default-76d6dff56d-qwbdn"
Sep 18 20:19:01.305194 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:19:01.305147    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/da42bbbc-b3fb-48dd-841e-e193c95ba582-workload-socket\") pod \"e2e-derive-6e56ebc0-openshift-default-76d6dff56d-qwbdn\" (UID: \"da42bbbc-b3fb-48dd-841e-e193c95ba582\") " pod="openshift-ingress/e2e-derive-6e56ebc0-openshift-default-76d6dff56d-qwbdn"
Sep 18 20:19:01.305194 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:19:01.305178    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/da42bbbc-b3fb-48dd-841e-e193c95ba582-istio-data\") pod \"e2e-derive-6e56ebc0-openshift-default-76d6dff56d-qwbdn\" (UID: \"da42bbbc-b3fb-48dd-841e-e193c95ba582\") " pod="openshift-ingress/e2e-derive-6e56ebc0-openshift-default-76d6dff56d-qwbdn"
Sep 18 20:19:01.305387 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:19:01.305231    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/da42bbbc-b3fb-48dd-841e-e193c95ba582-workload-certs\") pod \"e2e-derive-6e56ebc0-openshift-default-76d6dff56d-qwbdn\" (UID: \"da42bbbc-b3fb-48dd-841e-e193c95ba582\") " pod="openshift-ingress/e2e-derive-6e56ebc0-openshift-default-76d6dff56d-qwbdn"
Sep 18 20:19:01.305387 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:19:01.305245    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/da42bbbc-b3fb-48dd-841e-e193c95ba582-credential-socket\") pod \"e2e-derive-6e56ebc0-openshift-default-76d6dff56d-qwbdn\" (UID: \"da42bbbc-b3fb-48dd-841e-e193c95ba582\") " pod="openshift-ingress/e2e-derive-6e56ebc0-openshift-default-76d6dff56d-qwbdn"
Sep 18 20:19:01.306052 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:19:01.306028    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/da42bbbc-b3fb-48dd-841e-e193c95ba582-istiod-ca-cert\") pod \"e2e-derive-6e56ebc0-openshift-default-76d6dff56d-qwbdn\" (UID: \"da42bbbc-b3fb-48dd-841e-e193c95ba582\") " pod="openshift-ingress/e2e-derive-6e56ebc0-openshift-default-76d6dff56d-qwbdn"
Sep 18 20:19:01.306910 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:19:01.306892    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/da42bbbc-b3fb-48dd-841e-e193c95ba582-istio-envoy\") pod \"e2e-derive-6e56ebc0-openshift-default-76d6dff56d-qwbdn\" (UID: \"da42bbbc-b3fb-48dd-841e-e193c95ba582\") " pod="openshift-ingress/e2e-derive-6e56ebc0-openshift-default-76d6dff56d-qwbdn"
Sep 18 20:19:01.307234 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:19:01.307207    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/da42bbbc-b3fb-48dd-841e-e193c95ba582-istio-podinfo\") pod \"e2e-derive-6e56ebc0-openshift-default-76d6dff56d-qwbdn\" (UID: \"da42bbbc-b3fb-48dd-841e-e193c95ba582\") " pod="openshift-ingress/e2e-derive-6e56ebc0-openshift-default-76d6dff56d-qwbdn"
Sep 18 20:19:01.312589 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:19:01.312565    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8j2jp\" (UniqueName: \"kubernetes.io/projected/da42bbbc-b3fb-48dd-841e-e193c95ba582-kube-api-access-8j2jp\") pod \"e2e-derive-6e56ebc0-openshift-default-76d6dff56d-qwbdn\" (UID: \"da42bbbc-b3fb-48dd-841e-e193c95ba582\") " pod="openshift-ingress/e2e-derive-6e56ebc0-openshift-default-76d6dff56d-qwbdn"
Sep 18 20:19:01.312743 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:19:01.312727    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/da42bbbc-b3fb-48dd-841e-e193c95ba582-istio-token\") pod \"e2e-derive-6e56ebc0-openshift-default-76d6dff56d-qwbdn\" (UID: \"da42bbbc-b3fb-48dd-841e-e193c95ba582\") " pod="openshift-ingress/e2e-derive-6e56ebc0-openshift-default-76d6dff56d-qwbdn"
Sep 18 20:19:01.341862 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:19:01.341831    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:19:01.420265 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:19:01.420169    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-6e56ebc0-openshift-default-76d6dff56d-qwbdn"
Sep 18 20:19:01.695694 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:19:01.695668    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-6e56ebc0-openshift-default-76d6dff56d-qwbdn"]
Sep 18 20:19:01.698399 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:19:01.698367    2532 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podda42bbbc_b3fb_48dd_841e_e193c95ba582.slice/crio-560e267ea96d83698fc47c78b8e39a049383052c8c4135d64195a5fbd1e641ce WatchSource:0}: Error finding container 560e267ea96d83698fc47c78b8e39a049383052c8c4135d64195a5fbd1e641ce: Status 404 returned error can't find the container with id 560e267ea96d83698fc47c78b8e39a049383052c8c4135d64195a5fbd1e641ce
Sep 18 20:19:01.700703 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:19:01.700671    2532 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 18 20:19:01.700808 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:19:01.700755    2532 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 18 20:19:01.700808 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:19:01.700799    2532 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 18 20:19:01.843289 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:19:01.843252    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:19:02.349680 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:19:02.349643    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-6e56ebc0-openshift-default-76d6dff56d-qwbdn" event={"ID":"da42bbbc-b3fb-48dd-841e-e193c95ba582","Type":"ContainerStarted","Data":"32a9dc2ec36ce9bbf1f724730f3f14fae8ffa14fdceab7bbfc283634f3b1f80a"}
Sep 18 20:19:02.350068 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:19:02.349702    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-6e56ebc0-openshift-default-76d6dff56d-qwbdn" event={"ID":"da42bbbc-b3fb-48dd-841e-e193c95ba582","Type":"ContainerStarted","Data":"560e267ea96d83698fc47c78b8e39a049383052c8c4135d64195a5fbd1e641ce"}
Sep 18 20:19:02.374895 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:19:02.374846    2532 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-derive-6e56ebc0-openshift-default-76d6dff56d-qwbdn" podStartSLOduration=1.374831615 podStartE2EDuration="1.374831615s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 20:19:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 20:19:02.37357859 +0000 UTC m=+1074.138967995" watchObservedRunningTime="2026-09-18 20:19:02.374831615 +0000 UTC m=+1074.140221022"
Sep 18 20:19:02.421251 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:19:02.421217    2532 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-6e56ebc0-openshift-default-76d6dff56d-qwbdn"
Sep 18 20:19:02.426815 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:19:02.426790    2532 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-6e56ebc0-openshift-default-76d6dff56d-qwbdn"
Sep 18 20:19:02.515324 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:19:02.515291    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:19:03.344806 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:19:03.344757    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:19:03.355932 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:19:03.355901    2532 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-6e56ebc0-openshift-default-76d6dff56d-qwbdn"
Sep 18 20:19:03.356989 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:19:03.356965    2532 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-6e56ebc0-openshift-default-76d6dff56d-qwbdn"
Sep 18 20:19:03.959573 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:19:03.959544    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:19:04.051663 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:19:04.051628    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:19:04.149504 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:19:04.149449    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:19:05.000439 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:19:05.000402    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:19:05.822980 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:19:05.822946    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:19:06.398344 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:19:06.398308    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-vptdl"]
Sep 18 20:19:06.540775 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:19:06.540737    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:19:07.233939 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:19:07.233908    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:19:07.335305 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:19:07.335269    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:19:07.570176 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:19:07.570097    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:19:07.687789 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:19:07.687757    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:19:07.785740 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:19:07.785712    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:19:16.200211 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:19:16.200180    2532 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w3-b88002/premium-w3-b88002-kserve-688cff555c-tmdhz"]
Sep 18 20:19:16.210224 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:19:16.210191    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-b88002/premium-w3-b88002-kserve-688cff555c-tmdhz"
Sep 18 20:19:16.210694 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:19:16.210672    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w3-b88002/premium-w3-b88002-kserve-688cff555c-tmdhz"]
Sep 18 20:19:16.212943 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:19:16.212922    2532 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w3-b88002\"/\"premium-w3-b88002-kserve-self-signed-certs\""
Sep 18 20:19:16.248834 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:19:16.248801    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6dcac630-bd97-4a83-83d4-4ef5f299f9d1-dshm\") pod \"premium-w3-b88002-kserve-688cff555c-tmdhz\" (UID: \"6dcac630-bd97-4a83-83d4-4ef5f299f9d1\") " pod="e2e-models-e2e-worker-w3-b88002/premium-w3-b88002-kserve-688cff555c-tmdhz"
Sep 18 20:19:16.248997 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:19:16.248847    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6dcac630-bd97-4a83-83d4-4ef5f299f9d1-tls-certs\") pod \"premium-w3-b88002-kserve-688cff555c-tmdhz\" (UID: \"6dcac630-bd97-4a83-83d4-4ef5f299f9d1\") " pod="e2e-models-e2e-worker-w3-b88002/premium-w3-b88002-kserve-688cff555c-tmdhz"
Sep 18 20:19:16.248997 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:19:16.248916    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j29cp\" (UniqueName: \"kubernetes.io/projected/6dcac630-bd97-4a83-83d4-4ef5f299f9d1-kube-api-access-j29cp\") pod \"premium-w3-b88002-kserve-688cff555c-tmdhz\" (UID: \"6dcac630-bd97-4a83-83d4-4ef5f299f9d1\") " pod="e2e-models-e2e-worker-w3-b88002/premium-w3-b88002-kserve-688cff555c-tmdhz"
Sep 18 20:19:16.249068 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:19:16.248995    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6dcac630-bd97-4a83-83d4-4ef5f299f9d1-home\") pod \"premium-w3-b88002-kserve-688cff555c-tmdhz\" (UID: \"6dcac630-bd97-4a83-83d4-4ef5f299f9d1\") " pod="e2e-models-e2e-worker-w3-b88002/premium-w3-b88002-kserve-688cff555c-tmdhz"
Sep 18 20:19:16.249068 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:19:16.249018    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6dcac630-bd97-4a83-83d4-4ef5f299f9d1-tmp-dir\") pod \"premium-w3-b88002-kserve-688cff555c-tmdhz\" (UID: \"6dcac630-bd97-4a83-83d4-4ef5f299f9d1\") " pod="e2e-models-e2e-worker-w3-b88002/premium-w3-b88002-kserve-688cff555c-tmdhz"
Sep 18 20:19:16.249068 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:19:16.249039    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6dcac630-bd97-4a83-83d4-4ef5f299f9d1-model-cache\") pod \"premium-w3-b88002-kserve-688cff555c-tmdhz\" (UID: \"6dcac630-bd97-4a83-83d4-4ef5f299f9d1\") " pod="e2e-models-e2e-worker-w3-b88002/premium-w3-b88002-kserve-688cff555c-tmdhz"
Sep 18 20:19:16.300321 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:19:16.300290    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:19:16.349723 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:19:16.349687    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6dcac630-bd97-4a83-83d4-4ef5f299f9d1-home\") pod \"premium-w3-b88002-kserve-688cff555c-tmdhz\" (UID: \"6dcac630-bd97-4a83-83d4-4ef5f299f9d1\") " pod="e2e-models-e2e-worker-w3-b88002/premium-w3-b88002-kserve-688cff555c-tmdhz"
Sep 18 20:19:16.349723 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:19:16.349720    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6dcac630-bd97-4a83-83d4-4ef5f299f9d1-tmp-dir\") pod \"premium-w3-b88002-kserve-688cff555c-tmdhz\" (UID: \"6dcac630-bd97-4a83-83d4-4ef5f299f9d1\") " pod="e2e-models-e2e-worker-w3-b88002/premium-w3-b88002-kserve-688cff555c-tmdhz"
Sep 18 20:19:16.349942 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:19:16.349751    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6dcac630-bd97-4a83-83d4-4ef5f299f9d1-model-cache\") pod \"premium-w3-b88002-kserve-688cff555c-tmdhz\" (UID: \"6dcac630-bd97-4a83-83d4-4ef5f299f9d1\") " pod="e2e-models-e2e-worker-w3-b88002/premium-w3-b88002-kserve-688cff555c-tmdhz"
Sep 18 20:19:16.349942 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:19:16.349812    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6dcac630-bd97-4a83-83d4-4ef5f299f9d1-dshm\") pod \"premium-w3-b88002-kserve-688cff555c-tmdhz\" (UID: \"6dcac630-bd97-4a83-83d4-4ef5f299f9d1\") " pod="e2e-models-e2e-worker-w3-b88002/premium-w3-b88002-kserve-688cff555c-tmdhz"
Sep 18 20:19:16.349942 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:19:16.349839    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6dcac630-bd97-4a83-83d4-4ef5f299f9d1-tls-certs\") pod \"premium-w3-b88002-kserve-688cff555c-tmdhz\" (UID: \"6dcac630-bd97-4a83-83d4-4ef5f299f9d1\") " pod="e2e-models-e2e-worker-w3-b88002/premium-w3-b88002-kserve-688cff555c-tmdhz"
Sep 18 20:19:16.349942 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:19:16.349869    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j29cp\" (UniqueName: \"kubernetes.io/projected/6dcac630-bd97-4a83-83d4-4ef5f299f9d1-kube-api-access-j29cp\") pod \"premium-w3-b88002-kserve-688cff555c-tmdhz\" (UID: \"6dcac630-bd97-4a83-83d4-4ef5f299f9d1\") " pod="e2e-models-e2e-worker-w3-b88002/premium-w3-b88002-kserve-688cff555c-tmdhz"
Sep 18 20:19:16.350193 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:19:16.350161    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6dcac630-bd97-4a83-83d4-4ef5f299f9d1-tmp-dir\") pod \"premium-w3-b88002-kserve-688cff555c-tmdhz\" (UID: \"6dcac630-bd97-4a83-83d4-4ef5f299f9d1\") " pod="e2e-models-e2e-worker-w3-b88002/premium-w3-b88002-kserve-688cff555c-tmdhz"
Sep 18 20:19:16.350328 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:19:16.350205    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6dcac630-bd97-4a83-83d4-4ef5f299f9d1-model-cache\") pod \"premium-w3-b88002-kserve-688cff555c-tmdhz\" (UID: \"6dcac630-bd97-4a83-83d4-4ef5f299f9d1\") " pod="e2e-models-e2e-worker-w3-b88002/premium-w3-b88002-kserve-688cff555c-tmdhz"
Sep 18 20:19:16.350328 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:19:16.350282    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6dcac630-bd97-4a83-83d4-4ef5f299f9d1-home\") pod \"premium-w3-b88002-kserve-688cff555c-tmdhz\" (UID: \"6dcac630-bd97-4a83-83d4-4ef5f299f9d1\") " pod="e2e-models-e2e-worker-w3-b88002/premium-w3-b88002-kserve-688cff555c-tmdhz"
Sep 18 20:19:16.352094 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:19:16.352074    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6dcac630-bd97-4a83-83d4-4ef5f299f9d1-dshm\") pod \"premium-w3-b88002-kserve-688cff555c-tmdhz\" (UID: \"6dcac630-bd97-4a83-83d4-4ef5f299f9d1\") " pod="e2e-models-e2e-worker-w3-b88002/premium-w3-b88002-kserve-688cff555c-tmdhz"
Sep 18 20:19:16.352382 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:19:16.352361    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6dcac630-bd97-4a83-83d4-4ef5f299f9d1-tls-certs\") pod \"premium-w3-b88002-kserve-688cff555c-tmdhz\" (UID: \"6dcac630-bd97-4a83-83d4-4ef5f299f9d1\") " pod="e2e-models-e2e-worker-w3-b88002/premium-w3-b88002-kserve-688cff555c-tmdhz"
Sep 18 20:19:16.357656 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:19:16.357625    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j29cp\" (UniqueName: \"kubernetes.io/projected/6dcac630-bd97-4a83-83d4-4ef5f299f9d1-kube-api-access-j29cp\") pod \"premium-w3-b88002-kserve-688cff555c-tmdhz\" (UID: \"6dcac630-bd97-4a83-83d4-4ef5f299f9d1\") " pod="e2e-models-e2e-worker-w3-b88002/premium-w3-b88002-kserve-688cff555c-tmdhz"
Sep 18 20:19:16.520324 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:19:16.520287    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-b88002/premium-w3-b88002-kserve-688cff555c-tmdhz"
Sep 18 20:19:16.790621 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:19:16.790593    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w3-b88002/premium-w3-b88002-kserve-688cff555c-tmdhz"]
Sep 18 20:19:16.791559 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:19:16.791530    2532 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6dcac630_bd97_4a83_83d4_4ef5f299f9d1.slice/crio-5905ed8af01ed516ed3d2d3213e511a5b9987f376d65acd020c067d1b096385b WatchSource:0}: Error finding container 5905ed8af01ed516ed3d2d3213e511a5b9987f376d65acd020c067d1b096385b: Status 404 returned error can't find the container with id 5905ed8af01ed516ed3d2d3213e511a5b9987f376d65acd020c067d1b096385b
Sep 18 20:19:17.435335 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:19:17.435299    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-b88002/premium-w3-b88002-kserve-688cff555c-tmdhz" event={"ID":"6dcac630-bd97-4a83-83d4-4ef5f299f9d1","Type":"ContainerStarted","Data":"c2ed48bccfdd55afd9a2a7f50086a63608f9af700c0803c0fc6b9303cf290b0f"}
Sep 18 20:19:17.435335 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:19:17.435343    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-b88002/premium-w3-b88002-kserve-688cff555c-tmdhz" event={"ID":"6dcac630-bd97-4a83-83d4-4ef5f299f9d1","Type":"ContainerStarted","Data":"5905ed8af01ed516ed3d2d3213e511a5b9987f376d65acd020c067d1b096385b"}
Sep 18 20:19:17.461865 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:19:17.461819    2532 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-models-e2e-worker-w3-b88002/premium-w3-b88002-kserve-688cff555c-tmdhz" podStartSLOduration=1.461802136 podStartE2EDuration="1.461802136s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 20:19:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 20:19:17.460224896 +0000 UTC m=+1089.225614296" watchObservedRunningTime="2026-09-18 20:19:17.461802136 +0000 UTC m=+1089.227191534"
Sep 18 20:19:23.096973 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:19:23.096925    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-vptdl"]
Sep 18 20:19:26.521517 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:19:26.521461    2532 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w3-b88002/premium-w3-b88002-kserve-688cff555c-tmdhz"
Sep 18 20:19:26.522006 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:19:26.521549    2532 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-b88002/premium-w3-b88002-kserve-688cff555c-tmdhz"
Sep 18 20:19:26.533803 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:19:26.533778    2532 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w3-b88002/premium-w3-b88002-kserve-688cff555c-tmdhz"
Sep 18 20:19:27.262663 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:19:27.262630    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:19:27.505803 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:19:27.505772    2532 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-b88002/premium-w3-b88002-kserve-688cff555c-tmdhz"
Sep 18 20:19:29.177427 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:19:29.177394    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:19:29.896958 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:19:29.896903    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:19:30.711886 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:19:30.711750    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:19:31.337427 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:19:31.337401    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:19:31.946148 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:19:31.946112    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:19:32.702711 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:19:32.702656    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:19:34.605306 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:19:34.605269    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:19:35.247528 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:19:35.247479    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:19:35.943288 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:19:35.943257    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:19:36.729045 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:19:36.729006    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:19:37.328438 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:19:37.328387    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:19:37.565647 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:19:37.565566    2532 generic.go:356] "Generic (PLEG): container finished" podID="da42bbbc-b3fb-48dd-841e-e193c95ba582" containerID="32a9dc2ec36ce9bbf1f724730f3f14fae8ffa14fdceab7bbfc283634f3b1f80a" exitCode=0
Sep 18 20:19:37.565647 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:19:37.565609    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-6e56ebc0-openshift-default-76d6dff56d-qwbdn" event={"ID":"da42bbbc-b3fb-48dd-841e-e193c95ba582","Type":"ContainerDied","Data":"32a9dc2ec36ce9bbf1f724730f3f14fae8ffa14fdceab7bbfc283634f3b1f80a"}
Sep 18 20:19:37.566059 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:19:37.566045    2532 scope.go:117] "RemoveContainer" containerID="32a9dc2ec36ce9bbf1f724730f3f14fae8ffa14fdceab7bbfc283634f3b1f80a"
Sep 18 20:19:37.567392 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:19:37.567360    2532 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 18 20:19:37.567541 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:19:37.567431    2532 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 18 20:19:37.567541 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:19:37.567468    2532 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 18 20:19:37.958084 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:19:37.958049    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:19:38.501789 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:19:38.501755    2532 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w0-116778/premium-w0-116778-kserve-9b5787766-tth7r"]
Sep 18 20:19:38.512803 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:19:38.512769    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-116778/premium-w0-116778-kserve-9b5787766-tth7r"
Sep 18 20:19:38.517339 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:19:38.517275    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-116778/premium-w0-116778-kserve-9b5787766-tth7r"]
Sep 18 20:19:38.517613 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:19:38.517594    2532 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-116778\"/\"premium-w0-116778-kserve-self-signed-certs\""
Sep 18 20:19:38.531257 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:19:38.531230    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:19:38.572769 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:19:38.572737    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-6e56ebc0-openshift-default-76d6dff56d-qwbdn" event={"ID":"da42bbbc-b3fb-48dd-841e-e193c95ba582","Type":"ContainerStarted","Data":"ad1761e726cd0490dcae7ea5f660194e49082a5e09046b980629e00f1491ba68"}
Sep 18 20:19:38.589747 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:19:38.589713    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/53c9b26c-7ae0-418a-9296-08fc8ad7b84e-home\") pod \"premium-w0-116778-kserve-9b5787766-tth7r\" (UID: \"53c9b26c-7ae0-418a-9296-08fc8ad7b84e\") " pod="e2e-models-e2e-worker-w0-116778/premium-w0-116778-kserve-9b5787766-tth7r"
Sep 18 20:19:38.589909 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:19:38.589755    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/53c9b26c-7ae0-418a-9296-08fc8ad7b84e-tls-certs\") pod \"premium-w0-116778-kserve-9b5787766-tth7r\" (UID: \"53c9b26c-7ae0-418a-9296-08fc8ad7b84e\") " pod="e2e-models-e2e-worker-w0-116778/premium-w0-116778-kserve-9b5787766-tth7r"
Sep 18 20:19:38.589909 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:19:38.589779    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/53c9b26c-7ae0-418a-9296-08fc8ad7b84e-tmp-dir\") pod \"premium-w0-116778-kserve-9b5787766-tth7r\" (UID: \"53c9b26c-7ae0-418a-9296-08fc8ad7b84e\") " pod="e2e-models-e2e-worker-w0-116778/premium-w0-116778-kserve-9b5787766-tth7r"
Sep 18 20:19:38.589909 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:19:38.589874    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vz4fx\" (UniqueName: \"kubernetes.io/projected/53c9b26c-7ae0-418a-9296-08fc8ad7b84e-kube-api-access-vz4fx\") pod \"premium-w0-116778-kserve-9b5787766-tth7r\" (UID: \"53c9b26c-7ae0-418a-9296-08fc8ad7b84e\") " pod="e2e-models-e2e-worker-w0-116778/premium-w0-116778-kserve-9b5787766-tth7r"
Sep 18 20:19:38.590017 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:19:38.589912    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/53c9b26c-7ae0-418a-9296-08fc8ad7b84e-dshm\") pod \"premium-w0-116778-kserve-9b5787766-tth7r\" (UID: \"53c9b26c-7ae0-418a-9296-08fc8ad7b84e\") " pod="e2e-models-e2e-worker-w0-116778/premium-w0-116778-kserve-9b5787766-tth7r"
Sep 18 20:19:38.590017 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:19:38.589979    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/53c9b26c-7ae0-418a-9296-08fc8ad7b84e-model-cache\") pod \"premium-w0-116778-kserve-9b5787766-tth7r\" (UID: \"53c9b26c-7ae0-418a-9296-08fc8ad7b84e\") " pod="e2e-models-e2e-worker-w0-116778/premium-w0-116778-kserve-9b5787766-tth7r"
Sep 18 20:19:38.691529 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:19:38.691468    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/53c9b26c-7ae0-418a-9296-08fc8ad7b84e-home\") pod \"premium-w0-116778-kserve-9b5787766-tth7r\" (UID: \"53c9b26c-7ae0-418a-9296-08fc8ad7b84e\") " pod="e2e-models-e2e-worker-w0-116778/premium-w0-116778-kserve-9b5787766-tth7r"
Sep 18 20:19:38.691529 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:19:38.691532    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/53c9b26c-7ae0-418a-9296-08fc8ad7b84e-tls-certs\") pod \"premium-w0-116778-kserve-9b5787766-tth7r\" (UID: \"53c9b26c-7ae0-418a-9296-08fc8ad7b84e\") " pod="e2e-models-e2e-worker-w0-116778/premium-w0-116778-kserve-9b5787766-tth7r"
Sep 18 20:19:38.691741 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:19:38.691556    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/53c9b26c-7ae0-418a-9296-08fc8ad7b84e-tmp-dir\") pod \"premium-w0-116778-kserve-9b5787766-tth7r\" (UID: \"53c9b26c-7ae0-418a-9296-08fc8ad7b84e\") " pod="e2e-models-e2e-worker-w0-116778/premium-w0-116778-kserve-9b5787766-tth7r"
Sep 18 20:19:38.691741 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:19:38.691615    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vz4fx\" (UniqueName: \"kubernetes.io/projected/53c9b26c-7ae0-418a-9296-08fc8ad7b84e-kube-api-access-vz4fx\") pod \"premium-w0-116778-kserve-9b5787766-tth7r\" (UID: \"53c9b26c-7ae0-418a-9296-08fc8ad7b84e\") " pod="e2e-models-e2e-worker-w0-116778/premium-w0-116778-kserve-9b5787766-tth7r"
Sep 18 20:19:38.691741 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:19:38.691671    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/53c9b26c-7ae0-418a-9296-08fc8ad7b84e-dshm\") pod \"premium-w0-116778-kserve-9b5787766-tth7r\" (UID: \"53c9b26c-7ae0-418a-9296-08fc8ad7b84e\") " pod="e2e-models-e2e-worker-w0-116778/premium-w0-116778-kserve-9b5787766-tth7r"
Sep 18 20:19:38.691741 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:19:38.691734    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/53c9b26c-7ae0-418a-9296-08fc8ad7b84e-model-cache\") pod \"premium-w0-116778-kserve-9b5787766-tth7r\" (UID: \"53c9b26c-7ae0-418a-9296-08fc8ad7b84e\") " pod="e2e-models-e2e-worker-w0-116778/premium-w0-116778-kserve-9b5787766-tth7r"
Sep 18 20:19:38.692031 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:19:38.692008    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/53c9b26c-7ae0-418a-9296-08fc8ad7b84e-home\") pod \"premium-w0-116778-kserve-9b5787766-tth7r\" (UID: \"53c9b26c-7ae0-418a-9296-08fc8ad7b84e\") " pod="e2e-models-e2e-worker-w0-116778/premium-w0-116778-kserve-9b5787766-tth7r"
Sep 18 20:19:38.692102 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:19:38.692059    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/53c9b26c-7ae0-418a-9296-08fc8ad7b84e-tmp-dir\") pod \"premium-w0-116778-kserve-9b5787766-tth7r\" (UID: \"53c9b26c-7ae0-418a-9296-08fc8ad7b84e\") " pod="e2e-models-e2e-worker-w0-116778/premium-w0-116778-kserve-9b5787766-tth7r"
Sep 18 20:19:38.692102 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:19:38.692086    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/53c9b26c-7ae0-418a-9296-08fc8ad7b84e-model-cache\") pod \"premium-w0-116778-kserve-9b5787766-tth7r\" (UID: \"53c9b26c-7ae0-418a-9296-08fc8ad7b84e\") " pod="e2e-models-e2e-worker-w0-116778/premium-w0-116778-kserve-9b5787766-tth7r"
Sep 18 20:19:38.694012 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:19:38.693986    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/53c9b26c-7ae0-418a-9296-08fc8ad7b84e-dshm\") pod \"premium-w0-116778-kserve-9b5787766-tth7r\" (UID: \"53c9b26c-7ae0-418a-9296-08fc8ad7b84e\") " pod="e2e-models-e2e-worker-w0-116778/premium-w0-116778-kserve-9b5787766-tth7r"
Sep 18 20:19:38.694197 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:19:38.694179    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/53c9b26c-7ae0-418a-9296-08fc8ad7b84e-tls-certs\") pod \"premium-w0-116778-kserve-9b5787766-tth7r\" (UID: \"53c9b26c-7ae0-418a-9296-08fc8ad7b84e\") " pod="e2e-models-e2e-worker-w0-116778/premium-w0-116778-kserve-9b5787766-tth7r"
Sep 18 20:19:38.701470 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:19:38.701440    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vz4fx\" (UniqueName: \"kubernetes.io/projected/53c9b26c-7ae0-418a-9296-08fc8ad7b84e-kube-api-access-vz4fx\") pod \"premium-w0-116778-kserve-9b5787766-tth7r\" (UID: \"53c9b26c-7ae0-418a-9296-08fc8ad7b84e\") " pod="e2e-models-e2e-worker-w0-116778/premium-w0-116778-kserve-9b5787766-tth7r"
Sep 18 20:19:38.836572 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:19:38.836452    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-116778/premium-w0-116778-kserve-9b5787766-tth7r"
Sep 18 20:19:39.109451 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:19:39.109359    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-116778/premium-w0-116778-kserve-9b5787766-tth7r"]
Sep 18 20:19:39.134066 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:19:39.134032    2532 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod53c9b26c_7ae0_418a_9296_08fc8ad7b84e.slice/crio-e516d8e81f78c5dff4d6f26bd9fa3dae5d8c628f4071de983122155fac9c2f8d WatchSource:0}: Error finding container e516d8e81f78c5dff4d6f26bd9fa3dae5d8c628f4071de983122155fac9c2f8d: Status 404 returned error can't find the container with id e516d8e81f78c5dff4d6f26bd9fa3dae5d8c628f4071de983122155fac9c2f8d
Sep 18 20:19:39.424048 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:19:39.423940    2532 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-6e56ebc0-openshift-default-76d6dff56d-qwbdn"
Sep 18 20:19:39.424225 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:19:39.424080    2532 patch_prober.go:28] interesting pod/e2e-derive-6e56ebc0-openshift-default-76d6dff56d-qwbdn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" start-of-body=
Sep 18 20:19:39.424225 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:19:39.424148    2532 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-6e56ebc0-openshift-default-76d6dff56d-qwbdn" podUID="da42bbbc-b3fb-48dd-841e-e193c95ba582" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused"
Sep 18 20:19:39.581276 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:19:39.581235    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-116778/premium-w0-116778-kserve-9b5787766-tth7r" event={"ID":"53c9b26c-7ae0-418a-9296-08fc8ad7b84e","Type":"ContainerStarted","Data":"3070bd4e0384dae04a9a7b9df2da95f3c98c15597f4482ea27e79f795dd8d2bc"}
Sep 18 20:19:39.581276 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:19:39.581282    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-116778/premium-w0-116778-kserve-9b5787766-tth7r" event={"ID":"53c9b26c-7ae0-418a-9296-08fc8ad7b84e","Type":"ContainerStarted","Data":"e516d8e81f78c5dff4d6f26bd9fa3dae5d8c628f4071de983122155fac9c2f8d"}
Sep 18 20:19:39.599416 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:19:39.599367    2532 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-models-e2e-worker-w0-116778/premium-w0-116778-kserve-9b5787766-tth7r" podStartSLOduration=1.599346471 podStartE2EDuration="1.599346471s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 20:19:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 20:19:39.59886295 +0000 UTC m=+1111.364252362" watchObservedRunningTime="2026-09-18 20:19:39.599346471 +0000 UTC m=+1111.364735873"
Sep 18 20:19:40.420865 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:19:40.420822    2532 patch_prober.go:28] interesting pod/e2e-derive-6e56ebc0-openshift-default-76d6dff56d-qwbdn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" start-of-body=
Sep 18 20:19:40.421156 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:19:40.421125    2532 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-6e56ebc0-openshift-default-76d6dff56d-qwbdn" podUID="da42bbbc-b3fb-48dd-841e-e193c95ba582" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused"
Sep 18 20:19:41.420850 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:19:41.420813    2532 patch_prober.go:28] interesting pod/e2e-derive-6e56ebc0-openshift-default-76d6dff56d-qwbdn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" start-of-body=
Sep 18 20:19:41.421442 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:19:41.421411    2532 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-6e56ebc0-openshift-default-76d6dff56d-qwbdn" podUID="da42bbbc-b3fb-48dd-841e-e193c95ba582" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused"
Sep 18 20:19:42.421601 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:19:42.421564    2532 patch_prober.go:28] interesting pod/e2e-derive-6e56ebc0-openshift-default-76d6dff56d-qwbdn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" start-of-body=
Sep 18 20:19:42.422237 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:19:42.422202    2532 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-6e56ebc0-openshift-default-76d6dff56d-qwbdn" podUID="da42bbbc-b3fb-48dd-841e-e193c95ba582" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused"
Sep 18 20:19:43.421368 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:19:43.421322    2532 patch_prober.go:28] interesting pod/e2e-derive-6e56ebc0-openshift-default-76d6dff56d-qwbdn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" start-of-body=
Sep 18 20:19:43.421611 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:19:43.421415    2532 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-6e56ebc0-openshift-default-76d6dff56d-qwbdn" podUID="da42bbbc-b3fb-48dd-841e-e193c95ba582" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused"
Sep 18 20:19:44.425119 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:19:44.425089    2532 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-6e56ebc0-openshift-default-76d6dff56d-qwbdn"
Sep 18 20:19:44.425614 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:19:44.425514    2532 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-6e56ebc0-openshift-default-76d6dff56d-qwbdn"
Sep 18 20:19:44.426100 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:19:44.426079    2532 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-6e56ebc0-openshift-default-76d6dff56d-qwbdn"
Sep 18 20:19:44.701805 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:19:44.701721    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-vptdl"]
Sep 18 20:19:45.779709 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:19:45.779672    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:19:45.843922 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:19:45.843890    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:19:45.903051 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:19:45.903019    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:19:45.991008 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:19:45.990982    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:19:46.052852 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:19:46.052768    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:19:46.138129 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:19:46.138097    2532 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-6e56ebc0-openshift-default-76d6dff56d-qwbdn"]
Sep 18 20:19:46.146591 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:19:46.146562    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:19:46.421503 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:19:46.421406    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:19:46.624568 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:19:46.624529    2532 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-derive-6e56ebc0-openshift-default-76d6dff56d-qwbdn" podUID="da42bbbc-b3fb-48dd-841e-e193c95ba582" containerName="istio-proxy" containerID="cri-o://ad1761e726cd0490dcae7ea5f660194e49082a5e09046b980629e00f1491ba68" gracePeriod=30
Sep 18 20:19:46.806133 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:19:46.806100    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:19:46.904817 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:19:46.904784    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:19:47.015630 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:19:47.015599    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:19:47.311048 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:19:47.311019    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:19:48.012315 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:19:48.012278    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:19:48.399802 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:19:48.399715    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:19:48.781454 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:19:48.781420    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:19:48.836572 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:19:48.836542    2532 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w0-116778/premium-w0-116778-kserve-9b5787766-tth7r"
Sep 18 20:19:48.836752 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:19:48.836595    2532 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-116778/premium-w0-116778-kserve-9b5787766-tth7r"
Sep 18 20:19:48.849345 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:19:48.849322    2532 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w0-116778/premium-w0-116778-kserve-9b5787766-tth7r"
Sep 18 20:19:49.489928 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:19:49.489895    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:19:49.652496 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:19:49.652468    2532 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-116778/premium-w0-116778-kserve-9b5787766-tth7r"
Sep 18 20:19:51.870368 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:19:51.870343    2532 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-6e56ebc0-openshift-default-76d6dff56d-qwbdn"
Sep 18 20:19:51.932529 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:19:51.932376    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/da42bbbc-b3fb-48dd-841e-e193c95ba582-istio-envoy\") pod \"da42bbbc-b3fb-48dd-841e-e193c95ba582\" (UID: \"da42bbbc-b3fb-48dd-841e-e193c95ba582\") "
Sep 18 20:19:51.932529 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:19:51.932439    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/da42bbbc-b3fb-48dd-841e-e193c95ba582-istio-token\") pod \"da42bbbc-b3fb-48dd-841e-e193c95ba582\" (UID: \"da42bbbc-b3fb-48dd-841e-e193c95ba582\") "
Sep 18 20:19:51.932529 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:19:51.932475    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/da42bbbc-b3fb-48dd-841e-e193c95ba582-workload-certs\") pod \"da42bbbc-b3fb-48dd-841e-e193c95ba582\" (UID: \"da42bbbc-b3fb-48dd-841e-e193c95ba582\") "
Sep 18 20:19:51.932529 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:19:51.932528    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8j2jp\" (UniqueName: \"kubernetes.io/projected/da42bbbc-b3fb-48dd-841e-e193c95ba582-kube-api-access-8j2jp\") pod \"da42bbbc-b3fb-48dd-841e-e193c95ba582\" (UID: \"da42bbbc-b3fb-48dd-841e-e193c95ba582\") "
Sep 18 20:19:51.932858 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:19:51.932570    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/da42bbbc-b3fb-48dd-841e-e193c95ba582-credential-socket\") pod \"da42bbbc-b3fb-48dd-841e-e193c95ba582\" (UID: \"da42bbbc-b3fb-48dd-841e-e193c95ba582\") "
Sep 18 20:19:51.932858 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:19:51.932597    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/da42bbbc-b3fb-48dd-841e-e193c95ba582-istiod-ca-cert\") pod \"da42bbbc-b3fb-48dd-841e-e193c95ba582\" (UID: \"da42bbbc-b3fb-48dd-841e-e193c95ba582\") "
Sep 18 20:19:51.932858 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:19:51.932631    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/da42bbbc-b3fb-48dd-841e-e193c95ba582-istio-podinfo\") pod \"da42bbbc-b3fb-48dd-841e-e193c95ba582\" (UID: \"da42bbbc-b3fb-48dd-841e-e193c95ba582\") "
Sep 18 20:19:51.932858 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:19:51.932696    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/da42bbbc-b3fb-48dd-841e-e193c95ba582-istio-data\") pod \"da42bbbc-b3fb-48dd-841e-e193c95ba582\" (UID: \"da42bbbc-b3fb-48dd-841e-e193c95ba582\") "
Sep 18 20:19:51.932858 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:19:51.932720    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/da42bbbc-b3fb-48dd-841e-e193c95ba582-workload-socket\") pod \"da42bbbc-b3fb-48dd-841e-e193c95ba582\" (UID: \"da42bbbc-b3fb-48dd-841e-e193c95ba582\") "
Sep 18 20:19:51.933101 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:19:51.933037    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/da42bbbc-b3fb-48dd-841e-e193c95ba582-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "da42bbbc-b3fb-48dd-841e-e193c95ba582" (UID: "da42bbbc-b3fb-48dd-841e-e193c95ba582"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 20:19:51.933279 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:19:51.933259    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/da42bbbc-b3fb-48dd-841e-e193c95ba582-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "da42bbbc-b3fb-48dd-841e-e193c95ba582" (UID: "da42bbbc-b3fb-48dd-841e-e193c95ba582"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 20:19:51.933428 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:19:51.933287    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/da42bbbc-b3fb-48dd-841e-e193c95ba582-istio-data" (OuterVolumeSpecName: "istio-data") pod "da42bbbc-b3fb-48dd-841e-e193c95ba582" (UID: "da42bbbc-b3fb-48dd-841e-e193c95ba582"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 20:19:51.933428 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:19:51.933296    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/da42bbbc-b3fb-48dd-841e-e193c95ba582-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "da42bbbc-b3fb-48dd-841e-e193c95ba582" (UID: "da42bbbc-b3fb-48dd-841e-e193c95ba582"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 20:19:51.933428 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:19:51.933369    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/da42bbbc-b3fb-48dd-841e-e193c95ba582-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "da42bbbc-b3fb-48dd-841e-e193c95ba582" (UID: "da42bbbc-b3fb-48dd-841e-e193c95ba582"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 20:19:51.935118 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:19:51.935094    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/da42bbbc-b3fb-48dd-841e-e193c95ba582-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "da42bbbc-b3fb-48dd-841e-e193c95ba582" (UID: "da42bbbc-b3fb-48dd-841e-e193c95ba582"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 20:19:51.935221 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:19:51.935202    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/da42bbbc-b3fb-48dd-841e-e193c95ba582-istio-token" (OuterVolumeSpecName: "istio-token") pod "da42bbbc-b3fb-48dd-841e-e193c95ba582" (UID: "da42bbbc-b3fb-48dd-841e-e193c95ba582"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 20:19:51.935435 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:19:51.935416    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/da42bbbc-b3fb-48dd-841e-e193c95ba582-kube-api-access-8j2jp" (OuterVolumeSpecName: "kube-api-access-8j2jp") pod "da42bbbc-b3fb-48dd-841e-e193c95ba582" (UID: "da42bbbc-b3fb-48dd-841e-e193c95ba582"). InnerVolumeSpecName "kube-api-access-8j2jp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 20:19:51.935578 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:19:51.935556    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/da42bbbc-b3fb-48dd-841e-e193c95ba582-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "da42bbbc-b3fb-48dd-841e-e193c95ba582" (UID: "da42bbbc-b3fb-48dd-841e-e193c95ba582"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 18 20:19:52.033789 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:19:52.033748    2532 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/da42bbbc-b3fb-48dd-841e-e193c95ba582-istio-envoy\") on node \"ip-10-0-130-81.ec2.internal\" DevicePath \"\""
Sep 18 20:19:52.033789 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:19:52.033780    2532 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/da42bbbc-b3fb-48dd-841e-e193c95ba582-istio-token\") on node \"ip-10-0-130-81.ec2.internal\" DevicePath \"\""
Sep 18 20:19:52.033789 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:19:52.033792    2532 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/da42bbbc-b3fb-48dd-841e-e193c95ba582-workload-certs\") on node \"ip-10-0-130-81.ec2.internal\" DevicePath \"\""
Sep 18 20:19:52.034046 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:19:52.033806    2532 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8j2jp\" (UniqueName: \"kubernetes.io/projected/da42bbbc-b3fb-48dd-841e-e193c95ba582-kube-api-access-8j2jp\") on node \"ip-10-0-130-81.ec2.internal\" DevicePath \"\""
Sep 18 20:19:52.034046 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:19:52.033819    2532 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/da42bbbc-b3fb-48dd-841e-e193c95ba582-credential-socket\") on node \"ip-10-0-130-81.ec2.internal\" DevicePath \"\""
Sep 18 20:19:52.034046 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:19:52.033832    2532 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/da42bbbc-b3fb-48dd-841e-e193c95ba582-istiod-ca-cert\") on node \"ip-10-0-130-81.ec2.internal\" DevicePath \"\""
Sep 18 20:19:52.034046 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:19:52.033842    2532 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/da42bbbc-b3fb-48dd-841e-e193c95ba582-istio-podinfo\") on node \"ip-10-0-130-81.ec2.internal\" DevicePath \"\""
Sep 18 20:19:52.034046 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:19:52.033852    2532 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/da42bbbc-b3fb-48dd-841e-e193c95ba582-istio-data\") on node \"ip-10-0-130-81.ec2.internal\" DevicePath \"\""
Sep 18 20:19:52.034046 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:19:52.033862    2532 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/da42bbbc-b3fb-48dd-841e-e193c95ba582-workload-socket\") on node \"ip-10-0-130-81.ec2.internal\" DevicePath \"\""
Sep 18 20:19:52.657976 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:19:52.657885    2532 generic.go:356] "Generic (PLEG): container finished" podID="da42bbbc-b3fb-48dd-841e-e193c95ba582" containerID="ad1761e726cd0490dcae7ea5f660194e49082a5e09046b980629e00f1491ba68" exitCode=0
Sep 18 20:19:52.657976 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:19:52.657933    2532 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-6e56ebc0-openshift-default-76d6dff56d-qwbdn"
Sep 18 20:19:52.657976 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:19:52.657942    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-6e56ebc0-openshift-default-76d6dff56d-qwbdn" event={"ID":"da42bbbc-b3fb-48dd-841e-e193c95ba582","Type":"ContainerDied","Data":"ad1761e726cd0490dcae7ea5f660194e49082a5e09046b980629e00f1491ba68"}
Sep 18 20:19:52.658193 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:19:52.657983    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-6e56ebc0-openshift-default-76d6dff56d-qwbdn" event={"ID":"da42bbbc-b3fb-48dd-841e-e193c95ba582","Type":"ContainerDied","Data":"560e267ea96d83698fc47c78b8e39a049383052c8c4135d64195a5fbd1e641ce"}
Sep 18 20:19:52.658193 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:19:52.658005    2532 scope.go:117] "RemoveContainer" containerID="ad1761e726cd0490dcae7ea5f660194e49082a5e09046b980629e00f1491ba68"
Sep 18 20:19:52.666991 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:19:52.666973    2532 scope.go:117] "RemoveContainer" containerID="32a9dc2ec36ce9bbf1f724730f3f14fae8ffa14fdceab7bbfc283634f3b1f80a"
Sep 18 20:19:52.675200 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:19:52.675183    2532 scope.go:117] "RemoveContainer" containerID="ad1761e726cd0490dcae7ea5f660194e49082a5e09046b980629e00f1491ba68"
Sep 18 20:19:52.675460 ip-10-0-130-81 kubenswrapper[2532]: E0918 20:19:52.675441    2532 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ad1761e726cd0490dcae7ea5f660194e49082a5e09046b980629e00f1491ba68\": container with ID starting with ad1761e726cd0490dcae7ea5f660194e49082a5e09046b980629e00f1491ba68 not found: ID does not exist" containerID="ad1761e726cd0490dcae7ea5f660194e49082a5e09046b980629e00f1491ba68"
Sep 18 20:19:52.675547 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:19:52.675469    2532 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ad1761e726cd0490dcae7ea5f660194e49082a5e09046b980629e00f1491ba68"} err="failed to get container status \"ad1761e726cd0490dcae7ea5f660194e49082a5e09046b980629e00f1491ba68\": rpc error: code = NotFound desc = could not find container \"ad1761e726cd0490dcae7ea5f660194e49082a5e09046b980629e00f1491ba68\": container with ID starting with ad1761e726cd0490dcae7ea5f660194e49082a5e09046b980629e00f1491ba68 not found: ID does not exist"
Sep 18 20:19:52.675547 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:19:52.675504    2532 scope.go:117] "RemoveContainer" containerID="32a9dc2ec36ce9bbf1f724730f3f14fae8ffa14fdceab7bbfc283634f3b1f80a"
Sep 18 20:19:52.675746 ip-10-0-130-81 kubenswrapper[2532]: E0918 20:19:52.675731    2532 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"32a9dc2ec36ce9bbf1f724730f3f14fae8ffa14fdceab7bbfc283634f3b1f80a\": container with ID starting with 32a9dc2ec36ce9bbf1f724730f3f14fae8ffa14fdceab7bbfc283634f3b1f80a not found: ID does not exist" containerID="32a9dc2ec36ce9bbf1f724730f3f14fae8ffa14fdceab7bbfc283634f3b1f80a"
Sep 18 20:19:52.675786 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:19:52.675752    2532 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"32a9dc2ec36ce9bbf1f724730f3f14fae8ffa14fdceab7bbfc283634f3b1f80a"} err="failed to get container status \"32a9dc2ec36ce9bbf1f724730f3f14fae8ffa14fdceab7bbfc283634f3b1f80a\": rpc error: code = NotFound desc = could not find container \"32a9dc2ec36ce9bbf1f724730f3f14fae8ffa14fdceab7bbfc283634f3b1f80a\": container with ID starting with 32a9dc2ec36ce9bbf1f724730f3f14fae8ffa14fdceab7bbfc283634f3b1f80a not found: ID does not exist"
Sep 18 20:19:52.675976 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:19:52.675958    2532 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-6e56ebc0-openshift-default-76d6dff56d-qwbdn"]
Sep 18 20:19:52.679652 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:19:52.679627    2532 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-derive-6e56ebc0-openshift-default-76d6dff56d-qwbdn"]
Sep 18 20:19:54.306543 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:19:54.306514    2532 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="da42bbbc-b3fb-48dd-841e-e193c95ba582" path="/var/lib/kubelet/pods/da42bbbc-b3fb-48dd-841e-e193c95ba582/volumes"
Sep 18 20:19:54.387745 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:19:54.387712    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:19:54.447707 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:19:54.447677    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:19:54.545755 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:19:54.545716    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:19:54.645369 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:19:54.645279    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:19:54.745803 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:19:54.745761    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:19:55.077648 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:19:55.077620    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:19:59.929220 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:19:59.929170    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:20:03.196581 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:20:03.196540    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-vptdl"]
Sep 18 20:20:20.295225 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:20:20.295188    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-vptdl"]
Sep 18 20:20:20.968796 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:20:20.968753    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:20:26.758899 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:20:26.758818    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:20:26.866392 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:20:26.866358    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:20:26.959060 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:20:26.959034    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:20:28.115940 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:20:28.115906    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:20:39.800350 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:20:39.800313    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-vptdl"]
Sep 18 20:20:42.752298 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:20:42.752264    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:20:48.822005 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:20:48.821954    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:20:48.892628 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:20:48.892595    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:20:48.982363 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:20:48.982288    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:20:49.082754 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:20:49.082670    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:20:59.503572 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:20:59.503536    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:21:00.285553 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:21:00.285518    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:21:05.716520 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:21:05.716449    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:21:06.250441 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:21:06.250410    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:21:06.350348 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:21:06.350317    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:21:06.449330 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:21:06.449294    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:21:06.551369 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:21:06.551288    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:21:07.147476 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:21:07.147446    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:21:07.273481 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:21:07.273445    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:21:08.769157 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:21:08.769112    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:21:10.356585 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:21:10.356556    2532 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-m7wdd_32e33d39-7826-499d-bc01-0487b7580e7a/console-operator/1.log"
Sep 18 20:21:10.357024 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:21:10.356769    2532 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-m7wdd_32e33d39-7826-499d-bc01-0487b7580e7a/console-operator/1.log"
Sep 18 20:21:10.365101 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:21:10.365080    2532 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rtw9n_73fbf16b-3549-4a6c-a959-b633237ebd0c/ovn-acl-logging/0.log"
Sep 18 20:21:10.365245 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:21:10.365188    2532 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rtw9n_73fbf16b-3549-4a6c-a959-b633237ebd0c/ovn-acl-logging/0.log"
Sep 18 20:21:12.403223 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:21:12.403188    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-vptdl"]
Sep 18 20:21:13.837684 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:21:13.837653    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:21:14.860171 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:21:14.860139    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:21:16.152235 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:21:16.152198    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:21:21.356774 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:21:21.356741    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:21:21.419059 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:21:21.419027    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:21:21.587258 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:21:21.587224    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:21:21.889155 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:21:21.889124    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:21:22.101410 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:21:22.101377    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:21:24.540408 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:21:24.540375    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:21:24.920731 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:21:24.920651    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:21:25.020249 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:21:25.020222    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:21:25.314418 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:21:25.314387    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:21:25.999027 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:21:25.998990    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:21:26.772779 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:21:26.772743    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:21:27.524724 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:21:27.524697    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:21:27.627623 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:21:27.627594    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:21:27.728796 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:21:27.728759    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:21:27.969477 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:21:27.969449    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:21:28.068816 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:21:28.068780    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:21:47.806478 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:21:47.806439    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-vptdl"]
Sep 18 20:21:49.420696 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:21:49.420662    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:21:49.822969 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:21:49.822935    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:21:50.335178 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:21:50.335145    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:21:50.434465 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:21:50.434420    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:21:50.529257 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:21:50.529230    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:21:50.903641 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:21:50.903610    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:21:51.135338 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:21:51.135309    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:21:52.032695 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:21:52.032665    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:21:52.720641 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:21:52.720610    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:21:53.420802 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:21:53.420774    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:21:55.885726 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:21:55.885686    2532 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w0-116778/distinct-w0-116778-kserve-5f6955bd97-qf5vt"]
Sep 18 20:21:55.886109 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:21:55.886029    2532 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="da42bbbc-b3fb-48dd-841e-e193c95ba582" containerName="istio-proxy"
Sep 18 20:21:55.886109 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:21:55.886039    2532 state_mem.go:107] "Deleted CPUSet assignment" podUID="da42bbbc-b3fb-48dd-841e-e193c95ba582" containerName="istio-proxy"
Sep 18 20:21:55.886194 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:21:55.886118    2532 memory_manager.go:368] "RemoveStaleState removing state" podUID="da42bbbc-b3fb-48dd-841e-e193c95ba582" containerName="istio-proxy"
Sep 18 20:21:55.886194 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:21:55.886125    2532 memory_manager.go:368] "RemoveStaleState removing state" podUID="da42bbbc-b3fb-48dd-841e-e193c95ba582" containerName="istio-proxy"
Sep 18 20:21:55.889282 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:21:55.889263    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-116778/distinct-w0-116778-kserve-5f6955bd97-qf5vt"
Sep 18 20:21:55.892543 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:21:55.892523    2532 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-116778\"/\"distinct-w0-116778-kserve-self-signed-certs\""
Sep 18 20:21:55.897000 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:21:55.896977    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-116778/distinct-w0-116778-kserve-5f6955bd97-qf5vt"]
Sep 18 20:21:55.909509 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:21:55.909462    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:21:55.973691 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:21:55.973658    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f1270a9f-d699-4dc7-a101-df212a401b81-model-cache\") pod \"distinct-w0-116778-kserve-5f6955bd97-qf5vt\" (UID: \"f1270a9f-d699-4dc7-a101-df212a401b81\") " pod="e2e-models-e2e-worker-w0-116778/distinct-w0-116778-kserve-5f6955bd97-qf5vt"
Sep 18 20:21:55.973886 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:21:55.973700    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f1270a9f-d699-4dc7-a101-df212a401b81-tls-certs\") pod \"distinct-w0-116778-kserve-5f6955bd97-qf5vt\" (UID: \"f1270a9f-d699-4dc7-a101-df212a401b81\") " pod="e2e-models-e2e-worker-w0-116778/distinct-w0-116778-kserve-5f6955bd97-qf5vt"
Sep 18 20:21:55.973886 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:21:55.973739    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-97rsf\" (UniqueName: \"kubernetes.io/projected/f1270a9f-d699-4dc7-a101-df212a401b81-kube-api-access-97rsf\") pod \"distinct-w0-116778-kserve-5f6955bd97-qf5vt\" (UID: \"f1270a9f-d699-4dc7-a101-df212a401b81\") " pod="e2e-models-e2e-worker-w0-116778/distinct-w0-116778-kserve-5f6955bd97-qf5vt"
Sep 18 20:21:55.973886 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:21:55.973831    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f1270a9f-d699-4dc7-a101-df212a401b81-home\") pod \"distinct-w0-116778-kserve-5f6955bd97-qf5vt\" (UID: \"f1270a9f-d699-4dc7-a101-df212a401b81\") " pod="e2e-models-e2e-worker-w0-116778/distinct-w0-116778-kserve-5f6955bd97-qf5vt"
Sep 18 20:21:55.974052 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:21:55.973914    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f1270a9f-d699-4dc7-a101-df212a401b81-tmp-dir\") pod \"distinct-w0-116778-kserve-5f6955bd97-qf5vt\" (UID: \"f1270a9f-d699-4dc7-a101-df212a401b81\") " pod="e2e-models-e2e-worker-w0-116778/distinct-w0-116778-kserve-5f6955bd97-qf5vt"
Sep 18 20:21:55.974052 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:21:55.973965    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f1270a9f-d699-4dc7-a101-df212a401b81-dshm\") pod \"distinct-w0-116778-kserve-5f6955bd97-qf5vt\" (UID: \"f1270a9f-d699-4dc7-a101-df212a401b81\") " pod="e2e-models-e2e-worker-w0-116778/distinct-w0-116778-kserve-5f6955bd97-qf5vt"
Sep 18 20:21:56.075414 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:21:56.075379    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f1270a9f-d699-4dc7-a101-df212a401b81-tmp-dir\") pod \"distinct-w0-116778-kserve-5f6955bd97-qf5vt\" (UID: \"f1270a9f-d699-4dc7-a101-df212a401b81\") " pod="e2e-models-e2e-worker-w0-116778/distinct-w0-116778-kserve-5f6955bd97-qf5vt"
Sep 18 20:21:56.075414 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:21:56.075422    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f1270a9f-d699-4dc7-a101-df212a401b81-dshm\") pod \"distinct-w0-116778-kserve-5f6955bd97-qf5vt\" (UID: \"f1270a9f-d699-4dc7-a101-df212a401b81\") " pod="e2e-models-e2e-worker-w0-116778/distinct-w0-116778-kserve-5f6955bd97-qf5vt"
Sep 18 20:21:56.075676 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:21:56.075445    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f1270a9f-d699-4dc7-a101-df212a401b81-model-cache\") pod \"distinct-w0-116778-kserve-5f6955bd97-qf5vt\" (UID: \"f1270a9f-d699-4dc7-a101-df212a401b81\") " pod="e2e-models-e2e-worker-w0-116778/distinct-w0-116778-kserve-5f6955bd97-qf5vt"
Sep 18 20:21:56.075676 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:21:56.075466    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f1270a9f-d699-4dc7-a101-df212a401b81-tls-certs\") pod \"distinct-w0-116778-kserve-5f6955bd97-qf5vt\" (UID: \"f1270a9f-d699-4dc7-a101-df212a401b81\") " pod="e2e-models-e2e-worker-w0-116778/distinct-w0-116778-kserve-5f6955bd97-qf5vt"
Sep 18 20:21:56.075676 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:21:56.075517    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-97rsf\" (UniqueName: \"kubernetes.io/projected/f1270a9f-d699-4dc7-a101-df212a401b81-kube-api-access-97rsf\") pod \"distinct-w0-116778-kserve-5f6955bd97-qf5vt\" (UID: \"f1270a9f-d699-4dc7-a101-df212a401b81\") " pod="e2e-models-e2e-worker-w0-116778/distinct-w0-116778-kserve-5f6955bd97-qf5vt"
Sep 18 20:21:56.075859 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:21:56.075665    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f1270a9f-d699-4dc7-a101-df212a401b81-home\") pod \"distinct-w0-116778-kserve-5f6955bd97-qf5vt\" (UID: \"f1270a9f-d699-4dc7-a101-df212a401b81\") " pod="e2e-models-e2e-worker-w0-116778/distinct-w0-116778-kserve-5f6955bd97-qf5vt"
Sep 18 20:21:56.075916 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:21:56.075863    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f1270a9f-d699-4dc7-a101-df212a401b81-model-cache\") pod \"distinct-w0-116778-kserve-5f6955bd97-qf5vt\" (UID: \"f1270a9f-d699-4dc7-a101-df212a401b81\") " pod="e2e-models-e2e-worker-w0-116778/distinct-w0-116778-kserve-5f6955bd97-qf5vt"
Sep 18 20:21:56.075916 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:21:56.075876    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f1270a9f-d699-4dc7-a101-df212a401b81-tmp-dir\") pod \"distinct-w0-116778-kserve-5f6955bd97-qf5vt\" (UID: \"f1270a9f-d699-4dc7-a101-df212a401b81\") " pod="e2e-models-e2e-worker-w0-116778/distinct-w0-116778-kserve-5f6955bd97-qf5vt"
Sep 18 20:21:56.076008 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:21:56.075990    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f1270a9f-d699-4dc7-a101-df212a401b81-home\") pod \"distinct-w0-116778-kserve-5f6955bd97-qf5vt\" (UID: \"f1270a9f-d699-4dc7-a101-df212a401b81\") " pod="e2e-models-e2e-worker-w0-116778/distinct-w0-116778-kserve-5f6955bd97-qf5vt"
Sep 18 20:21:56.077659 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:21:56.077639    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f1270a9f-d699-4dc7-a101-df212a401b81-dshm\") pod \"distinct-w0-116778-kserve-5f6955bd97-qf5vt\" (UID: \"f1270a9f-d699-4dc7-a101-df212a401b81\") " pod="e2e-models-e2e-worker-w0-116778/distinct-w0-116778-kserve-5f6955bd97-qf5vt"
Sep 18 20:21:56.077947 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:21:56.077929    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f1270a9f-d699-4dc7-a101-df212a401b81-tls-certs\") pod \"distinct-w0-116778-kserve-5f6955bd97-qf5vt\" (UID: \"f1270a9f-d699-4dc7-a101-df212a401b81\") " pod="e2e-models-e2e-worker-w0-116778/distinct-w0-116778-kserve-5f6955bd97-qf5vt"
Sep 18 20:21:56.083089 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:21:56.083068    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-97rsf\" (UniqueName: \"kubernetes.io/projected/f1270a9f-d699-4dc7-a101-df212a401b81-kube-api-access-97rsf\") pod \"distinct-w0-116778-kserve-5f6955bd97-qf5vt\" (UID: \"f1270a9f-d699-4dc7-a101-df212a401b81\") " pod="e2e-models-e2e-worker-w0-116778/distinct-w0-116778-kserve-5f6955bd97-qf5vt"
Sep 18 20:21:56.198880 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:21:56.198793    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-116778/distinct-w0-116778-kserve-5f6955bd97-qf5vt"
Sep 18 20:21:56.469509 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:21:56.469465    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-116778/distinct-w0-116778-kserve-5f6955bd97-qf5vt"]
Sep 18 20:21:56.470747 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:21:56.470718    2532 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf1270a9f_d699_4dc7_a101_df212a401b81.slice/crio-eb16e34a15c943bf4acbea4ca42796ecf0dc5762f234f87079461083389b0b63 WatchSource:0}: Error finding container eb16e34a15c943bf4acbea4ca42796ecf0dc5762f234f87079461083389b0b63: Status 404 returned error can't find the container with id eb16e34a15c943bf4acbea4ca42796ecf0dc5762f234f87079461083389b0b63
Sep 18 20:21:56.695780 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:21:56.695749    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:21:56.758654 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:21:56.758571    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:21:57.354443 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:21:57.354410    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-116778/distinct-w0-116778-kserve-5f6955bd97-qf5vt" event={"ID":"f1270a9f-d699-4dc7-a101-df212a401b81","Type":"ContainerStarted","Data":"89925065f14a18f99d444f2c9e2b34d146713b2c14bee410b170f55b874eebe9"}
Sep 18 20:21:57.354443 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:21:57.354451    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-116778/distinct-w0-116778-kserve-5f6955bd97-qf5vt" event={"ID":"f1270a9f-d699-4dc7-a101-df212a401b81","Type":"ContainerStarted","Data":"eb16e34a15c943bf4acbea4ca42796ecf0dc5762f234f87079461083389b0b63"}
Sep 18 20:21:57.380821 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:21:57.380777    2532 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-models-e2e-worker-w0-116778/distinct-w0-116778-kserve-5f6955bd97-qf5vt" podStartSLOduration=2.380761819 podStartE2EDuration="2.380761819s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 20:21:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 20:21:57.379321718 +0000 UTC m=+1249.144711114" watchObservedRunningTime="2026-09-18 20:21:57.380761819 +0000 UTC m=+1249.146151219"
Sep 18 20:21:57.785232 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:21:57.785193    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:21:58.431708 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:21:58.431664    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:21:58.691765 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:21:58.691684    2532 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-e2dea3/test-model-e2dea3-kserve-84f88488c7-n2v8d"]
Sep 18 20:21:58.706827 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:21:58.706794    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:21:59.115283 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:21:59.115252    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:21:59.728623 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:21:59.728593    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:22:00.519420 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:22:00.519379    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:22:00.860695 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:22:00.860610    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:22:01.145265 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:22:01.145174    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:22:02.400406 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:22:02.400373    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:22:02.807749 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:22:02.807717    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:22:03.498991 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:22:03.498957    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:22:04.113186 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:22:04.113153    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:22:04.911074 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:22:04.911040    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:22:05.593230 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:22:05.593192    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:22:06.198979 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:22:06.198939    2532 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w0-116778/distinct-w0-116778-kserve-5f6955bd97-qf5vt"
Sep 18 20:22:06.199365 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:22:06.199002    2532 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-116778/distinct-w0-116778-kserve-5f6955bd97-qf5vt"
Sep 18 20:22:06.211557 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:22:06.211527    2532 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w0-116778/distinct-w0-116778-kserve-5f6955bd97-qf5vt"
Sep 18 20:22:06.409836 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:22:06.409808    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:22:06.418948 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:22:06.418922    2532 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-116778/distinct-w0-116778-kserve-5f6955bd97-qf5vt"
Sep 18 20:22:07.038354 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:22:07.038319    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:22:07.587699 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:22:07.587670    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:22:07.688221 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:22:07.688188    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:22:07.791644 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:22:07.791611    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:22:07.888323 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:22:07.888248    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:22:08.142418 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:22:08.142334    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:22:08.295858 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:22:08.295826    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:22:08.836813 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:22:08.836780    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:22:09.631731 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:22:09.631699    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:22:10.243392 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:22:10.243357    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:22:11.115197 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:22:11.115157    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:22:12.631479 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:22:12.631443    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:22:13.184600 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:22:13.184569    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:22:13.244670 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:22:13.244640    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:22:13.346358 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:22:13.346327    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:22:13.417761 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:22:13.417728    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:22:13.678660 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:22:13.678629    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:22:13.711000 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:22:13.710964    2532 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-a-e2dea3/test-model-e2dea3-kserve-84f88488c7-n2v8d" podUID="a8ac32f5-6438-4dfa-91ea-12e67ff6b4bf" containerName="main" containerID="cri-o://2d5735a568996d7a605ef4c995967f7509253a15094fad3ba907494061406369" gracePeriod=45
Sep 18 20:22:13.832422 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:22:13.832397    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:22:13.956716 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:22:13.956693    2532 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-e2dea3/test-model-e2dea3-kserve-84f88488c7-n2v8d"
Sep 18 20:22:14.043915 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:22:14.043880    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mtndb\" (UniqueName: \"kubernetes.io/projected/a8ac32f5-6438-4dfa-91ea-12e67ff6b4bf-kube-api-access-mtndb\") pod \"a8ac32f5-6438-4dfa-91ea-12e67ff6b4bf\" (UID: \"a8ac32f5-6438-4dfa-91ea-12e67ff6b4bf\") "
Sep 18 20:22:14.043915 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:22:14.043922    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a8ac32f5-6438-4dfa-91ea-12e67ff6b4bf-tls-certs\") pod \"a8ac32f5-6438-4dfa-91ea-12e67ff6b4bf\" (UID: \"a8ac32f5-6438-4dfa-91ea-12e67ff6b4bf\") "
Sep 18 20:22:14.044180 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:22:14.043958    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a8ac32f5-6438-4dfa-91ea-12e67ff6b4bf-home\") pod \"a8ac32f5-6438-4dfa-91ea-12e67ff6b4bf\" (UID: \"a8ac32f5-6438-4dfa-91ea-12e67ff6b4bf\") "
Sep 18 20:22:14.044180 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:22:14.044008    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a8ac32f5-6438-4dfa-91ea-12e67ff6b4bf-dshm\") pod \"a8ac32f5-6438-4dfa-91ea-12e67ff6b4bf\" (UID: \"a8ac32f5-6438-4dfa-91ea-12e67ff6b4bf\") "
Sep 18 20:22:14.044180 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:22:14.044036    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a8ac32f5-6438-4dfa-91ea-12e67ff6b4bf-tmp-dir\") pod \"a8ac32f5-6438-4dfa-91ea-12e67ff6b4bf\" (UID: \"a8ac32f5-6438-4dfa-91ea-12e67ff6b4bf\") "
Sep 18 20:22:14.044180 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:22:14.044109    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a8ac32f5-6438-4dfa-91ea-12e67ff6b4bf-model-cache\") pod \"a8ac32f5-6438-4dfa-91ea-12e67ff6b4bf\" (UID: \"a8ac32f5-6438-4dfa-91ea-12e67ff6b4bf\") "
Sep 18 20:22:14.044399 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:22:14.044319    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a8ac32f5-6438-4dfa-91ea-12e67ff6b4bf-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "a8ac32f5-6438-4dfa-91ea-12e67ff6b4bf" (UID: "a8ac32f5-6438-4dfa-91ea-12e67ff6b4bf"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 20:22:14.044399 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:22:14.044322    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a8ac32f5-6438-4dfa-91ea-12e67ff6b4bf-home" (OuterVolumeSpecName: "home") pod "a8ac32f5-6438-4dfa-91ea-12e67ff6b4bf" (UID: "a8ac32f5-6438-4dfa-91ea-12e67ff6b4bf"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 20:22:14.044517 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:22:14.044459    2532 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a8ac32f5-6438-4dfa-91ea-12e67ff6b4bf-tmp-dir\") on node \"ip-10-0-130-81.ec2.internal\" DevicePath \"\""
Sep 18 20:22:14.044517 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:22:14.044479    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a8ac32f5-6438-4dfa-91ea-12e67ff6b4bf-model-cache" (OuterVolumeSpecName: "model-cache") pod "a8ac32f5-6438-4dfa-91ea-12e67ff6b4bf" (UID: "a8ac32f5-6438-4dfa-91ea-12e67ff6b4bf"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 20:22:14.044517 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:22:14.044510    2532 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a8ac32f5-6438-4dfa-91ea-12e67ff6b4bf-home\") on node \"ip-10-0-130-81.ec2.internal\" DevicePath \"\""
Sep 18 20:22:14.046185 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:22:14.046165    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a8ac32f5-6438-4dfa-91ea-12e67ff6b4bf-dshm" (OuterVolumeSpecName: "dshm") pod "a8ac32f5-6438-4dfa-91ea-12e67ff6b4bf" (UID: "a8ac32f5-6438-4dfa-91ea-12e67ff6b4bf"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 20:22:14.046292 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:22:14.046271    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a8ac32f5-6438-4dfa-91ea-12e67ff6b4bf-kube-api-access-mtndb" (OuterVolumeSpecName: "kube-api-access-mtndb") pod "a8ac32f5-6438-4dfa-91ea-12e67ff6b4bf" (UID: "a8ac32f5-6438-4dfa-91ea-12e67ff6b4bf"). InnerVolumeSpecName "kube-api-access-mtndb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 20:22:14.046334 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:22:14.046310    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a8ac32f5-6438-4dfa-91ea-12e67ff6b4bf-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "a8ac32f5-6438-4dfa-91ea-12e67ff6b4bf" (UID: "a8ac32f5-6438-4dfa-91ea-12e67ff6b4bf"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 20:22:14.145237 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:22:14.145202    2532 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a8ac32f5-6438-4dfa-91ea-12e67ff6b4bf-dshm\") on node \"ip-10-0-130-81.ec2.internal\" DevicePath \"\""
Sep 18 20:22:14.145237 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:22:14.145229    2532 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a8ac32f5-6438-4dfa-91ea-12e67ff6b4bf-model-cache\") on node \"ip-10-0-130-81.ec2.internal\" DevicePath \"\""
Sep 18 20:22:14.145237 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:22:14.145240    2532 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mtndb\" (UniqueName: \"kubernetes.io/projected/a8ac32f5-6438-4dfa-91ea-12e67ff6b4bf-kube-api-access-mtndb\") on node \"ip-10-0-130-81.ec2.internal\" DevicePath \"\""
Sep 18 20:22:14.145237 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:22:14.145250    2532 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a8ac32f5-6438-4dfa-91ea-12e67ff6b4bf-tls-certs\") on node \"ip-10-0-130-81.ec2.internal\" DevicePath \"\""
Sep 18 20:22:14.453178 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:22:14.453142    2532 generic.go:356] "Generic (PLEG): container finished" podID="a8ac32f5-6438-4dfa-91ea-12e67ff6b4bf" containerID="2d5735a568996d7a605ef4c995967f7509253a15094fad3ba907494061406369" exitCode=0
Sep 18 20:22:14.453370 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:22:14.453189    2532 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-e2dea3/test-model-e2dea3-kserve-84f88488c7-n2v8d"
Sep 18 20:22:14.453370 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:22:14.453238    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-e2dea3/test-model-e2dea3-kserve-84f88488c7-n2v8d" event={"ID":"a8ac32f5-6438-4dfa-91ea-12e67ff6b4bf","Type":"ContainerDied","Data":"2d5735a568996d7a605ef4c995967f7509253a15094fad3ba907494061406369"}
Sep 18 20:22:14.453370 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:22:14.453287    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-e2dea3/test-model-e2dea3-kserve-84f88488c7-n2v8d" event={"ID":"a8ac32f5-6438-4dfa-91ea-12e67ff6b4bf","Type":"ContainerDied","Data":"5c0bc438407dde4ac234830e6744d451e8eab8aa238147943469e8021c8cbd8e"}
Sep 18 20:22:14.453370 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:22:14.453308    2532 scope.go:117] "RemoveContainer" containerID="2d5735a568996d7a605ef4c995967f7509253a15094fad3ba907494061406369"
Sep 18 20:22:14.462520 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:22:14.462476    2532 scope.go:117] "RemoveContainer" containerID="2d5735a568996d7a605ef4c995967f7509253a15094fad3ba907494061406369"
Sep 18 20:22:14.462784 ip-10-0-130-81 kubenswrapper[2532]: E0918 20:22:14.462756    2532 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2d5735a568996d7a605ef4c995967f7509253a15094fad3ba907494061406369\": container with ID starting with 2d5735a568996d7a605ef4c995967f7509253a15094fad3ba907494061406369 not found: ID does not exist" containerID="2d5735a568996d7a605ef4c995967f7509253a15094fad3ba907494061406369"
Sep 18 20:22:14.462864 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:22:14.462795    2532 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2d5735a568996d7a605ef4c995967f7509253a15094fad3ba907494061406369"} err="failed to get container status \"2d5735a568996d7a605ef4c995967f7509253a15094fad3ba907494061406369\": rpc error: code = NotFound desc = could not find container \"2d5735a568996d7a605ef4c995967f7509253a15094fad3ba907494061406369\": container with ID starting with 2d5735a568996d7a605ef4c995967f7509253a15094fad3ba907494061406369 not found: ID does not exist"
Sep 18 20:22:14.473284 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:22:14.473259    2532 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-e2dea3/test-model-e2dea3-kserve-84f88488c7-n2v8d"]
Sep 18 20:22:14.477220 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:22:14.477189    2532 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-a-e2dea3/test-model-e2dea3-kserve-84f88488c7-n2v8d"]
Sep 18 20:22:14.563120 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:22:14.563052    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:22:14.663971 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:22:14.663848    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:22:14.840560 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:22:14.840525    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:22:15.142701 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:22:15.142638    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:22:15.244720 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:22:15.244687    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:22:15.410738 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:22:15.410634    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:22:15.508656 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:22:15.508627    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:22:16.010027 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:22:16.009991    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:22:16.308341 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:22:16.308265    2532 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a8ac32f5-6438-4dfa-91ea-12e67ff6b4bf" path="/var/lib/kubelet/pods/a8ac32f5-6438-4dfa-91ea-12e67ff6b4bf/volumes"
Sep 18 20:22:19.594870 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:22:19.594835    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:22:19.662005 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:22:19.661969    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:22:19.829765 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:22:19.829734    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:22:19.927325 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:22:19.927243    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:22:20.805333 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:22:20.805304    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:22:21.445142 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:22:21.445074    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:22:21.546461 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:22:21.546428    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:22:21.647716 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:22:21.647670    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:22:21.769267 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:22:21.769209    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:22:22.221097 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:22:22.221020    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:22:22.946777 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:22:22.946743    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:22:27.332863 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:22:27.332825    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:22:27.396148 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:22:27.396101    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:22:28.171279 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:22:28.171246    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:22:28.459216 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:22:28.459187    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:22:28.646644 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:22:28.646612    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:22:28.909913 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:22:28.909815    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-vptdl"]
Sep 18 20:22:30.207129 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:22:30.207086    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:22:30.325567 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:22:30.325534    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:22:31.255114 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:22:31.255077    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:22:31.487813 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:22:31.487779    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:22:31.583798 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:22:31.583693    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:22:32.004304 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:22:32.004252    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:22:32.706646 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:22:32.706612    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:22:33.220224 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:22:33.220189    2532 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-a85f8b-75d49bbcb8-wj8lj"]
Sep 18 20:22:33.220510 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:22:33.220464    2532 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a85f8b-75d49bbcb8-wj8lj" podUID="b953a796-0402-46c7-bef7-696262fcc8fc" containerName="maas-api" containerID="cri-o://3868648a51b2df5ffc6d41f522eaab621010533b181cdb69678a62b361e70c88" gracePeriod=30
Sep 18 20:22:33.252794 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:22:33.252767    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:22:33.349932 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:22:33.349902    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:22:33.451174 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:22:33.451144    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:22:33.468948 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:22:33.468925    2532 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a85f8b-75d49bbcb8-wj8lj"
Sep 18 20:22:33.520920 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:22:33.520838    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jq2bq\" (UniqueName: \"kubernetes.io/projected/b953a796-0402-46c7-bef7-696262fcc8fc-kube-api-access-jq2bq\") pod \"b953a796-0402-46c7-bef7-696262fcc8fc\" (UID: \"b953a796-0402-46c7-bef7-696262fcc8fc\") "
Sep 18 20:22:33.520920 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:22:33.520878    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b953a796-0402-46c7-bef7-696262fcc8fc-openshift-service-ca\") pod \"b953a796-0402-46c7-bef7-696262fcc8fc\" (UID: \"b953a796-0402-46c7-bef7-696262fcc8fc\") "
Sep 18 20:22:33.521146 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:22:33.520925    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b953a796-0402-46c7-bef7-696262fcc8fc-maas-api-tls\") pod \"b953a796-0402-46c7-bef7-696262fcc8fc\" (UID: \"b953a796-0402-46c7-bef7-696262fcc8fc\") "
Sep 18 20:22:33.521146 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:22:33.521042    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b953a796-0402-46c7-bef7-696262fcc8fc-metrics-certs\") pod \"b953a796-0402-46c7-bef7-696262fcc8fc\" (UID: \"b953a796-0402-46c7-bef7-696262fcc8fc\") "
Sep 18 20:22:33.521361 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:22:33.521320    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b953a796-0402-46c7-bef7-696262fcc8fc-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "b953a796-0402-46c7-bef7-696262fcc8fc" (UID: "b953a796-0402-46c7-bef7-696262fcc8fc"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 20:22:33.523273 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:22:33.523243    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b953a796-0402-46c7-bef7-696262fcc8fc-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "b953a796-0402-46c7-bef7-696262fcc8fc" (UID: "b953a796-0402-46c7-bef7-696262fcc8fc"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 20:22:33.523668 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:22:33.523647    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b953a796-0402-46c7-bef7-696262fcc8fc-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "b953a796-0402-46c7-bef7-696262fcc8fc" (UID: "b953a796-0402-46c7-bef7-696262fcc8fc"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 20:22:33.523745 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:22:33.523670    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b953a796-0402-46c7-bef7-696262fcc8fc-kube-api-access-jq2bq" (OuterVolumeSpecName: "kube-api-access-jq2bq") pod "b953a796-0402-46c7-bef7-696262fcc8fc" (UID: "b953a796-0402-46c7-bef7-696262fcc8fc"). InnerVolumeSpecName "kube-api-access-jq2bq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 20:22:33.551710 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:22:33.551673    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:22:33.564754 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:22:33.564722    2532 generic.go:356] "Generic (PLEG): container finished" podID="b953a796-0402-46c7-bef7-696262fcc8fc" containerID="3868648a51b2df5ffc6d41f522eaab621010533b181cdb69678a62b361e70c88" exitCode=0
Sep 18 20:22:33.564867 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:22:33.564759    2532 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a85f8b-75d49bbcb8-wj8lj"
Sep 18 20:22:33.564867 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:22:33.564843    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a85f8b-75d49bbcb8-wj8lj" event={"ID":"b953a796-0402-46c7-bef7-696262fcc8fc","Type":"ContainerDied","Data":"3868648a51b2df5ffc6d41f522eaab621010533b181cdb69678a62b361e70c88"}
Sep 18 20:22:33.564967 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:22:33.564898    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a85f8b-75d49bbcb8-wj8lj" event={"ID":"b953a796-0402-46c7-bef7-696262fcc8fc","Type":"ContainerDied","Data":"f11db38c9a1d29b1204b2caa2a46c8aba60b88a1a1964ad748046430bb061ef5"}
Sep 18 20:22:33.564967 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:22:33.564932    2532 scope.go:117] "RemoveContainer" containerID="3868648a51b2df5ffc6d41f522eaab621010533b181cdb69678a62b361e70c88"
Sep 18 20:22:33.575115 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:22:33.575095    2532 scope.go:117] "RemoveContainer" containerID="3868648a51b2df5ffc6d41f522eaab621010533b181cdb69678a62b361e70c88"
Sep 18 20:22:33.575443 ip-10-0-130-81 kubenswrapper[2532]: E0918 20:22:33.575413    2532 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3868648a51b2df5ffc6d41f522eaab621010533b181cdb69678a62b361e70c88\": container with ID starting with 3868648a51b2df5ffc6d41f522eaab621010533b181cdb69678a62b361e70c88 not found: ID does not exist" containerID="3868648a51b2df5ffc6d41f522eaab621010533b181cdb69678a62b361e70c88"
Sep 18 20:22:33.575579 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:22:33.575448    2532 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3868648a51b2df5ffc6d41f522eaab621010533b181cdb69678a62b361e70c88"} err="failed to get container status \"3868648a51b2df5ffc6d41f522eaab621010533b181cdb69678a62b361e70c88\": rpc error: code = NotFound desc = could not find container \"3868648a51b2df5ffc6d41f522eaab621010533b181cdb69678a62b361e70c88\": container with ID starting with 3868648a51b2df5ffc6d41f522eaab621010533b181cdb69678a62b361e70c88 not found: ID does not exist"
Sep 18 20:22:33.591429 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:22:33.591394    2532 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-a85f8b-75d49bbcb8-wj8lj"]
Sep 18 20:22:33.594142 ip-10-0-130-81 kubenswrapper[2532]: E0918 20:22:33.594113    2532 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb953a796_0402_46c7_bef7_696262fcc8fc.slice/crio-f11db38c9a1d29b1204b2caa2a46c8aba60b88a1a1964ad748046430bb061ef5\": RecentStats: unable to find data in memory cache]"
Sep 18 20:22:33.600034 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:22:33.600013    2532 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-a85f8b-75d49bbcb8-wj8lj"]
Sep 18 20:22:33.622567 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:22:33.622536    2532 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b953a796-0402-46c7-bef7-696262fcc8fc-metrics-certs\") on node \"ip-10-0-130-81.ec2.internal\" DevicePath \"\""
Sep 18 20:22:33.622567 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:22:33.622570    2532 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jq2bq\" (UniqueName: \"kubernetes.io/projected/b953a796-0402-46c7-bef7-696262fcc8fc-kube-api-access-jq2bq\") on node \"ip-10-0-130-81.ec2.internal\" DevicePath \"\""
Sep 18 20:22:33.622757 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:22:33.622586    2532 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b953a796-0402-46c7-bef7-696262fcc8fc-openshift-service-ca\") on node \"ip-10-0-130-81.ec2.internal\" DevicePath \"\""
Sep 18 20:22:33.622757 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:22:33.622602    2532 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b953a796-0402-46c7-bef7-696262fcc8fc-maas-api-tls\") on node \"ip-10-0-130-81.ec2.internal\" DevicePath \"\""
Sep 18 20:22:33.650390 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:22:33.650330    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:22:33.776474 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:22:33.776369    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:22:33.945314 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:22:33.945284    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:22:34.110805 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:22:34.110701    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:22:34.191673 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:22:34.191643    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:22:34.307232 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:22:34.307194    2532 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b953a796-0402-46c7-bef7-696262fcc8fc" path="/var/lib/kubelet/pods/b953a796-0402-46c7-bef7-696262fcc8fc/volumes"
Sep 18 20:22:34.529803 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:22:34.520338    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:22:35.303560 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:22:35.303519    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:22:36.025539 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:22:36.025508    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:22:36.630084 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:22:36.630051    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:22:39.503537 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:22:39.503475    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:22:39.576376 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:22:39.576341    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:22:39.729129 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:22:39.729099    2532 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-a85f8b-openshift-default-68dd69554d-dkbnd"]
Sep 18 20:22:39.729383 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:22:39.729348    2532 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-a85f8b-openshift-default-68dd69554d-dkbnd" podUID="b1db05d0-08eb-47c0-ad79-65727da00166" containerName="istio-proxy" containerID="cri-o://0a2c5d67da837e175e585a1734b3f5c8a27d506c9e00170a7d1cb0454739476f" gracePeriod=30
Sep 18 20:22:39.754284 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:22:39.754208    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:22:40.259257 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:22:40.259225    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:22:40.531426 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:22:40.531336    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:22:41.172718 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:22:41.172687    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:22:41.877395 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:22:41.877357    2532 patch_prober.go:28] interesting pod/e2e-inf-b-a85f8b-openshift-default-68dd69554d-dkbnd container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 18 20:22:41.877793 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:22:41.877425    2532 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-b-a85f8b-openshift-default-68dd69554d-dkbnd" podUID="b1db05d0-08eb-47c0-ad79-65727da00166" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 18 20:22:42.817526 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:22:42.817469    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:22:43.343515 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:22:43.343456    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:22:43.445403 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:22:43.445366    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:22:43.544633 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:22:43.544602    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:22:44.113104 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:22:44.113069    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:22:44.981550 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:22:44.981523    2532 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-a85f8b-openshift-default-68dd69554d-dkbnd"
Sep 18 20:22:45.031533 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:22:45.031424    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:22:45.131803 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:22:45.131767    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b1db05d0-08eb-47c0-ad79-65727da00166-istio-envoy\") pod \"b1db05d0-08eb-47c0-ad79-65727da00166\" (UID: \"b1db05d0-08eb-47c0-ad79-65727da00166\") "
Sep 18 20:22:45.131803 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:22:45.131810    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b1db05d0-08eb-47c0-ad79-65727da00166-credential-socket\") pod \"b1db05d0-08eb-47c0-ad79-65727da00166\" (UID: \"b1db05d0-08eb-47c0-ad79-65727da00166\") "
Sep 18 20:22:45.132067 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:22:45.131836    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b1db05d0-08eb-47c0-ad79-65727da00166-istio-data\") pod \"b1db05d0-08eb-47c0-ad79-65727da00166\" (UID: \"b1db05d0-08eb-47c0-ad79-65727da00166\") "
Sep 18 20:22:45.132067 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:22:45.131869    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b1db05d0-08eb-47c0-ad79-65727da00166-workload-certs\") pod \"b1db05d0-08eb-47c0-ad79-65727da00166\" (UID: \"b1db05d0-08eb-47c0-ad79-65727da00166\") "
Sep 18 20:22:45.132067 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:22:45.131896    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b1db05d0-08eb-47c0-ad79-65727da00166-istio-podinfo\") pod \"b1db05d0-08eb-47c0-ad79-65727da00166\" (UID: \"b1db05d0-08eb-47c0-ad79-65727da00166\") "
Sep 18 20:22:45.132067 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:22:45.131919    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-g6dlm\" (UniqueName: \"kubernetes.io/projected/b1db05d0-08eb-47c0-ad79-65727da00166-kube-api-access-g6dlm\") pod \"b1db05d0-08eb-47c0-ad79-65727da00166\" (UID: \"b1db05d0-08eb-47c0-ad79-65727da00166\") "
Sep 18 20:22:45.132067 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:22:45.131974    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b1db05d0-08eb-47c0-ad79-65727da00166-istio-token\") pod \"b1db05d0-08eb-47c0-ad79-65727da00166\" (UID: \"b1db05d0-08eb-47c0-ad79-65727da00166\") "
Sep 18 20:22:45.132067 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:22:45.132021    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b1db05d0-08eb-47c0-ad79-65727da00166-workload-socket\") pod \"b1db05d0-08eb-47c0-ad79-65727da00166\" (UID: \"b1db05d0-08eb-47c0-ad79-65727da00166\") "
Sep 18 20:22:45.132067 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:22:45.132049    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b1db05d0-08eb-47c0-ad79-65727da00166-istiod-ca-cert\") pod \"b1db05d0-08eb-47c0-ad79-65727da00166\" (UID: \"b1db05d0-08eb-47c0-ad79-65727da00166\") "
Sep 18 20:22:45.132429 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:22:45.132168    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b1db05d0-08eb-47c0-ad79-65727da00166-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "b1db05d0-08eb-47c0-ad79-65727da00166" (UID: "b1db05d0-08eb-47c0-ad79-65727da00166"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 20:22:45.132429 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:22:45.132269    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b1db05d0-08eb-47c0-ad79-65727da00166-istio-data" (OuterVolumeSpecName: "istio-data") pod "b1db05d0-08eb-47c0-ad79-65727da00166" (UID: "b1db05d0-08eb-47c0-ad79-65727da00166"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 20:22:45.132577 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:22:45.132278    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b1db05d0-08eb-47c0-ad79-65727da00166-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "b1db05d0-08eb-47c0-ad79-65727da00166" (UID: "b1db05d0-08eb-47c0-ad79-65727da00166"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 20:22:45.132577 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:22:45.132381    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b1db05d0-08eb-47c0-ad79-65727da00166-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "b1db05d0-08eb-47c0-ad79-65727da00166" (UID: "b1db05d0-08eb-47c0-ad79-65727da00166"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 20:22:45.132684 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:22:45.132614    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b1db05d0-08eb-47c0-ad79-65727da00166-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "b1db05d0-08eb-47c0-ad79-65727da00166" (UID: "b1db05d0-08eb-47c0-ad79-65727da00166"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 20:22:45.134121 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:22:45.134098    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b1db05d0-08eb-47c0-ad79-65727da00166-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "b1db05d0-08eb-47c0-ad79-65727da00166" (UID: "b1db05d0-08eb-47c0-ad79-65727da00166"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 20:22:45.134403 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:22:45.134386    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b1db05d0-08eb-47c0-ad79-65727da00166-kube-api-access-g6dlm" (OuterVolumeSpecName: "kube-api-access-g6dlm") pod "b1db05d0-08eb-47c0-ad79-65727da00166" (UID: "b1db05d0-08eb-47c0-ad79-65727da00166"). InnerVolumeSpecName "kube-api-access-g6dlm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 20:22:45.134581 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:22:45.134560    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/b1db05d0-08eb-47c0-ad79-65727da00166-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "b1db05d0-08eb-47c0-ad79-65727da00166" (UID: "b1db05d0-08eb-47c0-ad79-65727da00166"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 18 20:22:45.134680 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:22:45.134608    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b1db05d0-08eb-47c0-ad79-65727da00166-istio-token" (OuterVolumeSpecName: "istio-token") pod "b1db05d0-08eb-47c0-ad79-65727da00166" (UID: "b1db05d0-08eb-47c0-ad79-65727da00166"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 20:22:45.233394 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:22:45.233358    2532 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g6dlm\" (UniqueName: \"kubernetes.io/projected/b1db05d0-08eb-47c0-ad79-65727da00166-kube-api-access-g6dlm\") on node \"ip-10-0-130-81.ec2.internal\" DevicePath \"\""
Sep 18 20:22:45.233394 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:22:45.233389    2532 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b1db05d0-08eb-47c0-ad79-65727da00166-istio-token\") on node \"ip-10-0-130-81.ec2.internal\" DevicePath \"\""
Sep 18 20:22:45.233394 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:22:45.233399    2532 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b1db05d0-08eb-47c0-ad79-65727da00166-workload-socket\") on node \"ip-10-0-130-81.ec2.internal\" DevicePath \"\""
Sep 18 20:22:45.233655 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:22:45.233407    2532 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b1db05d0-08eb-47c0-ad79-65727da00166-istiod-ca-cert\") on node \"ip-10-0-130-81.ec2.internal\" DevicePath \"\""
Sep 18 20:22:45.233655 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:22:45.233416    2532 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b1db05d0-08eb-47c0-ad79-65727da00166-istio-envoy\") on node \"ip-10-0-130-81.ec2.internal\" DevicePath \"\""
Sep 18 20:22:45.233655 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:22:45.233423    2532 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b1db05d0-08eb-47c0-ad79-65727da00166-credential-socket\") on node \"ip-10-0-130-81.ec2.internal\" DevicePath \"\""
Sep 18 20:22:45.233655 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:22:45.233431    2532 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b1db05d0-08eb-47c0-ad79-65727da00166-istio-data\") on node \"ip-10-0-130-81.ec2.internal\" DevicePath \"\""
Sep 18 20:22:45.233655 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:22:45.233439    2532 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b1db05d0-08eb-47c0-ad79-65727da00166-workload-certs\") on node \"ip-10-0-130-81.ec2.internal\" DevicePath \"\""
Sep 18 20:22:45.233655 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:22:45.233446    2532 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b1db05d0-08eb-47c0-ad79-65727da00166-istio-podinfo\") on node \"ip-10-0-130-81.ec2.internal\" DevicePath \"\""
Sep 18 20:22:45.636295 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:22:45.636262    2532 generic.go:356] "Generic (PLEG): container finished" podID="b1db05d0-08eb-47c0-ad79-65727da00166" containerID="0a2c5d67da837e175e585a1734b3f5c8a27d506c9e00170a7d1cb0454739476f" exitCode=0
Sep 18 20:22:45.636506 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:22:45.636330    2532 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-a85f8b-openshift-default-68dd69554d-dkbnd"
Sep 18 20:22:45.636506 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:22:45.636352    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-a85f8b-openshift-default-68dd69554d-dkbnd" event={"ID":"b1db05d0-08eb-47c0-ad79-65727da00166","Type":"ContainerDied","Data":"0a2c5d67da837e175e585a1734b3f5c8a27d506c9e00170a7d1cb0454739476f"}
Sep 18 20:22:45.636506 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:22:45.636415    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-a85f8b-openshift-default-68dd69554d-dkbnd" event={"ID":"b1db05d0-08eb-47c0-ad79-65727da00166","Type":"ContainerDied","Data":"8ae439f122df595798f6e1394818af119a89e6c16d48d842a0d5a407e997611d"}
Sep 18 20:22:45.636506 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:22:45.636445    2532 scope.go:117] "RemoveContainer" containerID="0a2c5d67da837e175e585a1734b3f5c8a27d506c9e00170a7d1cb0454739476f"
Sep 18 20:22:45.646260 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:22:45.646223    2532 scope.go:117] "RemoveContainer" containerID="a3762b002234f42683cf694e1366356522186f2c6a113a3464bd79ad83b50397"
Sep 18 20:22:45.654982 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:22:45.654967    2532 scope.go:117] "RemoveContainer" containerID="0a2c5d67da837e175e585a1734b3f5c8a27d506c9e00170a7d1cb0454739476f"
Sep 18 20:22:45.655291 ip-10-0-130-81 kubenswrapper[2532]: E0918 20:22:45.655258    2532 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0a2c5d67da837e175e585a1734b3f5c8a27d506c9e00170a7d1cb0454739476f\": container with ID starting with 0a2c5d67da837e175e585a1734b3f5c8a27d506c9e00170a7d1cb0454739476f not found: ID does not exist" containerID="0a2c5d67da837e175e585a1734b3f5c8a27d506c9e00170a7d1cb0454739476f"
Sep 18 20:22:45.655399 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:22:45.655287    2532 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0a2c5d67da837e175e585a1734b3f5c8a27d506c9e00170a7d1cb0454739476f"} err="failed to get container status \"0a2c5d67da837e175e585a1734b3f5c8a27d506c9e00170a7d1cb0454739476f\": rpc error: code = NotFound desc = could not find container \"0a2c5d67da837e175e585a1734b3f5c8a27d506c9e00170a7d1cb0454739476f\": container with ID starting with 0a2c5d67da837e175e585a1734b3f5c8a27d506c9e00170a7d1cb0454739476f not found: ID does not exist"
Sep 18 20:22:45.655399 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:22:45.655304    2532 scope.go:117] "RemoveContainer" containerID="a3762b002234f42683cf694e1366356522186f2c6a113a3464bd79ad83b50397"
Sep 18 20:22:45.655558 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:22:45.655522    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:22:45.655767 ip-10-0-130-81 kubenswrapper[2532]: E0918 20:22:45.655746    2532 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a3762b002234f42683cf694e1366356522186f2c6a113a3464bd79ad83b50397\": container with ID starting with a3762b002234f42683cf694e1366356522186f2c6a113a3464bd79ad83b50397 not found: ID does not exist" containerID="a3762b002234f42683cf694e1366356522186f2c6a113a3464bd79ad83b50397"
Sep 18 20:22:45.655822 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:22:45.655773    2532 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a3762b002234f42683cf694e1366356522186f2c6a113a3464bd79ad83b50397"} err="failed to get container status \"a3762b002234f42683cf694e1366356522186f2c6a113a3464bd79ad83b50397\": rpc error: code = NotFound desc = could not find container \"a3762b002234f42683cf694e1366356522186f2c6a113a3464bd79ad83b50397\": container with ID starting with a3762b002234f42683cf694e1366356522186f2c6a113a3464bd79ad83b50397 not found: ID does not exist"
Sep 18 20:22:45.662633 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:22:45.662610    2532 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-a85f8b-openshift-default-68dd69554d-dkbnd"]
Sep 18 20:22:45.666655 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:22:45.666628    2532 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-a85f8b-openshift-default-68dd69554d-dkbnd"]
Sep 18 20:22:46.172297 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:22:46.172264    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:22:46.297605 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:22:46.297573    2532 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-bab50d-58db4bc787-q2s49"]
Sep 18 20:22:46.298023 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:22:46.298002    2532 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="da42bbbc-b3fb-48dd-841e-e193c95ba582" containerName="istio-proxy"
Sep 18 20:22:46.298138 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:22:46.298043    2532 state_mem.go:107] "Deleted CPUSet assignment" podUID="da42bbbc-b3fb-48dd-841e-e193c95ba582" containerName="istio-proxy"
Sep 18 20:22:46.298138 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:22:46.298082    2532 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b1db05d0-08eb-47c0-ad79-65727da00166" containerName="istio-proxy"
Sep 18 20:22:46.298138 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:22:46.298091    2532 state_mem.go:107] "Deleted CPUSet assignment" podUID="b1db05d0-08eb-47c0-ad79-65727da00166" containerName="istio-proxy"
Sep 18 20:22:46.298138 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:22:46.298108    2532 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a8ac32f5-6438-4dfa-91ea-12e67ff6b4bf" containerName="main"
Sep 18 20:22:46.298138 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:22:46.298116    2532 state_mem.go:107] "Deleted CPUSet assignment" podUID="a8ac32f5-6438-4dfa-91ea-12e67ff6b4bf" containerName="main"
Sep 18 20:22:46.298138 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:22:46.298130    2532 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b953a796-0402-46c7-bef7-696262fcc8fc" containerName="maas-api"
Sep 18 20:22:46.298138 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:22:46.298138    2532 state_mem.go:107] "Deleted CPUSet assignment" podUID="b953a796-0402-46c7-bef7-696262fcc8fc" containerName="maas-api"
Sep 18 20:22:46.298472 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:22:46.298150    2532 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b1db05d0-08eb-47c0-ad79-65727da00166" containerName="istio-proxy"
Sep 18 20:22:46.298472 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:22:46.298161    2532 state_mem.go:107] "Deleted CPUSet assignment" podUID="b1db05d0-08eb-47c0-ad79-65727da00166" containerName="istio-proxy"
Sep 18 20:22:46.298472 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:22:46.298259    2532 memory_manager.go:368] "RemoveStaleState removing state" podUID="b1db05d0-08eb-47c0-ad79-65727da00166" containerName="istio-proxy"
Sep 18 20:22:46.298472 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:22:46.298272    2532 memory_manager.go:368] "RemoveStaleState removing state" podUID="b1db05d0-08eb-47c0-ad79-65727da00166" containerName="istio-proxy"
Sep 18 20:22:46.298472 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:22:46.298280    2532 memory_manager.go:368] "RemoveStaleState removing state" podUID="a8ac32f5-6438-4dfa-91ea-12e67ff6b4bf" containerName="main"
Sep 18 20:22:46.298472 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:22:46.298288    2532 memory_manager.go:368] "RemoveStaleState removing state" podUID="b953a796-0402-46c7-bef7-696262fcc8fc" containerName="maas-api"
Sep 18 20:22:46.303473 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:22:46.303450    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-bab50d-58db4bc787-q2s49"
Sep 18 20:22:46.306395 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:22:46.306217    2532 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-w2-bab50d-serving-cert\""
Sep 18 20:22:46.306602 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:22:46.306412    2532 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Sep 18 20:22:46.307855 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:22:46.307623    2532 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 18 20:22:46.307855 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:22:46.307838    2532 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Sep 18 20:22:46.308023 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:22:46.307919    2532 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-fhqv4\""
Sep 18 20:22:46.320589 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:22:46.320561    2532 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b1db05d0-08eb-47c0-ad79-65727da00166" path="/var/lib/kubelet/pods/b1db05d0-08eb-47c0-ad79-65727da00166/volumes"
Sep 18 20:22:46.321121 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:22:46.321102    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-bab50d-58db4bc787-q2s49"]
Sep 18 20:22:46.334670 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:22:46.334640    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:22:46.436077 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:22:46.435997    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:22:46.444259 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:22:46.444218    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/80f6c7c5-c42b-4c88-b5c5-bf1de8a683d1-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w2-bab50d-58db4bc787-q2s49\" (UID: \"80f6c7c5-c42b-4c88-b5c5-bf1de8a683d1\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-bab50d-58db4bc787-q2s49"
Sep 18 20:22:46.444417 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:22:46.444339    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/80f6c7c5-c42b-4c88-b5c5-bf1de8a683d1-maas-api-tls\") pod \"maas-api-e2e-shared-a-w2-bab50d-58db4bc787-q2s49\" (UID: \"80f6c7c5-c42b-4c88-b5c5-bf1de8a683d1\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-bab50d-58db4bc787-q2s49"
Sep 18 20:22:46.444505 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:22:46.444420    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/80f6c7c5-c42b-4c88-b5c5-bf1de8a683d1-metrics-certs\") pod \"maas-api-e2e-shared-a-w2-bab50d-58db4bc787-q2s49\" (UID: \"80f6c7c5-c42b-4c88-b5c5-bf1de8a683d1\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-bab50d-58db4bc787-q2s49"
Sep 18 20:22:46.444604 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:22:46.444583    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5vnbk\" (UniqueName: \"kubernetes.io/projected/80f6c7c5-c42b-4c88-b5c5-bf1de8a683d1-kube-api-access-5vnbk\") pod \"maas-api-e2e-shared-a-w2-bab50d-58db4bc787-q2s49\" (UID: \"80f6c7c5-c42b-4c88-b5c5-bf1de8a683d1\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-bab50d-58db4bc787-q2s49"
Sep 18 20:22:46.536751 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:22:46.536721    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:22:46.545381 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:22:46.545351    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/80f6c7c5-c42b-4c88-b5c5-bf1de8a683d1-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w2-bab50d-58db4bc787-q2s49\" (UID: \"80f6c7c5-c42b-4c88-b5c5-bf1de8a683d1\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-bab50d-58db4bc787-q2s49"
Sep 18 20:22:46.545529 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:22:46.545421    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/80f6c7c5-c42b-4c88-b5c5-bf1de8a683d1-maas-api-tls\") pod \"maas-api-e2e-shared-a-w2-bab50d-58db4bc787-q2s49\" (UID: \"80f6c7c5-c42b-4c88-b5c5-bf1de8a683d1\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-bab50d-58db4bc787-q2s49"
Sep 18 20:22:46.545529 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:22:46.545461    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/80f6c7c5-c42b-4c88-b5c5-bf1de8a683d1-metrics-certs\") pod \"maas-api-e2e-shared-a-w2-bab50d-58db4bc787-q2s49\" (UID: \"80f6c7c5-c42b-4c88-b5c5-bf1de8a683d1\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-bab50d-58db4bc787-q2s49"
Sep 18 20:22:46.545529 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:22:46.545509    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5vnbk\" (UniqueName: \"kubernetes.io/projected/80f6c7c5-c42b-4c88-b5c5-bf1de8a683d1-kube-api-access-5vnbk\") pod \"maas-api-e2e-shared-a-w2-bab50d-58db4bc787-q2s49\" (UID: \"80f6c7c5-c42b-4c88-b5c5-bf1de8a683d1\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-bab50d-58db4bc787-q2s49"
Sep 18 20:22:46.546134 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:22:46.546115    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/80f6c7c5-c42b-4c88-b5c5-bf1de8a683d1-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w2-bab50d-58db4bc787-q2s49\" (UID: \"80f6c7c5-c42b-4c88-b5c5-bf1de8a683d1\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-bab50d-58db4bc787-q2s49"
Sep 18 20:22:46.547868 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:22:46.547847    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/80f6c7c5-c42b-4c88-b5c5-bf1de8a683d1-maas-api-tls\") pod \"maas-api-e2e-shared-a-w2-bab50d-58db4bc787-q2s49\" (UID: \"80f6c7c5-c42b-4c88-b5c5-bf1de8a683d1\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-bab50d-58db4bc787-q2s49"
Sep 18 20:22:46.547963 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:22:46.547953    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/80f6c7c5-c42b-4c88-b5c5-bf1de8a683d1-metrics-certs\") pod \"maas-api-e2e-shared-a-w2-bab50d-58db4bc787-q2s49\" (UID: \"80f6c7c5-c42b-4c88-b5c5-bf1de8a683d1\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-bab50d-58db4bc787-q2s49"
Sep 18 20:22:46.559633 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:22:46.559609    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5vnbk\" (UniqueName: \"kubernetes.io/projected/80f6c7c5-c42b-4c88-b5c5-bf1de8a683d1-kube-api-access-5vnbk\") pod \"maas-api-e2e-shared-a-w2-bab50d-58db4bc787-q2s49\" (UID: \"80f6c7c5-c42b-4c88-b5c5-bf1de8a683d1\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-bab50d-58db4bc787-q2s49"
Sep 18 20:22:46.618692 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:22:46.618659    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-bab50d-58db4bc787-q2s49"
Sep 18 20:22:46.811226 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:22:46.811176    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:22:46.898827 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:22:46.898796    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-bab50d-58db4bc787-q2s49"]
Sep 18 20:22:46.901616 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:22:46.901586    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:22:46.902161 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:22:46.902138    2532 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod80f6c7c5_c42b_4c88_b5c5_bf1de8a683d1.slice/crio-3dd0967ddbc3617b023ce953619f0c59dbde9ae346665bb61e6bc3600a91d2ea WatchSource:0}: Error finding container 3dd0967ddbc3617b023ce953619f0c59dbde9ae346665bb61e6bc3600a91d2ea: Status 404 returned error can't find the container with id 3dd0967ddbc3617b023ce953619f0c59dbde9ae346665bb61e6bc3600a91d2ea
Sep 18 20:22:46.903589 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:22:46.903570    2532 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 18 20:22:46.980780 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:22:46.980748    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:22:47.081166 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:22:47.081083    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:22:47.257823 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:22:47.257787    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:22:47.652267 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:22:47.652226    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-bab50d-58db4bc787-q2s49" event={"ID":"80f6c7c5-c42b-4c88-b5c5-bf1de8a683d1","Type":"ContainerStarted","Data":"ec342752243625a2a80024be5d723a0cfcd9426a60f0c3a7be6c264d1327e166"}
Sep 18 20:22:47.652466 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:22:47.652282    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-bab50d-58db4bc787-q2s49" event={"ID":"80f6c7c5-c42b-4c88-b5c5-bf1de8a683d1","Type":"ContainerStarted","Data":"3dd0967ddbc3617b023ce953619f0c59dbde9ae346665bb61e6bc3600a91d2ea"}
Sep 18 20:22:47.652466 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:22:47.652304    2532 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-bab50d-58db4bc787-q2s49"
Sep 18 20:22:47.673211 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:22:47.673146    2532 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-bab50d-58db4bc787-q2s49" podStartSLOduration=1.3052851730000001 podStartE2EDuration="1.673131078s" totalImagesPullingTime="367.845905ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 20:22:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 20:22:47.669716935 +0000 UTC m=+1299.435106333" watchObservedRunningTime="2026-09-18 20:22:47.673131078 +0000 UTC m=+1299.438520525"
Sep 18 20:22:48.127782 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:22:48.127746    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:22:49.217216 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:22:49.217178    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:22:49.555862 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:22:49.555762    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:22:49.666788 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:22:49.666754    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:22:50.218739 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:22:50.218709    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:22:50.882553 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:22:50.882521    2532 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-9a5190-5ccb744cb6-drw2b"]
Sep 18 20:22:50.886574 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:22:50.886552    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-9a5190-5ccb744cb6-drw2b"
Sep 18 20:22:50.889273 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:22:50.889245    2532 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-w1-9a5190-serving-cert\""
Sep 18 20:22:50.900252 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:22:50.900230    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-9a5190-5ccb744cb6-drw2b"]
Sep 18 20:22:50.918626 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:22:50.918599    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:22:50.989708 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:22:50.989677    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ae7e4693-ee7e-4cf7-a545-c43fa542d35f-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w1-9a5190-5ccb744cb6-drw2b\" (UID: \"ae7e4693-ee7e-4cf7-a545-c43fa542d35f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-9a5190-5ccb744cb6-drw2b"
Sep 18 20:22:50.989908 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:22:50.989749    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ae7e4693-ee7e-4cf7-a545-c43fa542d35f-maas-api-tls\") pod \"maas-api-e2e-shared-b-w1-9a5190-5ccb744cb6-drw2b\" (UID: \"ae7e4693-ee7e-4cf7-a545-c43fa542d35f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-9a5190-5ccb744cb6-drw2b"
Sep 18 20:22:50.989908 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:22:50.989887    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ae7e4693-ee7e-4cf7-a545-c43fa542d35f-metrics-certs\") pod \"maas-api-e2e-shared-b-w1-9a5190-5ccb744cb6-drw2b\" (UID: \"ae7e4693-ee7e-4cf7-a545-c43fa542d35f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-9a5190-5ccb744cb6-drw2b"
Sep 18 20:22:50.990025 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:22:50.989943    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-86km4\" (UniqueName: \"kubernetes.io/projected/ae7e4693-ee7e-4cf7-a545-c43fa542d35f-kube-api-access-86km4\") pod \"maas-api-e2e-shared-b-w1-9a5190-5ccb744cb6-drw2b\" (UID: \"ae7e4693-ee7e-4cf7-a545-c43fa542d35f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-9a5190-5ccb744cb6-drw2b"
Sep 18 20:22:51.019902 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:22:51.019869    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:22:51.091206 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:22:51.091173    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ae7e4693-ee7e-4cf7-a545-c43fa542d35f-metrics-certs\") pod \"maas-api-e2e-shared-b-w1-9a5190-5ccb744cb6-drw2b\" (UID: \"ae7e4693-ee7e-4cf7-a545-c43fa542d35f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-9a5190-5ccb744cb6-drw2b"
Sep 18 20:22:51.091425 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:22:51.091221    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-86km4\" (UniqueName: \"kubernetes.io/projected/ae7e4693-ee7e-4cf7-a545-c43fa542d35f-kube-api-access-86km4\") pod \"maas-api-e2e-shared-b-w1-9a5190-5ccb744cb6-drw2b\" (UID: \"ae7e4693-ee7e-4cf7-a545-c43fa542d35f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-9a5190-5ccb744cb6-drw2b"
Sep 18 20:22:51.091425 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:22:51.091362    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ae7e4693-ee7e-4cf7-a545-c43fa542d35f-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w1-9a5190-5ccb744cb6-drw2b\" (UID: \"ae7e4693-ee7e-4cf7-a545-c43fa542d35f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-9a5190-5ccb744cb6-drw2b"
Sep 18 20:22:51.091560 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:22:51.091456    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ae7e4693-ee7e-4cf7-a545-c43fa542d35f-maas-api-tls\") pod \"maas-api-e2e-shared-b-w1-9a5190-5ccb744cb6-drw2b\" (UID: \"ae7e4693-ee7e-4cf7-a545-c43fa542d35f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-9a5190-5ccb744cb6-drw2b"
Sep 18 20:22:51.091628 ip-10-0-130-81 kubenswrapper[2532]: E0918 20:22:51.091608    2532 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-9a5190-serving-cert: secret "maas-api-e2e-shared-b-w1-9a5190-serving-cert" not found
Sep 18 20:22:51.091700 ip-10-0-130-81 kubenswrapper[2532]: E0918 20:22:51.091689    2532 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ae7e4693-ee7e-4cf7-a545-c43fa542d35f-maas-api-tls podName:ae7e4693-ee7e-4cf7-a545-c43fa542d35f nodeName:}" failed. No retries permitted until 2026-09-18 20:22:51.591672135 +0000 UTC m=+1303.357061511 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/ae7e4693-ee7e-4cf7-a545-c43fa542d35f-maas-api-tls") pod "maas-api-e2e-shared-b-w1-9a5190-5ccb744cb6-drw2b" (UID: "ae7e4693-ee7e-4cf7-a545-c43fa542d35f") : secret "maas-api-e2e-shared-b-w1-9a5190-serving-cert" not found
Sep 18 20:22:51.092072 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:22:51.092053    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ae7e4693-ee7e-4cf7-a545-c43fa542d35f-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w1-9a5190-5ccb744cb6-drw2b\" (UID: \"ae7e4693-ee7e-4cf7-a545-c43fa542d35f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-9a5190-5ccb744cb6-drw2b"
Sep 18 20:22:51.093837 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:22:51.093816    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ae7e4693-ee7e-4cf7-a545-c43fa542d35f-metrics-certs\") pod \"maas-api-e2e-shared-b-w1-9a5190-5ccb744cb6-drw2b\" (UID: \"ae7e4693-ee7e-4cf7-a545-c43fa542d35f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-9a5190-5ccb744cb6-drw2b"
Sep 18 20:22:51.099413 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:22:51.099393    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-86km4\" (UniqueName: \"kubernetes.io/projected/ae7e4693-ee7e-4cf7-a545-c43fa542d35f-kube-api-access-86km4\") pod \"maas-api-e2e-shared-b-w1-9a5190-5ccb744cb6-drw2b\" (UID: \"ae7e4693-ee7e-4cf7-a545-c43fa542d35f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-9a5190-5ccb744cb6-drw2b"
Sep 18 20:22:51.367083 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:22:51.367055    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:22:51.454064 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:22:51.454031    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:22:51.596647 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:22:51.596615    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ae7e4693-ee7e-4cf7-a545-c43fa542d35f-maas-api-tls\") pod \"maas-api-e2e-shared-b-w1-9a5190-5ccb744cb6-drw2b\" (UID: \"ae7e4693-ee7e-4cf7-a545-c43fa542d35f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-9a5190-5ccb744cb6-drw2b"
Sep 18 20:22:51.599029 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:22:51.599008    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ae7e4693-ee7e-4cf7-a545-c43fa542d35f-maas-api-tls\") pod \"maas-api-e2e-shared-b-w1-9a5190-5ccb744cb6-drw2b\" (UID: \"ae7e4693-ee7e-4cf7-a545-c43fa542d35f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-9a5190-5ccb744cb6-drw2b"
Sep 18 20:22:51.738749 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:22:51.738720    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:22:51.799331 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:22:51.799296    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-9a5190-5ccb744cb6-drw2b"
Sep 18 20:22:52.075345 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:22:52.075322    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-9a5190-5ccb744cb6-drw2b"]
Sep 18 20:22:52.077532 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:22:52.077504    2532 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podae7e4693_ee7e_4cf7_a545_c43fa542d35f.slice/crio-10167baf2402a593d5327db4ae00ea3aaef8811605e1cc5e7a79d4e473122cdc WatchSource:0}: Error finding container 10167baf2402a593d5327db4ae00ea3aaef8811605e1cc5e7a79d4e473122cdc: Status 404 returned error can't find the container with id 10167baf2402a593d5327db4ae00ea3aaef8811605e1cc5e7a79d4e473122cdc
Sep 18 20:22:52.433227 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:22:52.433135    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:22:52.681826 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:22:52.681790    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-9a5190-5ccb744cb6-drw2b" event={"ID":"ae7e4693-ee7e-4cf7-a545-c43fa542d35f","Type":"ContainerStarted","Data":"093a51b40286172019d8f08e674aabcc943f362ef7161074cbf8f6aed610e26d"}
Sep 18 20:22:52.681990 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:22:52.681851    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-9a5190-5ccb744cb6-drw2b" event={"ID":"ae7e4693-ee7e-4cf7-a545-c43fa542d35f","Type":"ContainerStarted","Data":"10167baf2402a593d5327db4ae00ea3aaef8811605e1cc5e7a79d4e473122cdc"}
Sep 18 20:22:52.681990 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:22:52.681873    2532 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-9a5190-5ccb744cb6-drw2b"
Sep 18 20:22:52.697890 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:22:52.697783    2532 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-9a5190-5ccb744cb6-drw2b" podStartSLOduration=2.296529421 podStartE2EDuration="2.697764636s" totalImagesPullingTime="401.235215ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 20:22:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 20:22:52.696831494 +0000 UTC m=+1304.462220894" watchObservedRunningTime="2026-09-18 20:22:52.697764636 +0000 UTC m=+1304.463154075"
Sep 18 20:22:53.082205 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:22:53.082175    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:22:53.664227 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:22:53.664198    2532 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-bab50d-58db4bc787-q2s49"
Sep 18 20:22:53.724613 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:22:53.724585    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:22:54.421371 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:22:54.421337    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:22:55.109093 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:22:55.109059    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:22:58.692752 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:22:58.692725    2532 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-9a5190-5ccb744cb6-drw2b"
Sep 18 20:23:07.266083 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:23:07.266047    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:23:08.106084 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:23:08.106027    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-vptdl"]
Sep 18 20:23:08.337770 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:23:08.337738    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:23:08.533790 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:23:08.533759    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:23:08.645025 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:23:08.644984    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:23:08.901420 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:23:08.901332    2532 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-w2-35558b-openshift-default-6bbb5bcc84-6gxxh"]
Sep 18 20:23:08.904822 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:23:08.904801    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w2-35558b-openshift-default-6bbb5bcc84-6gxxh"
Sep 18 20:23:08.907820 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:23:08.907781    2532 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Sep 18 20:23:08.907936 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:23:08.907898    2532 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-w2-35558b-openshift-default-dockercfg-p9ggd\""
Sep 18 20:23:08.925144 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:23:08.925115    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-35558b-openshift-default-6bbb5bcc84-6gxxh"]
Sep 18 20:23:08.935514 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:23:08.935465    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:23:08.965217 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:23:08.965182    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/66b1dac9-2d9a-4f89-9e49-17293b282e3a-istio-data\") pod \"e2e-shared-b-w2-35558b-openshift-default-6bbb5bcc84-6gxxh\" (UID: \"66b1dac9-2d9a-4f89-9e49-17293b282e3a\") " pod="openshift-ingress/e2e-shared-b-w2-35558b-openshift-default-6bbb5bcc84-6gxxh"
Sep 18 20:23:08.965402 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:23:08.965237    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/66b1dac9-2d9a-4f89-9e49-17293b282e3a-workload-socket\") pod \"e2e-shared-b-w2-35558b-openshift-default-6bbb5bcc84-6gxxh\" (UID: \"66b1dac9-2d9a-4f89-9e49-17293b282e3a\") " pod="openshift-ingress/e2e-shared-b-w2-35558b-openshift-default-6bbb5bcc84-6gxxh"
Sep 18 20:23:08.965402 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:23:08.965278    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/66b1dac9-2d9a-4f89-9e49-17293b282e3a-workload-certs\") pod \"e2e-shared-b-w2-35558b-openshift-default-6bbb5bcc84-6gxxh\" (UID: \"66b1dac9-2d9a-4f89-9e49-17293b282e3a\") " pod="openshift-ingress/e2e-shared-b-w2-35558b-openshift-default-6bbb5bcc84-6gxxh"
Sep 18 20:23:08.965402 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:23:08.965328    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/66b1dac9-2d9a-4f89-9e49-17293b282e3a-credential-socket\") pod \"e2e-shared-b-w2-35558b-openshift-default-6bbb5bcc84-6gxxh\" (UID: \"66b1dac9-2d9a-4f89-9e49-17293b282e3a\") " pod="openshift-ingress/e2e-shared-b-w2-35558b-openshift-default-6bbb5bcc84-6gxxh"
Sep 18 20:23:08.965402 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:23:08.965364    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/66b1dac9-2d9a-4f89-9e49-17293b282e3a-istiod-ca-cert\") pod \"e2e-shared-b-w2-35558b-openshift-default-6bbb5bcc84-6gxxh\" (UID: \"66b1dac9-2d9a-4f89-9e49-17293b282e3a\") " pod="openshift-ingress/e2e-shared-b-w2-35558b-openshift-default-6bbb5bcc84-6gxxh"
Sep 18 20:23:08.965651 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:23:08.965408    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kvnfm\" (UniqueName: \"kubernetes.io/projected/66b1dac9-2d9a-4f89-9e49-17293b282e3a-kube-api-access-kvnfm\") pod \"e2e-shared-b-w2-35558b-openshift-default-6bbb5bcc84-6gxxh\" (UID: \"66b1dac9-2d9a-4f89-9e49-17293b282e3a\") " pod="openshift-ingress/e2e-shared-b-w2-35558b-openshift-default-6bbb5bcc84-6gxxh"
Sep 18 20:23:08.965651 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:23:08.965469    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/66b1dac9-2d9a-4f89-9e49-17293b282e3a-istio-envoy\") pod \"e2e-shared-b-w2-35558b-openshift-default-6bbb5bcc84-6gxxh\" (UID: \"66b1dac9-2d9a-4f89-9e49-17293b282e3a\") " pod="openshift-ingress/e2e-shared-b-w2-35558b-openshift-default-6bbb5bcc84-6gxxh"
Sep 18 20:23:08.965651 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:23:08.965530    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/66b1dac9-2d9a-4f89-9e49-17293b282e3a-istio-token\") pod \"e2e-shared-b-w2-35558b-openshift-default-6bbb5bcc84-6gxxh\" (UID: \"66b1dac9-2d9a-4f89-9e49-17293b282e3a\") " pod="openshift-ingress/e2e-shared-b-w2-35558b-openshift-default-6bbb5bcc84-6gxxh"
Sep 18 20:23:08.965651 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:23:08.965560    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/66b1dac9-2d9a-4f89-9e49-17293b282e3a-istio-podinfo\") pod \"e2e-shared-b-w2-35558b-openshift-default-6bbb5bcc84-6gxxh\" (UID: \"66b1dac9-2d9a-4f89-9e49-17293b282e3a\") " pod="openshift-ingress/e2e-shared-b-w2-35558b-openshift-default-6bbb5bcc84-6gxxh"
Sep 18 20:23:09.066853 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:23:09.066813    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/66b1dac9-2d9a-4f89-9e49-17293b282e3a-credential-socket\") pod \"e2e-shared-b-w2-35558b-openshift-default-6bbb5bcc84-6gxxh\" (UID: \"66b1dac9-2d9a-4f89-9e49-17293b282e3a\") " pod="openshift-ingress/e2e-shared-b-w2-35558b-openshift-default-6bbb5bcc84-6gxxh"
Sep 18 20:23:09.067042 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:23:09.066860    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/66b1dac9-2d9a-4f89-9e49-17293b282e3a-istiod-ca-cert\") pod \"e2e-shared-b-w2-35558b-openshift-default-6bbb5bcc84-6gxxh\" (UID: \"66b1dac9-2d9a-4f89-9e49-17293b282e3a\") " pod="openshift-ingress/e2e-shared-b-w2-35558b-openshift-default-6bbb5bcc84-6gxxh"
Sep 18 20:23:09.067042 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:23:09.066898    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kvnfm\" (UniqueName: \"kubernetes.io/projected/66b1dac9-2d9a-4f89-9e49-17293b282e3a-kube-api-access-kvnfm\") pod \"e2e-shared-b-w2-35558b-openshift-default-6bbb5bcc84-6gxxh\" (UID: \"66b1dac9-2d9a-4f89-9e49-17293b282e3a\") " pod="openshift-ingress/e2e-shared-b-w2-35558b-openshift-default-6bbb5bcc84-6gxxh"
Sep 18 20:23:09.067042 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:23:09.066946    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/66b1dac9-2d9a-4f89-9e49-17293b282e3a-istio-envoy\") pod \"e2e-shared-b-w2-35558b-openshift-default-6bbb5bcc84-6gxxh\" (UID: \"66b1dac9-2d9a-4f89-9e49-17293b282e3a\") " pod="openshift-ingress/e2e-shared-b-w2-35558b-openshift-default-6bbb5bcc84-6gxxh"
Sep 18 20:23:09.067042 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:23:09.066973    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/66b1dac9-2d9a-4f89-9e49-17293b282e3a-istio-token\") pod \"e2e-shared-b-w2-35558b-openshift-default-6bbb5bcc84-6gxxh\" (UID: \"66b1dac9-2d9a-4f89-9e49-17293b282e3a\") " pod="openshift-ingress/e2e-shared-b-w2-35558b-openshift-default-6bbb5bcc84-6gxxh"
Sep 18 20:23:09.067281 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:23:09.067197    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/66b1dac9-2d9a-4f89-9e49-17293b282e3a-istio-podinfo\") pod \"e2e-shared-b-w2-35558b-openshift-default-6bbb5bcc84-6gxxh\" (UID: \"66b1dac9-2d9a-4f89-9e49-17293b282e3a\") " pod="openshift-ingress/e2e-shared-b-w2-35558b-openshift-default-6bbb5bcc84-6gxxh"
Sep 18 20:23:09.067350 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:23:09.067274    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/66b1dac9-2d9a-4f89-9e49-17293b282e3a-credential-socket\") pod \"e2e-shared-b-w2-35558b-openshift-default-6bbb5bcc84-6gxxh\" (UID: \"66b1dac9-2d9a-4f89-9e49-17293b282e3a\") " pod="openshift-ingress/e2e-shared-b-w2-35558b-openshift-default-6bbb5bcc84-6gxxh"
Sep 18 20:23:09.067350 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:23:09.067281    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/66b1dac9-2d9a-4f89-9e49-17293b282e3a-istio-data\") pod \"e2e-shared-b-w2-35558b-openshift-default-6bbb5bcc84-6gxxh\" (UID: \"66b1dac9-2d9a-4f89-9e49-17293b282e3a\") " pod="openshift-ingress/e2e-shared-b-w2-35558b-openshift-default-6bbb5bcc84-6gxxh"
Sep 18 20:23:09.067469 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:23:09.067356    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/66b1dac9-2d9a-4f89-9e49-17293b282e3a-workload-socket\") pod \"e2e-shared-b-w2-35558b-openshift-default-6bbb5bcc84-6gxxh\" (UID: \"66b1dac9-2d9a-4f89-9e49-17293b282e3a\") " pod="openshift-ingress/e2e-shared-b-w2-35558b-openshift-default-6bbb5bcc84-6gxxh"
Sep 18 20:23:09.067469 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:23:09.067403    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/66b1dac9-2d9a-4f89-9e49-17293b282e3a-workload-certs\") pod \"e2e-shared-b-w2-35558b-openshift-default-6bbb5bcc84-6gxxh\" (UID: \"66b1dac9-2d9a-4f89-9e49-17293b282e3a\") " pod="openshift-ingress/e2e-shared-b-w2-35558b-openshift-default-6bbb5bcc84-6gxxh"
Sep 18 20:23:09.067607 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:23:09.067578    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/66b1dac9-2d9a-4f89-9e49-17293b282e3a-istio-data\") pod \"e2e-shared-b-w2-35558b-openshift-default-6bbb5bcc84-6gxxh\" (UID: \"66b1dac9-2d9a-4f89-9e49-17293b282e3a\") " pod="openshift-ingress/e2e-shared-b-w2-35558b-openshift-default-6bbb5bcc84-6gxxh"
Sep 18 20:23:09.067789 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:23:09.067765    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/66b1dac9-2d9a-4f89-9e49-17293b282e3a-workload-socket\") pod \"e2e-shared-b-w2-35558b-openshift-default-6bbb5bcc84-6gxxh\" (UID: \"66b1dac9-2d9a-4f89-9e49-17293b282e3a\") " pod="openshift-ingress/e2e-shared-b-w2-35558b-openshift-default-6bbb5bcc84-6gxxh"
Sep 18 20:23:09.067835 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:23:09.067774    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/66b1dac9-2d9a-4f89-9e49-17293b282e3a-workload-certs\") pod \"e2e-shared-b-w2-35558b-openshift-default-6bbb5bcc84-6gxxh\" (UID: \"66b1dac9-2d9a-4f89-9e49-17293b282e3a\") " pod="openshift-ingress/e2e-shared-b-w2-35558b-openshift-default-6bbb5bcc84-6gxxh"
Sep 18 20:23:09.068064 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:23:09.068038    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/66b1dac9-2d9a-4f89-9e49-17293b282e3a-istiod-ca-cert\") pod \"e2e-shared-b-w2-35558b-openshift-default-6bbb5bcc84-6gxxh\" (UID: \"66b1dac9-2d9a-4f89-9e49-17293b282e3a\") " pod="openshift-ingress/e2e-shared-b-w2-35558b-openshift-default-6bbb5bcc84-6gxxh"
Sep 18 20:23:09.069480 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:23:09.069441    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/66b1dac9-2d9a-4f89-9e49-17293b282e3a-istio-envoy\") pod \"e2e-shared-b-w2-35558b-openshift-default-6bbb5bcc84-6gxxh\" (UID: \"66b1dac9-2d9a-4f89-9e49-17293b282e3a\") " pod="openshift-ingress/e2e-shared-b-w2-35558b-openshift-default-6bbb5bcc84-6gxxh"
Sep 18 20:23:09.069781 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:23:09.069762    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/66b1dac9-2d9a-4f89-9e49-17293b282e3a-istio-podinfo\") pod \"e2e-shared-b-w2-35558b-openshift-default-6bbb5bcc84-6gxxh\" (UID: \"66b1dac9-2d9a-4f89-9e49-17293b282e3a\") " pod="openshift-ingress/e2e-shared-b-w2-35558b-openshift-default-6bbb5bcc84-6gxxh"
Sep 18 20:23:09.074512 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:23:09.074466    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/66b1dac9-2d9a-4f89-9e49-17293b282e3a-istio-token\") pod \"e2e-shared-b-w2-35558b-openshift-default-6bbb5bcc84-6gxxh\" (UID: \"66b1dac9-2d9a-4f89-9e49-17293b282e3a\") " pod="openshift-ingress/e2e-shared-b-w2-35558b-openshift-default-6bbb5bcc84-6gxxh"
Sep 18 20:23:09.074652 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:23:09.074635    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kvnfm\" (UniqueName: \"kubernetes.io/projected/66b1dac9-2d9a-4f89-9e49-17293b282e3a-kube-api-access-kvnfm\") pod \"e2e-shared-b-w2-35558b-openshift-default-6bbb5bcc84-6gxxh\" (UID: \"66b1dac9-2d9a-4f89-9e49-17293b282e3a\") " pod="openshift-ingress/e2e-shared-b-w2-35558b-openshift-default-6bbb5bcc84-6gxxh"
Sep 18 20:23:09.152277 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:23:09.152186    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:23:09.216116 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:23:09.216075    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w2-35558b-openshift-default-6bbb5bcc84-6gxxh"
Sep 18 20:23:09.444241 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:23:09.444155    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:23:09.500849 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:23:09.500826    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-35558b-openshift-default-6bbb5bcc84-6gxxh"]
Sep 18 20:23:09.502667 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:23:09.502636    2532 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod66b1dac9_2d9a_4f89_9e49_17293b282e3a.slice/crio-33bab6861cd3a965f30ff2424562e563d943ddfe6884586cbd4de9c2b84b3eb3 WatchSource:0}: Error finding container 33bab6861cd3a965f30ff2424562e563d943ddfe6884586cbd4de9c2b84b3eb3: Status 404 returned error can't find the container with id 33bab6861cd3a965f30ff2424562e563d943ddfe6884586cbd4de9c2b84b3eb3
Sep 18 20:23:09.505090 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:23:09.505057    2532 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 18 20:23:09.505207 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:23:09.505145    2532 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 18 20:23:09.505207 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:23:09.505189    2532 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 18 20:23:09.782433 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:23:09.782398    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-35558b-openshift-default-6bbb5bcc84-6gxxh" event={"ID":"66b1dac9-2d9a-4f89-9e49-17293b282e3a","Type":"ContainerStarted","Data":"b341ce93ad3047427fbc641fdab61eacca106a3d9f4619ad84dc904f8ae38e82"}
Sep 18 20:23:09.782654 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:23:09.782463    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-35558b-openshift-default-6bbb5bcc84-6gxxh" event={"ID":"66b1dac9-2d9a-4f89-9e49-17293b282e3a","Type":"ContainerStarted","Data":"33bab6861cd3a965f30ff2424562e563d943ddfe6884586cbd4de9c2b84b3eb3"}
Sep 18 20:23:09.804731 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:23:09.804671    2532 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-w2-35558b-openshift-default-6bbb5bcc84-6gxxh" podStartSLOduration=1.8046500669999999 podStartE2EDuration="1.804650067s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 20:23:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 20:23:09.801894009 +0000 UTC m=+1321.567283462" watchObservedRunningTime="2026-09-18 20:23:09.804650067 +0000 UTC m=+1321.570039467"
Sep 18 20:23:10.119078 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:23:10.118968    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:23:10.217263 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:23:10.217225    2532 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w2-35558b-openshift-default-6bbb5bcc84-6gxxh"
Sep 18 20:23:10.222295 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:23:10.222268    2532 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w2-35558b-openshift-default-6bbb5bcc84-6gxxh"
Sep 18 20:23:10.787829 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:23:10.787802    2532 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w2-35558b-openshift-default-6bbb5bcc84-6gxxh"
Sep 18 20:23:10.788859 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:23:10.788836    2532 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w2-35558b-openshift-default-6bbb5bcc84-6gxxh"
Sep 18 20:23:11.082112 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:23:11.082016    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:23:11.715881 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:23:11.715851    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:23:11.811872 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:23:11.811841    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:23:12.692273 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:23:12.692243    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:23:13.421463 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:23:13.421431    2532 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-35558b-646544f8fb-cnxv2"]
Sep 18 20:23:13.426891 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:23:13.426865    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-35558b-646544f8fb-cnxv2"
Sep 18 20:23:13.430109 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:23:13.429447    2532 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-w2-35558b-serving-cert\""
Sep 18 20:23:13.433455 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:23:13.433429    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-35558b-646544f8fb-cnxv2"]
Sep 18 20:23:13.446842 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:23:13.446818    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:23:13.515069 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:23:13.515037    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e919b8e7-fb73-484a-9992-72991abbeb4d-metrics-certs\") pod \"maas-api-e2e-shared-b-w2-35558b-646544f8fb-cnxv2\" (UID: \"e919b8e7-fb73-484a-9992-72991abbeb4d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-35558b-646544f8fb-cnxv2"
Sep 18 20:23:13.515264 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:23:13.515085    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e919b8e7-fb73-484a-9992-72991abbeb4d-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-35558b-646544f8fb-cnxv2\" (UID: \"e919b8e7-fb73-484a-9992-72991abbeb4d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-35558b-646544f8fb-cnxv2"
Sep 18 20:23:13.515264 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:23:13.515156    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e919b8e7-fb73-484a-9992-72991abbeb4d-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w2-35558b-646544f8fb-cnxv2\" (UID: \"e919b8e7-fb73-484a-9992-72991abbeb4d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-35558b-646544f8fb-cnxv2"
Sep 18 20:23:13.515264 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:23:13.515177    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c7txq\" (UniqueName: \"kubernetes.io/projected/e919b8e7-fb73-484a-9992-72991abbeb4d-kube-api-access-c7txq\") pod \"maas-api-e2e-shared-b-w2-35558b-646544f8fb-cnxv2\" (UID: \"e919b8e7-fb73-484a-9992-72991abbeb4d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-35558b-646544f8fb-cnxv2"
Sep 18 20:23:13.548391 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:23:13.548360    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:23:13.616338 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:23:13.616302    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e919b8e7-fb73-484a-9992-72991abbeb4d-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w2-35558b-646544f8fb-cnxv2\" (UID: \"e919b8e7-fb73-484a-9992-72991abbeb4d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-35558b-646544f8fb-cnxv2"
Sep 18 20:23:13.616338 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:23:13.616344    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c7txq\" (UniqueName: \"kubernetes.io/projected/e919b8e7-fb73-484a-9992-72991abbeb4d-kube-api-access-c7txq\") pod \"maas-api-e2e-shared-b-w2-35558b-646544f8fb-cnxv2\" (UID: \"e919b8e7-fb73-484a-9992-72991abbeb4d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-35558b-646544f8fb-cnxv2"
Sep 18 20:23:13.616598 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:23:13.616428    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e919b8e7-fb73-484a-9992-72991abbeb4d-metrics-certs\") pod \"maas-api-e2e-shared-b-w2-35558b-646544f8fb-cnxv2\" (UID: \"e919b8e7-fb73-484a-9992-72991abbeb4d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-35558b-646544f8fb-cnxv2"
Sep 18 20:23:13.616598 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:23:13.616504    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e919b8e7-fb73-484a-9992-72991abbeb4d-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-35558b-646544f8fb-cnxv2\" (UID: \"e919b8e7-fb73-484a-9992-72991abbeb4d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-35558b-646544f8fb-cnxv2"
Sep 18 20:23:13.617070 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:23:13.617053    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e919b8e7-fb73-484a-9992-72991abbeb4d-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w2-35558b-646544f8fb-cnxv2\" (UID: \"e919b8e7-fb73-484a-9992-72991abbeb4d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-35558b-646544f8fb-cnxv2"
Sep 18 20:23:13.618994 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:23:13.618966    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e919b8e7-fb73-484a-9992-72991abbeb4d-metrics-certs\") pod \"maas-api-e2e-shared-b-w2-35558b-646544f8fb-cnxv2\" (UID: \"e919b8e7-fb73-484a-9992-72991abbeb4d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-35558b-646544f8fb-cnxv2"
Sep 18 20:23:13.619106 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:23:13.619013    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e919b8e7-fb73-484a-9992-72991abbeb4d-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-35558b-646544f8fb-cnxv2\" (UID: \"e919b8e7-fb73-484a-9992-72991abbeb4d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-35558b-646544f8fb-cnxv2"
Sep 18 20:23:13.623726 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:23:13.623701    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c7txq\" (UniqueName: \"kubernetes.io/projected/e919b8e7-fb73-484a-9992-72991abbeb4d-kube-api-access-c7txq\") pod \"maas-api-e2e-shared-b-w2-35558b-646544f8fb-cnxv2\" (UID: \"e919b8e7-fb73-484a-9992-72991abbeb4d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-35558b-646544f8fb-cnxv2"
Sep 18 20:23:13.646012 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:23:13.645984    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:23:13.741133 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:23:13.741105    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-35558b-646544f8fb-cnxv2"
Sep 18 20:23:14.015584 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:23:14.014112    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-35558b-646544f8fb-cnxv2"]
Sep 18 20:23:14.018580 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:23:14.018554    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:23:14.076910 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:23:14.076883    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:23:14.418918 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:23:14.418836    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:23:14.519056 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:23:14.519025    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:23:14.814179 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:23:14.814139    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-35558b-646544f8fb-cnxv2" event={"ID":"e919b8e7-fb73-484a-9992-72991abbeb4d","Type":"ContainerStarted","Data":"a7a138cca229cd21aecea07f43a84a3b0c5876cfda3ff1d2c3321feb385a7b52"}
Sep 18 20:23:14.814359 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:23:14.814210    2532 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-35558b-646544f8fb-cnxv2"
Sep 18 20:23:14.814359 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:23:14.814232    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-35558b-646544f8fb-cnxv2" event={"ID":"e919b8e7-fb73-484a-9992-72991abbeb4d","Type":"ContainerStarted","Data":"5dfc41ca8a9c28ef1588ce2f7f842bfbaf94b336d4b4638cc6b4a868be10415b"}
Sep 18 20:23:14.837135 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:23:14.837088    2532 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-35558b-646544f8fb-cnxv2" podStartSLOduration=1.302700194 podStartE2EDuration="1.837072413s" totalImagesPullingTime="534.372219ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 20:23:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 20:23:14.836187405 +0000 UTC m=+1326.601576807" watchObservedRunningTime="2026-09-18 20:23:14.837072413 +0000 UTC m=+1326.602461812"
Sep 18 20:23:15.057885 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:23:15.057844    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:23:15.110296 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:23:15.110222    2532 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-8716-6da3a4"]
Sep 18 20:23:15.113657 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:23:15.113639    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-8716-6da3a4"
Sep 18 20:23:15.116632 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:23:15.116608    2532 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-pqc57\""
Sep 18 20:23:15.121320 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:23:15.121297    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-8716-6da3a4"]
Sep 18 20:23:15.234824 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:23:15.234780    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z76k5\" (UniqueName: \"kubernetes.io/projected/9b756649-dd7e-4062-9ee2-2ac723b1edce-kube-api-access-z76k5\") pod \"test-curl-8716-6da3a4\" (UID: \"9b756649-dd7e-4062-9ee2-2ac723b1edce\") " pod="opendatahub/test-curl-8716-6da3a4"
Sep 18 20:23:15.335827 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:23:15.335785    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z76k5\" (UniqueName: \"kubernetes.io/projected/9b756649-dd7e-4062-9ee2-2ac723b1edce-kube-api-access-z76k5\") pod \"test-curl-8716-6da3a4\" (UID: \"9b756649-dd7e-4062-9ee2-2ac723b1edce\") " pod="opendatahub/test-curl-8716-6da3a4"
Sep 18 20:23:15.343548 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:23:15.343512    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z76k5\" (UniqueName: \"kubernetes.io/projected/9b756649-dd7e-4062-9ee2-2ac723b1edce-kube-api-access-z76k5\") pod \"test-curl-8716-6da3a4\" (UID: \"9b756649-dd7e-4062-9ee2-2ac723b1edce\") " pod="opendatahub/test-curl-8716-6da3a4"
Sep 18 20:23:15.436023 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:23:15.435937    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-8716-6da3a4"
Sep 18 20:23:15.703612 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:23:15.703589    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-8716-6da3a4"]
Sep 18 20:23:15.705374 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:23:15.705345    2532 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9b756649_dd7e_4062_9ee2_2ac723b1edce.slice/crio-b8115c4bf421635540be66323b32748402eb78e6348a9564230fd6c5244a8e45 WatchSource:0}: Error finding container b8115c4bf421635540be66323b32748402eb78e6348a9564230fd6c5244a8e45: Status 404 returned error can't find the container with id b8115c4bf421635540be66323b32748402eb78e6348a9564230fd6c5244a8e45
Sep 18 20:23:15.812666 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:23:15.812635    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:23:15.820832 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:23:15.820801    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-8716-6da3a4" event={"ID":"9b756649-dd7e-4062-9ee2-2ac723b1edce","Type":"ContainerStarted","Data":"7d9f2ca755b9b8a72be57d0b2066ad45508e9913fae5a41780b4cf9e60f4c229"}
Sep 18 20:23:15.820988 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:23:15.820846    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-8716-6da3a4" event={"ID":"9b756649-dd7e-4062-9ee2-2ac723b1edce","Type":"ContainerStarted","Data":"b8115c4bf421635540be66323b32748402eb78e6348a9564230fd6c5244a8e45"}
Sep 18 20:23:15.837691 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:23:15.837646    2532 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-8716-6da3a4" podStartSLOduration=0.837630842 podStartE2EDuration="837.630842ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 20:23:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 20:23:15.835659674 +0000 UTC m=+1327.601049073" watchObservedRunningTime="2026-09-18 20:23:15.837630842 +0000 UTC m=+1327.603020240"
Sep 18 20:23:16.397085 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:23:16.396993    2532 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-8716-6da3a4"]
Sep 18 20:23:16.401027 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:23:16.400989    2532 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-8716-6da3a4"]
Sep 18 20:23:16.515034 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:23:16.514998    2532 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-8716-66411d"]
Sep 18 20:23:16.515599 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:23:16.515582    2532 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9b756649-dd7e-4062-9ee2-2ac723b1edce" containerName="test-curl-8716-6da3a4"
Sep 18 20:23:16.515661 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:23:16.515601    2532 state_mem.go:107] "Deleted CPUSet assignment" podUID="9b756649-dd7e-4062-9ee2-2ac723b1edce" containerName="test-curl-8716-6da3a4"
Sep 18 20:23:16.515698 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:23:16.515671    2532 memory_manager.go:368] "RemoveStaleState removing state" podUID="9b756649-dd7e-4062-9ee2-2ac723b1edce" containerName="test-curl-8716-6da3a4"
Sep 18 20:23:16.518782 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:23:16.518767    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-8716-66411d"
Sep 18 20:23:16.525264 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:23:16.525235    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-8716-66411d"]
Sep 18 20:23:16.597700 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:23:16.597667    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:23:16.651044 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:23:16.650957    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l8gtr\" (UniqueName: \"kubernetes.io/projected/fea8b5bd-7b22-42f2-a844-33eda7c0fdef-kube-api-access-l8gtr\") pod \"test-curl-8716-66411d\" (UID: \"fea8b5bd-7b22-42f2-a844-33eda7c0fdef\") " pod="opendatahub/test-curl-8716-66411d"
Sep 18 20:23:16.751987 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:23:16.751955    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l8gtr\" (UniqueName: \"kubernetes.io/projected/fea8b5bd-7b22-42f2-a844-33eda7c0fdef-kube-api-access-l8gtr\") pod \"test-curl-8716-66411d\" (UID: \"fea8b5bd-7b22-42f2-a844-33eda7c0fdef\") " pod="opendatahub/test-curl-8716-66411d"
Sep 18 20:23:16.759303 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:23:16.759283    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l8gtr\" (UniqueName: \"kubernetes.io/projected/fea8b5bd-7b22-42f2-a844-33eda7c0fdef-kube-api-access-l8gtr\") pod \"test-curl-8716-66411d\" (UID: \"fea8b5bd-7b22-42f2-a844-33eda7c0fdef\") " pod="opendatahub/test-curl-8716-66411d"
Sep 18 20:23:16.826714 ip-10-0-130-81 kubenswrapper[2532]: E0918 20:23:16.826680    2532 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-8716-6da3a4" container="test-curl-8716-6da3a4"
Sep 18 20:23:16.830391 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:23:16.830375    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-8716-66411d"
Sep 18 20:23:17.102252 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:23:17.102221    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-8716-66411d"]
Sep 18 20:23:17.103374 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:23:17.103346    2532 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podfea8b5bd_7b22_42f2_a844_33eda7c0fdef.slice/crio-b90c64e12fb7a18a4d90d55dac12ceff8be3d4b66102ddb66a80e2682e03538b WatchSource:0}: Error finding container b90c64e12fb7a18a4d90d55dac12ceff8be3d4b66102ddb66a80e2682e03538b: Status 404 returned error can't find the container with id b90c64e12fb7a18a4d90d55dac12ceff8be3d4b66102ddb66a80e2682e03538b
Sep 18 20:23:17.313329 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:23:17.313300    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:23:17.832952 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:23:17.832901    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-8716-66411d" event={"ID":"fea8b5bd-7b22-42f2-a844-33eda7c0fdef","Type":"ContainerStarted","Data":"54d91fa23e988652c7487bfa157be38004d235b918c9203aec1fb564dc0d67d0"}
Sep 18 20:23:17.832952 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:23:17.832957    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-8716-66411d" event={"ID":"fea8b5bd-7b22-42f2-a844-33eda7c0fdef","Type":"ContainerStarted","Data":"b90c64e12fb7a18a4d90d55dac12ceff8be3d4b66102ddb66a80e2682e03538b"}
Sep 18 20:23:17.833457 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:23:17.832969    2532 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-8716-6da3a4" podUID="9b756649-dd7e-4062-9ee2-2ac723b1edce" containerName="test-curl-8716-6da3a4" containerID="cri-o://7d9f2ca755b9b8a72be57d0b2066ad45508e9913fae5a41780b4cf9e60f4c229" gracePeriod=2
Sep 18 20:23:17.835177 ip-10-0-130-81 kubenswrapper[2532]: E0918 20:23:17.835150    2532 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-8716-6da3a4\" is forbidden: User \"system:node:ip-10-0-130-81.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-130-81.ec2.internal' and this object" podUID="9b756649-dd7e-4062-9ee2-2ac723b1edce" pod="opendatahub/test-curl-8716-6da3a4"
Sep 18 20:23:17.848480 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:23:17.848436    2532 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-8716-66411d" podStartSLOduration=1.848423748 podStartE2EDuration="1.848423748s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 20:23:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 20:23:17.847437832 +0000 UTC m=+1329.612827235" watchObservedRunningTime="2026-09-18 20:23:17.848423748 +0000 UTC m=+1329.613813148"
Sep 18 20:23:17.849520 ip-10-0-130-81 kubenswrapper[2532]: E0918 20:23:17.849475    2532 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-8716-6da3a4\" is forbidden: User \"system:node:ip-10-0-130-81.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-130-81.ec2.internal' and this object" podUID="9b756649-dd7e-4062-9ee2-2ac723b1edce" pod="opendatahub/test-curl-8716-6da3a4"
Sep 18 20:23:18.008312 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:23:18.008283    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:23:18.467631 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:23:18.467597    2532 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-8716-66411d"]
Sep 18 20:23:18.473352 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:23:18.473324    2532 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-8716-66411d"]
Sep 18 20:23:18.583479 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:23:18.583398    2532 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-8716-399bad"]
Sep 18 20:23:18.583803 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:23:18.583790    2532 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fea8b5bd-7b22-42f2-a844-33eda7c0fdef" containerName="test-curl-8716-66411d"
Sep 18 20:23:18.583845 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:23:18.583805    2532 state_mem.go:107] "Deleted CPUSet assignment" podUID="fea8b5bd-7b22-42f2-a844-33eda7c0fdef" containerName="test-curl-8716-66411d"
Sep 18 20:23:18.583885 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:23:18.583876    2532 memory_manager.go:368] "RemoveStaleState removing state" podUID="fea8b5bd-7b22-42f2-a844-33eda7c0fdef" containerName="test-curl-8716-66411d"
Sep 18 20:23:18.587018 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:23:18.587001    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-8716-399bad"
Sep 18 20:23:18.592883 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:23:18.592858    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-8716-399bad"]
Sep 18 20:23:18.672682 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:23:18.672641    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bfgqg\" (UniqueName: \"kubernetes.io/projected/fc2c4bf4-0218-44c2-b411-e1c11b92aaf8-kube-api-access-bfgqg\") pod \"test-curl-8716-399bad\" (UID: \"fc2c4bf4-0218-44c2-b411-e1c11b92aaf8\") " pod="opendatahub/test-curl-8716-399bad"
Sep 18 20:23:18.682732 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:23:18.682701    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:23:18.774040 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:23:18.774004    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bfgqg\" (UniqueName: \"kubernetes.io/projected/fc2c4bf4-0218-44c2-b411-e1c11b92aaf8-kube-api-access-bfgqg\") pod \"test-curl-8716-399bad\" (UID: \"fc2c4bf4-0218-44c2-b411-e1c11b92aaf8\") " pod="opendatahub/test-curl-8716-399bad"
Sep 18 20:23:18.781884 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:23:18.781858    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bfgqg\" (UniqueName: \"kubernetes.io/projected/fc2c4bf4-0218-44c2-b411-e1c11b92aaf8-kube-api-access-bfgqg\") pod \"test-curl-8716-399bad\" (UID: \"fc2c4bf4-0218-44c2-b411-e1c11b92aaf8\") " pod="opendatahub/test-curl-8716-399bad"
Sep 18 20:23:18.839372 ip-10-0-130-81 kubenswrapper[2532]: E0918 20:23:18.839296    2532 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-8716-66411d" container="test-curl-8716-66411d"
Sep 18 20:23:18.897666 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:23:18.897629    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-8716-399bad"
Sep 18 20:23:19.168378 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:23:19.168353    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-8716-399bad"]
Sep 18 20:23:19.169732 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:23:19.169700    2532 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podfc2c4bf4_0218_44c2_b411_e1c11b92aaf8.slice/crio-6c5ea33aa9768a3841068bf10a236538f56ae3beb59afc7496e2c7d1d0ad19f5 WatchSource:0}: Error finding container 6c5ea33aa9768a3841068bf10a236538f56ae3beb59afc7496e2c7d1d0ad19f5: Status 404 returned error can't find the container with id 6c5ea33aa9768a3841068bf10a236538f56ae3beb59afc7496e2c7d1d0ad19f5
Sep 18 20:23:19.845698 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:23:19.845665    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-8716-399bad" event={"ID":"fc2c4bf4-0218-44c2-b411-e1c11b92aaf8","Type":"ContainerStarted","Data":"66f56f46c03fa7dc7f19b28dc4d44b510b38dbcbbe1727ad3964619b767ba3b1"}
Sep 18 20:23:19.846029 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:23:19.845711    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-8716-399bad" event={"ID":"fc2c4bf4-0218-44c2-b411-e1c11b92aaf8","Type":"ContainerStarted","Data":"6c5ea33aa9768a3841068bf10a236538f56ae3beb59afc7496e2c7d1d0ad19f5"}
Sep 18 20:23:19.846029 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:23:19.845720    2532 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-8716-66411d" podUID="fea8b5bd-7b22-42f2-a844-33eda7c0fdef" containerName="test-curl-8716-66411d" containerID="cri-o://54d91fa23e988652c7487bfa157be38004d235b918c9203aec1fb564dc0d67d0" gracePeriod=2
Sep 18 20:23:19.860876 ip-10-0-130-81 kubenswrapper[2532]: E0918 20:23:19.860840    2532 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-8716-66411d\" is forbidden: User \"system:node:ip-10-0-130-81.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-130-81.ec2.internal' and this object" podUID="fea8b5bd-7b22-42f2-a844-33eda7c0fdef" pod="opendatahub/test-curl-8716-66411d"
Sep 18 20:23:19.861370 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:23:19.861328    2532 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-8716-399bad" podStartSLOduration=1.861313253 podStartE2EDuration="1.861313253s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 20:23:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 20:23:19.858696075 +0000 UTC m=+1331.624085470" watchObservedRunningTime="2026-09-18 20:23:19.861313253 +0000 UTC m=+1331.626702652"
Sep 18 20:23:19.862742 ip-10-0-130-81 kubenswrapper[2532]: E0918 20:23:19.862720    2532 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-8716-66411d\" is forbidden: User \"system:node:ip-10-0-130-81.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-130-81.ec2.internal' and this object" podUID="fea8b5bd-7b22-42f2-a844-33eda7c0fdef" pod="opendatahub/test-curl-8716-66411d"
Sep 18 20:23:20.023538 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:23:20.023516    2532 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-8716-6da3a4"
Sep 18 20:23:20.025642 ip-10-0-130-81 kubenswrapper[2532]: E0918 20:23:20.025616    2532 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-8716-66411d\" is forbidden: User \"system:node:ip-10-0-130-81.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-130-81.ec2.internal' and this object" podUID="fea8b5bd-7b22-42f2-a844-33eda7c0fdef" pod="opendatahub/test-curl-8716-66411d"
Sep 18 20:23:20.188961 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:23:20.188866    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z76k5\" (UniqueName: \"kubernetes.io/projected/9b756649-dd7e-4062-9ee2-2ac723b1edce-kube-api-access-z76k5\") pod \"9b756649-dd7e-4062-9ee2-2ac723b1edce\" (UID: \"9b756649-dd7e-4062-9ee2-2ac723b1edce\") "
Sep 18 20:23:20.190975 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:23:20.190948    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9b756649-dd7e-4062-9ee2-2ac723b1edce-kube-api-access-z76k5" (OuterVolumeSpecName: "kube-api-access-z76k5") pod "9b756649-dd7e-4062-9ee2-2ac723b1edce" (UID: "9b756649-dd7e-4062-9ee2-2ac723b1edce"). InnerVolumeSpecName "kube-api-access-z76k5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 20:23:20.290031 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:23:20.290003    2532 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z76k5\" (UniqueName: \"kubernetes.io/projected/9b756649-dd7e-4062-9ee2-2ac723b1edce-kube-api-access-z76k5\") on node \"ip-10-0-130-81.ec2.internal\" DevicePath \"\""
Sep 18 20:23:20.305239 ip-10-0-130-81 kubenswrapper[2532]: E0918 20:23:20.305195    2532 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-8716-66411d\" is forbidden: User \"system:node:ip-10-0-130-81.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-130-81.ec2.internal' and this object" podUID="fea8b5bd-7b22-42f2-a844-33eda7c0fdef" pod="opendatahub/test-curl-8716-66411d"
Sep 18 20:23:20.307128 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:23:20.307099    2532 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9b756649-dd7e-4062-9ee2-2ac723b1edce" path="/var/lib/kubelet/pods/9b756649-dd7e-4062-9ee2-2ac723b1edce/volumes"
Sep 18 20:23:20.406813 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:23:20.406782    2532 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-8716-399bad"]
Sep 18 20:23:20.410566 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:23:20.410538    2532 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-8716-399bad"]
Sep 18 20:23:20.517816 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:23:20.517782    2532 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-8716-8f3c88"]
Sep 18 20:23:20.518163 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:23:20.518151    2532 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fc2c4bf4-0218-44c2-b411-e1c11b92aaf8" containerName="test-curl-8716-399bad"
Sep 18 20:23:20.518208 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:23:20.518165    2532 state_mem.go:107] "Deleted CPUSet assignment" podUID="fc2c4bf4-0218-44c2-b411-e1c11b92aaf8" containerName="test-curl-8716-399bad"
Sep 18 20:23:20.518251 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:23:20.518243    2532 memory_manager.go:368] "RemoveStaleState removing state" podUID="fc2c4bf4-0218-44c2-b411-e1c11b92aaf8" containerName="test-curl-8716-399bad"
Sep 18 20:23:20.521596 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:23:20.521575    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-8716-8f3c88"
Sep 18 20:23:20.526604 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:23:20.526584    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-8716-8f3c88"]
Sep 18 20:23:20.592640 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:23:20.592610    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dgn9g\" (UniqueName: \"kubernetes.io/projected/00e220c9-8f3e-4926-b72b-655f14e9846c-kube-api-access-dgn9g\") pod \"test-curl-8716-8f3c88\" (UID: \"00e220c9-8f3e-4926-b72b-655f14e9846c\") " pod="opendatahub/test-curl-8716-8f3c88"
Sep 18 20:23:20.693452 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:23:20.693413    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dgn9g\" (UniqueName: \"kubernetes.io/projected/00e220c9-8f3e-4926-b72b-655f14e9846c-kube-api-access-dgn9g\") pod \"test-curl-8716-8f3c88\" (UID: \"00e220c9-8f3e-4926-b72b-655f14e9846c\") " pod="opendatahub/test-curl-8716-8f3c88"
Sep 18 20:23:20.701141 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:23:20.701111    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dgn9g\" (UniqueName: \"kubernetes.io/projected/00e220c9-8f3e-4926-b72b-655f14e9846c-kube-api-access-dgn9g\") pod \"test-curl-8716-8f3c88\" (UID: \"00e220c9-8f3e-4926-b72b-655f14e9846c\") " pod="opendatahub/test-curl-8716-8f3c88"
Sep 18 20:23:20.826980 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:23:20.826896    2532 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-35558b-646544f8fb-cnxv2"
Sep 18 20:23:20.832317 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:23:20.832299    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-8716-8f3c88"
Sep 18 20:23:20.856350 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:23:20.856314    2532 generic.go:356] "Generic (PLEG): container finished" podID="9b756649-dd7e-4062-9ee2-2ac723b1edce" containerID="7d9f2ca755b9b8a72be57d0b2066ad45508e9913fae5a41780b4cf9e60f4c229" exitCode=137
Sep 18 20:23:20.856782 ip-10-0-130-81 kubenswrapper[2532]: E0918 20:23:20.856684    2532 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-8716-399bad" container="test-curl-8716-399bad"
Sep 18 20:23:20.856940 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:23:20.856926    2532 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-8716-6da3a4"
Sep 18 20:23:20.857030 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:23:20.856961    2532 scope.go:117] "RemoveContainer" containerID="7d9f2ca755b9b8a72be57d0b2066ad45508e9913fae5a41780b4cf9e60f4c229"
Sep 18 20:23:20.870634 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:23:20.870604    2532 scope.go:117] "RemoveContainer" containerID="7d9f2ca755b9b8a72be57d0b2066ad45508e9913fae5a41780b4cf9e60f4c229"
Sep 18 20:23:20.871025 ip-10-0-130-81 kubenswrapper[2532]: E0918 20:23:20.870996    2532 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7d9f2ca755b9b8a72be57d0b2066ad45508e9913fae5a41780b4cf9e60f4c229\": container with ID starting with 7d9f2ca755b9b8a72be57d0b2066ad45508e9913fae5a41780b4cf9e60f4c229 not found: ID does not exist" containerID="7d9f2ca755b9b8a72be57d0b2066ad45508e9913fae5a41780b4cf9e60f4c229"
Sep 18 20:23:20.871112 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:23:20.871037    2532 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7d9f2ca755b9b8a72be57d0b2066ad45508e9913fae5a41780b4cf9e60f4c229"} err="failed to get container status \"7d9f2ca755b9b8a72be57d0b2066ad45508e9913fae5a41780b4cf9e60f4c229\": rpc error: code = NotFound desc = could not find container \"7d9f2ca755b9b8a72be57d0b2066ad45508e9913fae5a41780b4cf9e60f4c229\": container with ID starting with 7d9f2ca755b9b8a72be57d0b2066ad45508e9913fae5a41780b4cf9e60f4c229 not found: ID does not exist"
Sep 18 20:23:21.110981 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:23:21.110953    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-8716-8f3c88"]
Sep 18 20:23:21.112621 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:23:21.112597    2532 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod00e220c9_8f3e_4926_b72b_655f14e9846c.slice/crio-fd28232a8787215643eca14bd11dd4aeff8acfee987efef8577419ea8eedf3f9 WatchSource:0}: Error finding container fd28232a8787215643eca14bd11dd4aeff8acfee987efef8577419ea8eedf3f9: Status 404 returned error can't find the container with id fd28232a8787215643eca14bd11dd4aeff8acfee987efef8577419ea8eedf3f9
Sep 18 20:23:21.864199 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:23:21.864173    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-8716-8f3c88" event={"ID":"00e220c9-8f3e-4926-b72b-655f14e9846c","Type":"ContainerStarted","Data":"923c84c54acf50c9a5ee889203eb91af26f5533edae61ec179bee2191af28318"}
Sep 18 20:23:21.864595 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:23:21.864211    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-8716-8f3c88" event={"ID":"00e220c9-8f3e-4926-b72b-655f14e9846c","Type":"ContainerStarted","Data":"fd28232a8787215643eca14bd11dd4aeff8acfee987efef8577419ea8eedf3f9"}
Sep 18 20:23:21.871978 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:23:21.871948    2532 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-8716-399bad" podUID="fc2c4bf4-0218-44c2-b411-e1c11b92aaf8" containerName="test-curl-8716-399bad" containerID="cri-o://66f56f46c03fa7dc7f19b28dc4d44b510b38dbcbbe1727ad3964619b767ba3b1" gracePeriod=2
Sep 18 20:23:21.883865 ip-10-0-130-81 kubenswrapper[2532]: E0918 20:23:21.883833    2532 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-8716-399bad\" is forbidden: User \"system:node:ip-10-0-130-81.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-130-81.ec2.internal' and this object" podUID="fc2c4bf4-0218-44c2-b411-e1c11b92aaf8" pod="opendatahub/test-curl-8716-399bad"
Sep 18 20:23:21.884694 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:23:21.884654    2532 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-8716-8f3c88" podStartSLOduration=1.8846396749999998 podStartE2EDuration="1.884639675s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 20:23:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 20:23:21.881854017 +0000 UTC m=+1333.647243417" watchObservedRunningTime="2026-09-18 20:23:21.884639675 +0000 UTC m=+1333.650029075"
Sep 18 20:23:22.034280 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:23:22.034252    2532 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-8716-66411d"
Sep 18 20:23:22.036466 ip-10-0-130-81 kubenswrapper[2532]: E0918 20:23:22.036436    2532 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-8716-399bad\" is forbidden: User \"system:node:ip-10-0-130-81.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-130-81.ec2.internal' and this object" podUID="fc2c4bf4-0218-44c2-b411-e1c11b92aaf8" pod="opendatahub/test-curl-8716-399bad"
Sep 18 20:23:22.106685 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:23:22.106605    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-l8gtr\" (UniqueName: \"kubernetes.io/projected/fea8b5bd-7b22-42f2-a844-33eda7c0fdef-kube-api-access-l8gtr\") pod \"fea8b5bd-7b22-42f2-a844-33eda7c0fdef\" (UID: \"fea8b5bd-7b22-42f2-a844-33eda7c0fdef\") "
Sep 18 20:23:22.108520 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:23:22.108496    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fea8b5bd-7b22-42f2-a844-33eda7c0fdef-kube-api-access-l8gtr" (OuterVolumeSpecName: "kube-api-access-l8gtr") pod "fea8b5bd-7b22-42f2-a844-33eda7c0fdef" (UID: "fea8b5bd-7b22-42f2-a844-33eda7c0fdef"). InnerVolumeSpecName "kube-api-access-l8gtr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 20:23:22.207339 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:23:22.207306    2532 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l8gtr\" (UniqueName: \"kubernetes.io/projected/fea8b5bd-7b22-42f2-a844-33eda7c0fdef-kube-api-access-l8gtr\") on node \"ip-10-0-130-81.ec2.internal\" DevicePath \"\""
Sep 18 20:23:22.307526 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:23:22.307433    2532 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fea8b5bd-7b22-42f2-a844-33eda7c0fdef" path="/var/lib/kubelet/pods/fea8b5bd-7b22-42f2-a844-33eda7c0fdef/volumes"
Sep 18 20:23:22.428579 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:23:22.428459    2532 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-8716-8f3c88"]
Sep 18 20:23:22.431420 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:23:22.431395    2532 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-8716-8f3c88"]
Sep 18 20:23:22.878439 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:23:22.878403    2532 generic.go:356] "Generic (PLEG): container finished" podID="fea8b5bd-7b22-42f2-a844-33eda7c0fdef" containerID="54d91fa23e988652c7487bfa157be38004d235b918c9203aec1fb564dc0d67d0" exitCode=137
Sep 18 20:23:22.878439 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:23:22.878442    2532 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-8716-66411d"
Sep 18 20:23:22.878958 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:23:22.878526    2532 scope.go:117] "RemoveContainer" containerID="54d91fa23e988652c7487bfa157be38004d235b918c9203aec1fb564dc0d67d0"
Sep 18 20:23:22.878958 ip-10-0-130-81 kubenswrapper[2532]: E0918 20:23:22.878780    2532 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-8716-8f3c88" container="test-curl-8716-8f3c88"
Sep 18 20:23:22.889738 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:23:22.889718    2532 scope.go:117] "RemoveContainer" containerID="54d91fa23e988652c7487bfa157be38004d235b918c9203aec1fb564dc0d67d0"
Sep 18 20:23:22.890003 ip-10-0-130-81 kubenswrapper[2532]: E0918 20:23:22.889983    2532 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"54d91fa23e988652c7487bfa157be38004d235b918c9203aec1fb564dc0d67d0\": container with ID starting with 54d91fa23e988652c7487bfa157be38004d235b918c9203aec1fb564dc0d67d0 not found: ID does not exist" containerID="54d91fa23e988652c7487bfa157be38004d235b918c9203aec1fb564dc0d67d0"
Sep 18 20:23:22.890048 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:23:22.890013    2532 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"54d91fa23e988652c7487bfa157be38004d235b918c9203aec1fb564dc0d67d0"} err="failed to get container status \"54d91fa23e988652c7487bfa157be38004d235b918c9203aec1fb564dc0d67d0\": rpc error: code = NotFound desc = could not find container \"54d91fa23e988652c7487bfa157be38004d235b918c9203aec1fb564dc0d67d0\": container with ID starting with 54d91fa23e988652c7487bfa157be38004d235b918c9203aec1fb564dc0d67d0 not found: ID does not exist"
Sep 18 20:23:22.977879 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:23:22.977827    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:23:23.340858 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:23:23.340820    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:23:23.507900 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:23:23.507863    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:23:23.885623 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:23:23.885589    2532 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-8716-8f3c88" podUID="00e220c9-8f3e-4926-b72b-655f14e9846c" containerName="test-curl-8716-8f3c88" containerID="cri-o://923c84c54acf50c9a5ee889203eb91af26f5533edae61ec179bee2191af28318" gracePeriod=2
Sep 18 20:23:23.887845 ip-10-0-130-81 kubenswrapper[2532]: E0918 20:23:23.887813    2532 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-8716-8f3c88\" is forbidden: User \"system:node:ip-10-0-130-81.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-130-81.ec2.internal' and this object" podUID="00e220c9-8f3e-4926-b72b-655f14e9846c" pod="opendatahub/test-curl-8716-8f3c88"
Sep 18 20:23:24.041731 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:23:24.041703    2532 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-8716-399bad"
Sep 18 20:23:24.044021 ip-10-0-130-81 kubenswrapper[2532]: E0918 20:23:24.043998    2532 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-8716-8f3c88\" is forbidden: User \"system:node:ip-10-0-130-81.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-130-81.ec2.internal' and this object" podUID="00e220c9-8f3e-4926-b72b-655f14e9846c" pod="opendatahub/test-curl-8716-8f3c88"
Sep 18 20:23:24.125824 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:23:24.125726    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bfgqg\" (UniqueName: \"kubernetes.io/projected/fc2c4bf4-0218-44c2-b411-e1c11b92aaf8-kube-api-access-bfgqg\") pod \"fc2c4bf4-0218-44c2-b411-e1c11b92aaf8\" (UID: \"fc2c4bf4-0218-44c2-b411-e1c11b92aaf8\") "
Sep 18 20:23:24.127809 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:23:24.127784    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fc2c4bf4-0218-44c2-b411-e1c11b92aaf8-kube-api-access-bfgqg" (OuterVolumeSpecName: "kube-api-access-bfgqg") pod "fc2c4bf4-0218-44c2-b411-e1c11b92aaf8" (UID: "fc2c4bf4-0218-44c2-b411-e1c11b92aaf8"). InnerVolumeSpecName "kube-api-access-bfgqg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 20:23:24.227367 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:23:24.227333    2532 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bfgqg\" (UniqueName: \"kubernetes.io/projected/fc2c4bf4-0218-44c2-b411-e1c11b92aaf8-kube-api-access-bfgqg\") on node \"ip-10-0-130-81.ec2.internal\" DevicePath \"\""
Sep 18 20:23:24.306482 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:23:24.306448    2532 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fc2c4bf4-0218-44c2-b411-e1c11b92aaf8" path="/var/lib/kubelet/pods/fc2c4bf4-0218-44c2-b411-e1c11b92aaf8/volumes"
Sep 18 20:23:24.892334 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:23:24.892299    2532 generic.go:356] "Generic (PLEG): container finished" podID="fc2c4bf4-0218-44c2-b411-e1c11b92aaf8" containerID="66f56f46c03fa7dc7f19b28dc4d44b510b38dbcbbe1727ad3964619b767ba3b1" exitCode=137
Sep 18 20:23:24.892334 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:23:24.892338    2532 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-8716-399bad"
Sep 18 20:23:24.892818 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:23:24.892388    2532 scope.go:117] "RemoveContainer" containerID="66f56f46c03fa7dc7f19b28dc4d44b510b38dbcbbe1727ad3964619b767ba3b1"
Sep 18 20:23:24.901321 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:23:24.901305    2532 scope.go:117] "RemoveContainer" containerID="66f56f46c03fa7dc7f19b28dc4d44b510b38dbcbbe1727ad3964619b767ba3b1"
Sep 18 20:23:24.901614 ip-10-0-130-81 kubenswrapper[2532]: E0918 20:23:24.901594    2532 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"66f56f46c03fa7dc7f19b28dc4d44b510b38dbcbbe1727ad3964619b767ba3b1\": container with ID starting with 66f56f46c03fa7dc7f19b28dc4d44b510b38dbcbbe1727ad3964619b767ba3b1 not found: ID does not exist" containerID="66f56f46c03fa7dc7f19b28dc4d44b510b38dbcbbe1727ad3964619b767ba3b1"
Sep 18 20:23:24.901689 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:23:24.901625    2532 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"66f56f46c03fa7dc7f19b28dc4d44b510b38dbcbbe1727ad3964619b767ba3b1"} err="failed to get container status \"66f56f46c03fa7dc7f19b28dc4d44b510b38dbcbbe1727ad3964619b767ba3b1\": rpc error: code = NotFound desc = could not find container \"66f56f46c03fa7dc7f19b28dc4d44b510b38dbcbbe1727ad3964619b767ba3b1\": container with ID starting with 66f56f46c03fa7dc7f19b28dc4d44b510b38dbcbbe1727ad3964619b767ba3b1 not found: ID does not exist"
Sep 18 20:23:26.054152 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:23:26.054130    2532 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-8716-8f3c88"
Sep 18 20:23:26.111689 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:23:26.111653    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:23:26.144865 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:23:26.144780    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dgn9g\" (UniqueName: \"kubernetes.io/projected/00e220c9-8f3e-4926-b72b-655f14e9846c-kube-api-access-dgn9g\") pod \"00e220c9-8f3e-4926-b72b-655f14e9846c\" (UID: \"00e220c9-8f3e-4926-b72b-655f14e9846c\") "
Sep 18 20:23:26.146871 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:23:26.146849    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/00e220c9-8f3e-4926-b72b-655f14e9846c-kube-api-access-dgn9g" (OuterVolumeSpecName: "kube-api-access-dgn9g") pod "00e220c9-8f3e-4926-b72b-655f14e9846c" (UID: "00e220c9-8f3e-4926-b72b-655f14e9846c"). InnerVolumeSpecName "kube-api-access-dgn9g". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 20:23:26.245836 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:23:26.245807    2532 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dgn9g\" (UniqueName: \"kubernetes.io/projected/00e220c9-8f3e-4926-b72b-655f14e9846c-kube-api-access-dgn9g\") on node \"ip-10-0-130-81.ec2.internal\" DevicePath \"\""
Sep 18 20:23:26.306628 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:23:26.306599    2532 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="00e220c9-8f3e-4926-b72b-655f14e9846c" path="/var/lib/kubelet/pods/00e220c9-8f3e-4926-b72b-655f14e9846c/volumes"
Sep 18 20:23:26.787110 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:23:26.787073    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:23:26.902656 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:23:26.902621    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:23:26.907623 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:23:26.907598    2532 generic.go:356] "Generic (PLEG): container finished" podID="00e220c9-8f3e-4926-b72b-655f14e9846c" containerID="923c84c54acf50c9a5ee889203eb91af26f5533edae61ec179bee2191af28318" exitCode=137
Sep 18 20:23:26.907756 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:23:26.907649    2532 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-8716-8f3c88"
Sep 18 20:23:26.907756 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:23:26.907692    2532 scope.go:117] "RemoveContainer" containerID="923c84c54acf50c9a5ee889203eb91af26f5533edae61ec179bee2191af28318"
Sep 18 20:23:26.916415 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:23:26.916391    2532 scope.go:117] "RemoveContainer" containerID="923c84c54acf50c9a5ee889203eb91af26f5533edae61ec179bee2191af28318"
Sep 18 20:23:26.916698 ip-10-0-130-81 kubenswrapper[2532]: E0918 20:23:26.916678    2532 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"923c84c54acf50c9a5ee889203eb91af26f5533edae61ec179bee2191af28318\": container with ID starting with 923c84c54acf50c9a5ee889203eb91af26f5533edae61ec179bee2191af28318 not found: ID does not exist" containerID="923c84c54acf50c9a5ee889203eb91af26f5533edae61ec179bee2191af28318"
Sep 18 20:23:26.916761 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:23:26.916707    2532 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"923c84c54acf50c9a5ee889203eb91af26f5533edae61ec179bee2191af28318"} err="failed to get container status \"923c84c54acf50c9a5ee889203eb91af26f5533edae61ec179bee2191af28318\": rpc error: code = NotFound desc = could not find container \"923c84c54acf50c9a5ee889203eb91af26f5533edae61ec179bee2191af28318\": container with ID starting with 923c84c54acf50c9a5ee889203eb91af26f5533edae61ec179bee2191af28318 not found: ID does not exist"
Sep 18 20:23:26.994185 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:23:26.994158    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:23:27.604926 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:23:27.604895    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:23:28.424702 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:23:28.424675    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:23:29.096738 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:23:29.096706    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:23:29.704624 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:23:29.704588    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:23:30.509769 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:23:30.509733    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:23:31.217783 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:23:31.217755    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:23:31.504886 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:23:31.504808    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:23:31.890000 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:23:31.889921    2532 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-10f4b77d-85ff88f794-fnpwg"]
Sep 18 20:23:31.890516 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:23:31.890478    2532 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="00e220c9-8f3e-4926-b72b-655f14e9846c" containerName="test-curl-8716-8f3c88"
Sep 18 20:23:31.890670 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:23:31.890515    2532 state_mem.go:107] "Deleted CPUSet assignment" podUID="00e220c9-8f3e-4926-b72b-655f14e9846c" containerName="test-curl-8716-8f3c88"
Sep 18 20:23:31.890670 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:23:31.890627    2532 memory_manager.go:368] "RemoveStaleState removing state" podUID="00e220c9-8f3e-4926-b72b-655f14e9846c" containerName="test-curl-8716-8f3c88"
Sep 18 20:23:31.893940 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:23:31.893921    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-10f4b77d-85ff88f794-fnpwg"
Sep 18 20:23:31.896327 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:23:31.896308    2532 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-10f4b77d-serving-cert\""
Sep 18 20:23:31.907026 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:23:31.907000    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-10f4b77d-85ff88f794-fnpwg"]
Sep 18 20:23:31.921759 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:23:31.921728    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:23:31.987361 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:23:31.987335    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:23:32.005018 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:23:32.004989    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f719fcf6-cc6a-4e98-883a-00270a9985c0-openshift-service-ca\") pod \"maas-api-e2e-mt-10f4b77d-85ff88f794-fnpwg\" (UID: \"f719fcf6-cc6a-4e98-883a-00270a9985c0\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-10f4b77d-85ff88f794-fnpwg"
Sep 18 20:23:32.005168 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:23:32.005022    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f719fcf6-cc6a-4e98-883a-00270a9985c0-maas-api-tls\") pod \"maas-api-e2e-mt-10f4b77d-85ff88f794-fnpwg\" (UID: \"f719fcf6-cc6a-4e98-883a-00270a9985c0\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-10f4b77d-85ff88f794-fnpwg"
Sep 18 20:23:32.005168 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:23:32.005085    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r8f8z\" (UniqueName: \"kubernetes.io/projected/f719fcf6-cc6a-4e98-883a-00270a9985c0-kube-api-access-r8f8z\") pod \"maas-api-e2e-mt-10f4b77d-85ff88f794-fnpwg\" (UID: \"f719fcf6-cc6a-4e98-883a-00270a9985c0\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-10f4b77d-85ff88f794-fnpwg"
Sep 18 20:23:32.005168 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:23:32.005117    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f719fcf6-cc6a-4e98-883a-00270a9985c0-metrics-certs\") pod \"maas-api-e2e-mt-10f4b77d-85ff88f794-fnpwg\" (UID: \"f719fcf6-cc6a-4e98-883a-00270a9985c0\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-10f4b77d-85ff88f794-fnpwg"
Sep 18 20:23:32.106675 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:23:32.106640    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f719fcf6-cc6a-4e98-883a-00270a9985c0-openshift-service-ca\") pod \"maas-api-e2e-mt-10f4b77d-85ff88f794-fnpwg\" (UID: \"f719fcf6-cc6a-4e98-883a-00270a9985c0\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-10f4b77d-85ff88f794-fnpwg"
Sep 18 20:23:32.106675 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:23:32.106677    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f719fcf6-cc6a-4e98-883a-00270a9985c0-maas-api-tls\") pod \"maas-api-e2e-mt-10f4b77d-85ff88f794-fnpwg\" (UID: \"f719fcf6-cc6a-4e98-883a-00270a9985c0\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-10f4b77d-85ff88f794-fnpwg"
Sep 18 20:23:32.106897 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:23:32.106725    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r8f8z\" (UniqueName: \"kubernetes.io/projected/f719fcf6-cc6a-4e98-883a-00270a9985c0-kube-api-access-r8f8z\") pod \"maas-api-e2e-mt-10f4b77d-85ff88f794-fnpwg\" (UID: \"f719fcf6-cc6a-4e98-883a-00270a9985c0\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-10f4b77d-85ff88f794-fnpwg"
Sep 18 20:23:32.106897 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:23:32.106773    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f719fcf6-cc6a-4e98-883a-00270a9985c0-metrics-certs\") pod \"maas-api-e2e-mt-10f4b77d-85ff88f794-fnpwg\" (UID: \"f719fcf6-cc6a-4e98-883a-00270a9985c0\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-10f4b77d-85ff88f794-fnpwg"
Sep 18 20:23:32.106897 ip-10-0-130-81 kubenswrapper[2532]: E0918 20:23:32.106881    2532 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-10f4b77d-serving-cert: secret "maas-api-e2e-mt-10f4b77d-serving-cert" not found
Sep 18 20:23:32.107017 ip-10-0-130-81 kubenswrapper[2532]: E0918 20:23:32.106958    2532 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f719fcf6-cc6a-4e98-883a-00270a9985c0-maas-api-tls podName:f719fcf6-cc6a-4e98-883a-00270a9985c0 nodeName:}" failed. No retries permitted until 2026-09-18 20:23:32.606937501 +0000 UTC m=+1344.372326890 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/f719fcf6-cc6a-4e98-883a-00270a9985c0-maas-api-tls") pod "maas-api-e2e-mt-10f4b77d-85ff88f794-fnpwg" (UID: "f719fcf6-cc6a-4e98-883a-00270a9985c0") : secret "maas-api-e2e-mt-10f4b77d-serving-cert" not found
Sep 18 20:23:32.107416 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:23:32.107397    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f719fcf6-cc6a-4e98-883a-00270a9985c0-openshift-service-ca\") pod \"maas-api-e2e-mt-10f4b77d-85ff88f794-fnpwg\" (UID: \"f719fcf6-cc6a-4e98-883a-00270a9985c0\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-10f4b77d-85ff88f794-fnpwg"
Sep 18 20:23:32.109177 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:23:32.109155    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f719fcf6-cc6a-4e98-883a-00270a9985c0-metrics-certs\") pod \"maas-api-e2e-mt-10f4b77d-85ff88f794-fnpwg\" (UID: \"f719fcf6-cc6a-4e98-883a-00270a9985c0\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-10f4b77d-85ff88f794-fnpwg"
Sep 18 20:23:32.116805 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:23:32.116786    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r8f8z\" (UniqueName: \"kubernetes.io/projected/f719fcf6-cc6a-4e98-883a-00270a9985c0-kube-api-access-r8f8z\") pod \"maas-api-e2e-mt-10f4b77d-85ff88f794-fnpwg\" (UID: \"f719fcf6-cc6a-4e98-883a-00270a9985c0\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-10f4b77d-85ff88f794-fnpwg"
Sep 18 20:23:32.358831 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:23:32.358796    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:23:32.468680 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:23:32.468640    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:23:32.613304 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:23:32.613208    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f719fcf6-cc6a-4e98-883a-00270a9985c0-maas-api-tls\") pod \"maas-api-e2e-mt-10f4b77d-85ff88f794-fnpwg\" (UID: \"f719fcf6-cc6a-4e98-883a-00270a9985c0\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-10f4b77d-85ff88f794-fnpwg"
Sep 18 20:23:32.613470 ip-10-0-130-81 kubenswrapper[2532]: E0918 20:23:32.613359    2532 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-10f4b77d-serving-cert: secret "maas-api-e2e-mt-10f4b77d-serving-cert" not found
Sep 18 20:23:32.613470 ip-10-0-130-81 kubenswrapper[2532]: E0918 20:23:32.613438    2532 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f719fcf6-cc6a-4e98-883a-00270a9985c0-maas-api-tls podName:f719fcf6-cc6a-4e98-883a-00270a9985c0 nodeName:}" failed. No retries permitted until 2026-09-18 20:23:33.613417889 +0000 UTC m=+1345.378807282 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/f719fcf6-cc6a-4e98-883a-00270a9985c0-maas-api-tls") pod "maas-api-e2e-mt-10f4b77d-85ff88f794-fnpwg" (UID: "f719fcf6-cc6a-4e98-883a-00270a9985c0") : secret "maas-api-e2e-mt-10f4b77d-serving-cert" not found
Sep 18 20:23:32.727531 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:23:32.727478    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:23:32.828178 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:23:32.828130    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:23:33.538159 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:23:33.538127    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:23:33.621743 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:23:33.621710    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f719fcf6-cc6a-4e98-883a-00270a9985c0-maas-api-tls\") pod \"maas-api-e2e-mt-10f4b77d-85ff88f794-fnpwg\" (UID: \"f719fcf6-cc6a-4e98-883a-00270a9985c0\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-10f4b77d-85ff88f794-fnpwg"
Sep 18 20:23:33.624144 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:23:33.624116    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f719fcf6-cc6a-4e98-883a-00270a9985c0-maas-api-tls\") pod \"maas-api-e2e-mt-10f4b77d-85ff88f794-fnpwg\" (UID: \"f719fcf6-cc6a-4e98-883a-00270a9985c0\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-10f4b77d-85ff88f794-fnpwg"
Sep 18 20:23:33.707791 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:23:33.707757    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-10f4b77d-85ff88f794-fnpwg"
Sep 18 20:23:33.983858 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:23:33.983835    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-10f4b77d-85ff88f794-fnpwg"]
Sep 18 20:23:33.985642 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:23:33.985609    2532 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf719fcf6_cc6a_4e98_883a_00270a9985c0.slice/crio-a4b31a0386642cc11545d5c7389d7612efb12a9b704de072df1ba283dab2d1cc WatchSource:0}: Error finding container a4b31a0386642cc11545d5c7389d7612efb12a9b704de072df1ba283dab2d1cc: Status 404 returned error can't find the container with id a4b31a0386642cc11545d5c7389d7612efb12a9b704de072df1ba283dab2d1cc
Sep 18 20:23:34.188234 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:23:34.188194    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:23:34.915415 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:23:34.915371    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:23:34.955967 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:23:34.955927    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-10f4b77d-85ff88f794-fnpwg" event={"ID":"f719fcf6-cc6a-4e98-883a-00270a9985c0","Type":"ContainerStarted","Data":"a42b2df7f95b7bf27bd4d3b1e772e8ad899afa8033f0abf253c52e0e27e36f89"}
Sep 18 20:23:34.956145 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:23:34.956011    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-10f4b77d-85ff88f794-fnpwg" event={"ID":"f719fcf6-cc6a-4e98-883a-00270a9985c0","Type":"ContainerStarted","Data":"a4b31a0386642cc11545d5c7389d7612efb12a9b704de072df1ba283dab2d1cc"}
Sep 18 20:23:34.956145 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:23:34.956033    2532 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-10f4b77d-85ff88f794-fnpwg"
Sep 18 20:23:34.974828 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:23:34.974783    2532 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-10f4b77d-85ff88f794-fnpwg" podStartSLOduration=3.581762414 podStartE2EDuration="3.974770245s" totalImagesPullingTime="393.007831ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 20:23:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 20:23:34.973816144 +0000 UTC m=+1346.739205556" watchObservedRunningTime="2026-09-18 20:23:34.974770245 +0000 UTC m=+1346.740159643"
Sep 18 20:23:35.534429 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:23:35.534392    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:23:36.121586 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:23:36.121550    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:23:36.732165 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:23:36.732133    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:23:37.417429 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:23:37.417392    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:23:37.628952 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:23:37.628918    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:23:40.967348 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:23:40.967318    2532 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-10f4b77d-85ff88f794-fnpwg"
Sep 18 20:23:41.829692 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:23:41.829652    2532 patch_prober.go:28] interesting pod/e2e-shared-b-w2-35558b-openshift-default-6bbb5bcc84-6gxxh container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.57:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Sep 18 20:23:41.829895 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:23:41.829727    2532 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-w2-35558b-openshift-default-6bbb5bcc84-6gxxh" podUID="66b1dac9-2d9a-4f89-9e49-17293b282e3a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.57:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Sep 18 20:23:44.008560 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:23:44.008524    2532 generic.go:356] "Generic (PLEG): container finished" podID="66b1dac9-2d9a-4f89-9e49-17293b282e3a" containerID="b341ce93ad3047427fbc641fdab61eacca106a3d9f4619ad84dc904f8ae38e82" exitCode=0
Sep 18 20:23:44.009094 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:23:44.008600    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-35558b-openshift-default-6bbb5bcc84-6gxxh" event={"ID":"66b1dac9-2d9a-4f89-9e49-17293b282e3a","Type":"ContainerDied","Data":"b341ce93ad3047427fbc641fdab61eacca106a3d9f4619ad84dc904f8ae38e82"}
Sep 18 20:23:44.009163 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:23:44.009149    2532 scope.go:117] "RemoveContainer" containerID="b341ce93ad3047427fbc641fdab61eacca106a3d9f4619ad84dc904f8ae38e82"
Sep 18 20:23:44.010342 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:23:44.010313    2532 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 18 20:23:44.010430 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:23:44.010380    2532 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 18 20:23:44.010430 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:23:44.010413    2532 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 18 20:23:45.023391 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:23:45.023347    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-35558b-openshift-default-6bbb5bcc84-6gxxh" event={"ID":"66b1dac9-2d9a-4f89-9e49-17293b282e3a","Type":"ContainerStarted","Data":"810359afa0469f6a6ab2be449203c0cdd3c9b61f4469ebf4551c26cdecd5e846"}
Sep 18 20:23:45.216526 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:23:45.216471    2532 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w2-35558b-openshift-default-6bbb5bcc84-6gxxh"
Sep 18 20:23:45.216859 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:23:45.216826    2532 patch_prober.go:28] interesting pod/e2e-shared-b-w2-35558b-openshift-default-6bbb5bcc84-6gxxh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" start-of-body=
Sep 18 20:23:45.216954 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:23:45.216894    2532 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-35558b-openshift-default-6bbb5bcc84-6gxxh" podUID="66b1dac9-2d9a-4f89-9e49-17293b282e3a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused"
Sep 18 20:23:46.216772 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:23:46.216734    2532 patch_prober.go:28] interesting pod/e2e-shared-b-w2-35558b-openshift-default-6bbb5bcc84-6gxxh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" start-of-body=
Sep 18 20:23:46.217373 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:23:46.217340    2532 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-35558b-openshift-default-6bbb5bcc84-6gxxh" podUID="66b1dac9-2d9a-4f89-9e49-17293b282e3a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused"
Sep 18 20:23:47.216948 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:23:47.216902    2532 patch_prober.go:28] interesting pod/e2e-shared-b-w2-35558b-openshift-default-6bbb5bcc84-6gxxh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" start-of-body=
Sep 18 20:23:47.217599 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:23:47.217563    2532 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-35558b-openshift-default-6bbb5bcc84-6gxxh" podUID="66b1dac9-2d9a-4f89-9e49-17293b282e3a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused"
Sep 18 20:23:48.217292 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:23:48.217246    2532 patch_prober.go:28] interesting pod/e2e-shared-b-w2-35558b-openshift-default-6bbb5bcc84-6gxxh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" start-of-body=
Sep 18 20:23:48.217804 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:23:48.217331    2532 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-35558b-openshift-default-6bbb5bcc84-6gxxh" podUID="66b1dac9-2d9a-4f89-9e49-17293b282e3a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused"
Sep 18 20:23:49.217096 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:23:49.217053    2532 patch_prober.go:28] interesting pod/e2e-shared-b-w2-35558b-openshift-default-6bbb5bcc84-6gxxh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" start-of-body=
Sep 18 20:23:49.217348 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:23:49.217127    2532 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-35558b-openshift-default-6bbb5bcc84-6gxxh" podUID="66b1dac9-2d9a-4f89-9e49-17293b282e3a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused"
Sep 18 20:23:50.260139 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:23:50.260109    2532 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w2-35558b-openshift-default-6bbb5bcc84-6gxxh"
Sep 18 20:23:50.260609 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:23:50.260524    2532 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w2-35558b-openshift-default-6bbb5bcc84-6gxxh"
Sep 18 20:23:50.261434 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:23:50.261414    2532 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w2-35558b-openshift-default-6bbb5bcc84-6gxxh"
Sep 18 20:23:51.894639 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:23:51.894594    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-vptdl"]
Sep 18 20:23:52.731067 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:23:52.731009    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:23:54.018796 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:23:54.018742    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:23:54.736752 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:23:54.736724    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:23:55.406161 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:23:55.406127    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:23:56.126511 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:23:56.126456    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:23:56.722644 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:23:56.722612    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:23:56.821737 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:23:56.821706    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:23:57.410248 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:23:57.410215    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:23:58.102699 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:23:58.102668    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:23:58.885354 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:23:58.885323    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:23:59.598977 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:23:59.598944    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:23:59.891967 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:23:59.891887    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:24:00.665771 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:24:00.665736    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:24:01.412896 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:24:01.412867    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:24:01.922733 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:24:01.922701    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:24:02.602732 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:24:02.602692    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:24:03.475842 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:24:03.475808    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:24:04.102689 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:24:04.102657    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:24:04.603247 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:24:04.603212    2532 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w3-b88002/premium-w3-b88002-kserve-688cff555c-tmdhz"]
Sep 18 20:24:04.611978 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:24:04.611942    2532 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w3-b88002/simulator-w3-b88002-kserve-7f78b4fbdd-w7dzh"]
Sep 18 20:24:04.684191 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:24:04.684162    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:24:04.784442 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:24:04.784410    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:24:05.519703 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:24:05.519654    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:24:15.193553 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:24:15.193521    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:24:15.260873 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:24:15.260833    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:24:15.316163 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:24:15.316135    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:24:15.415942 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:24:15.415910    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:24:15.477246 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:24:15.477213    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:24:15.753248 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:24:15.753159    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:24:15.976324 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:24:15.976286    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:24:16.534511 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:24:16.534465    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:24:17.161628 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:24:17.161582    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:24:17.907273 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:24:17.907237    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:24:18.535761 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:24:18.535726    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:24:19.311148 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:24:19.311119    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:24:19.623308 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:24:19.623217    2532 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-models-e2e-worker-w3-b88002/premium-w3-b88002-kserve-688cff555c-tmdhz" podUID="6dcac630-bd97-4a83-83d4-4ef5f299f9d1" containerName="main" containerID="cri-o://c2ed48bccfdd55afd9a2a7f50086a63608f9af700c0803c0fc6b9303cf290b0f" gracePeriod=45
Sep 18 20:24:19.635998 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:24:19.635952    2532 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-models-e2e-worker-w3-b88002/simulator-w3-b88002-kserve-7f78b4fbdd-w7dzh" podUID="268e7f4d-7c4e-4f88-8fcf-8d076415de22" containerName="main" containerID="cri-o://0e8721fcbb421d919d6aecebb2b8d49404cc85b0e285866fa7df82c967fe9633" gracePeriod=45
Sep 18 20:24:19.913433 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:24:19.913406    2532 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-b88002/premium-w3-b88002-kserve-688cff555c-tmdhz"
Sep 18 20:24:19.916111 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:24:19.916091    2532 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-b88002/simulator-w3-b88002-kserve-7f78b4fbdd-w7dzh"
Sep 18 20:24:19.976228 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:24:19.976199    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/268e7f4d-7c4e-4f88-8fcf-8d076415de22-home\") pod \"268e7f4d-7c4e-4f88-8fcf-8d076415de22\" (UID: \"268e7f4d-7c4e-4f88-8fcf-8d076415de22\") "
Sep 18 20:24:19.976228 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:24:19.976232    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/268e7f4d-7c4e-4f88-8fcf-8d076415de22-model-cache\") pod \"268e7f4d-7c4e-4f88-8fcf-8d076415de22\" (UID: \"268e7f4d-7c4e-4f88-8fcf-8d076415de22\") "
Sep 18 20:24:19.976471 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:24:19.976276    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6dcac630-bd97-4a83-83d4-4ef5f299f9d1-tls-certs\") pod \"6dcac630-bd97-4a83-83d4-4ef5f299f9d1\" (UID: \"6dcac630-bd97-4a83-83d4-4ef5f299f9d1\") "
Sep 18 20:24:19.976471 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:24:19.976293    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6dcac630-bd97-4a83-83d4-4ef5f299f9d1-tmp-dir\") pod \"6dcac630-bd97-4a83-83d4-4ef5f299f9d1\" (UID: \"6dcac630-bd97-4a83-83d4-4ef5f299f9d1\") "
Sep 18 20:24:19.976471 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:24:19.976317    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pjjm7\" (UniqueName: \"kubernetes.io/projected/268e7f4d-7c4e-4f88-8fcf-8d076415de22-kube-api-access-pjjm7\") pod \"268e7f4d-7c4e-4f88-8fcf-8d076415de22\" (UID: \"268e7f4d-7c4e-4f88-8fcf-8d076415de22\") "
Sep 18 20:24:19.976471 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:24:19.976347    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/268e7f4d-7c4e-4f88-8fcf-8d076415de22-tmp-dir\") pod \"268e7f4d-7c4e-4f88-8fcf-8d076415de22\" (UID: \"268e7f4d-7c4e-4f88-8fcf-8d076415de22\") "
Sep 18 20:24:19.976704 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:24:19.976512    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/268e7f4d-7c4e-4f88-8fcf-8d076415de22-home" (OuterVolumeSpecName: "home") pod "268e7f4d-7c4e-4f88-8fcf-8d076415de22" (UID: "268e7f4d-7c4e-4f88-8fcf-8d076415de22"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 20:24:19.976704 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:24:19.976588    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/268e7f4d-7c4e-4f88-8fcf-8d076415de22-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "268e7f4d-7c4e-4f88-8fcf-8d076415de22" (UID: "268e7f4d-7c4e-4f88-8fcf-8d076415de22"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 20:24:19.976704 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:24:19.976591    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6dcac630-bd97-4a83-83d4-4ef5f299f9d1-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "6dcac630-bd97-4a83-83d4-4ef5f299f9d1" (UID: "6dcac630-bd97-4a83-83d4-4ef5f299f9d1"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 20:24:19.976704 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:24:19.976638    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/268e7f4d-7c4e-4f88-8fcf-8d076415de22-model-cache" (OuterVolumeSpecName: "model-cache") pod "268e7f4d-7c4e-4f88-8fcf-8d076415de22" (UID: "268e7f4d-7c4e-4f88-8fcf-8d076415de22"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 20:24:19.976704 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:24:19.976648    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/268e7f4d-7c4e-4f88-8fcf-8d076415de22-tls-certs\") pod \"268e7f4d-7c4e-4f88-8fcf-8d076415de22\" (UID: \"268e7f4d-7c4e-4f88-8fcf-8d076415de22\") "
Sep 18 20:24:19.976704 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:24:19.976688    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-j29cp\" (UniqueName: \"kubernetes.io/projected/6dcac630-bd97-4a83-83d4-4ef5f299f9d1-kube-api-access-j29cp\") pod \"6dcac630-bd97-4a83-83d4-4ef5f299f9d1\" (UID: \"6dcac630-bd97-4a83-83d4-4ef5f299f9d1\") "
Sep 18 20:24:19.976990 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:24:19.976718    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6dcac630-bd97-4a83-83d4-4ef5f299f9d1-home\") pod \"6dcac630-bd97-4a83-83d4-4ef5f299f9d1\" (UID: \"6dcac630-bd97-4a83-83d4-4ef5f299f9d1\") "
Sep 18 20:24:19.976990 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:24:19.976794    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6dcac630-bd97-4a83-83d4-4ef5f299f9d1-model-cache\") pod \"6dcac630-bd97-4a83-83d4-4ef5f299f9d1\" (UID: \"6dcac630-bd97-4a83-83d4-4ef5f299f9d1\") "
Sep 18 20:24:19.976990 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:24:19.976827    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6dcac630-bd97-4a83-83d4-4ef5f299f9d1-dshm\") pod \"6dcac630-bd97-4a83-83d4-4ef5f299f9d1\" (UID: \"6dcac630-bd97-4a83-83d4-4ef5f299f9d1\") "
Sep 18 20:24:19.976990 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:24:19.976859    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/268e7f4d-7c4e-4f88-8fcf-8d076415de22-dshm\") pod \"268e7f4d-7c4e-4f88-8fcf-8d076415de22\" (UID: \"268e7f4d-7c4e-4f88-8fcf-8d076415de22\") "
Sep 18 20:24:19.977210 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:24:19.977028    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6dcac630-bd97-4a83-83d4-4ef5f299f9d1-model-cache" (OuterVolumeSpecName: "model-cache") pod "6dcac630-bd97-4a83-83d4-4ef5f299f9d1" (UID: "6dcac630-bd97-4a83-83d4-4ef5f299f9d1"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 20:24:19.977210 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:24:19.977042    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6dcac630-bd97-4a83-83d4-4ef5f299f9d1-home" (OuterVolumeSpecName: "home") pod "6dcac630-bd97-4a83-83d4-4ef5f299f9d1" (UID: "6dcac630-bd97-4a83-83d4-4ef5f299f9d1"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 20:24:19.977320 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:24:19.977230    2532 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/268e7f4d-7c4e-4f88-8fcf-8d076415de22-tmp-dir\") on node \"ip-10-0-130-81.ec2.internal\" DevicePath \"\""
Sep 18 20:24:19.977320 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:24:19.977250    2532 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6dcac630-bd97-4a83-83d4-4ef5f299f9d1-home\") on node \"ip-10-0-130-81.ec2.internal\" DevicePath \"\""
Sep 18 20:24:19.977320 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:24:19.977264    2532 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6dcac630-bd97-4a83-83d4-4ef5f299f9d1-model-cache\") on node \"ip-10-0-130-81.ec2.internal\" DevicePath \"\""
Sep 18 20:24:19.977320 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:24:19.977276    2532 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/268e7f4d-7c4e-4f88-8fcf-8d076415de22-home\") on node \"ip-10-0-130-81.ec2.internal\" DevicePath \"\""
Sep 18 20:24:19.977320 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:24:19.977292    2532 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/268e7f4d-7c4e-4f88-8fcf-8d076415de22-model-cache\") on node \"ip-10-0-130-81.ec2.internal\" DevicePath \"\""
Sep 18 20:24:19.977320 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:24:19.977308    2532 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6dcac630-bd97-4a83-83d4-4ef5f299f9d1-tmp-dir\") on node \"ip-10-0-130-81.ec2.internal\" DevicePath \"\""
Sep 18 20:24:19.979225 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:24:19.979190    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/268e7f4d-7c4e-4f88-8fcf-8d076415de22-kube-api-access-pjjm7" (OuterVolumeSpecName: "kube-api-access-pjjm7") pod "268e7f4d-7c4e-4f88-8fcf-8d076415de22" (UID: "268e7f4d-7c4e-4f88-8fcf-8d076415de22"). InnerVolumeSpecName "kube-api-access-pjjm7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 20:24:19.979346 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:24:19.979264    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/268e7f4d-7c4e-4f88-8fcf-8d076415de22-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "268e7f4d-7c4e-4f88-8fcf-8d076415de22" (UID: "268e7f4d-7c4e-4f88-8fcf-8d076415de22"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 20:24:19.979346 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:24:19.979332    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6dcac630-bd97-4a83-83d4-4ef5f299f9d1-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "6dcac630-bd97-4a83-83d4-4ef5f299f9d1" (UID: "6dcac630-bd97-4a83-83d4-4ef5f299f9d1"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 20:24:19.979460 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:24:19.979357    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6dcac630-bd97-4a83-83d4-4ef5f299f9d1-kube-api-access-j29cp" (OuterVolumeSpecName: "kube-api-access-j29cp") pod "6dcac630-bd97-4a83-83d4-4ef5f299f9d1" (UID: "6dcac630-bd97-4a83-83d4-4ef5f299f9d1"). InnerVolumeSpecName "kube-api-access-j29cp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 20:24:19.979575 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:24:19.979559    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/268e7f4d-7c4e-4f88-8fcf-8d076415de22-dshm" (OuterVolumeSpecName: "dshm") pod "268e7f4d-7c4e-4f88-8fcf-8d076415de22" (UID: "268e7f4d-7c4e-4f88-8fcf-8d076415de22"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 20:24:19.979697 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:24:19.979677    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6dcac630-bd97-4a83-83d4-4ef5f299f9d1-dshm" (OuterVolumeSpecName: "dshm") pod "6dcac630-bd97-4a83-83d4-4ef5f299f9d1" (UID: "6dcac630-bd97-4a83-83d4-4ef5f299f9d1"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 20:24:20.078502 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:24:20.078449    2532 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6dcac630-bd97-4a83-83d4-4ef5f299f9d1-tls-certs\") on node \"ip-10-0-130-81.ec2.internal\" DevicePath \"\""
Sep 18 20:24:20.078502 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:24:20.078476    2532 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pjjm7\" (UniqueName: \"kubernetes.io/projected/268e7f4d-7c4e-4f88-8fcf-8d076415de22-kube-api-access-pjjm7\") on node \"ip-10-0-130-81.ec2.internal\" DevicePath \"\""
Sep 18 20:24:20.078502 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:24:20.078498    2532 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/268e7f4d-7c4e-4f88-8fcf-8d076415de22-tls-certs\") on node \"ip-10-0-130-81.ec2.internal\" DevicePath \"\""
Sep 18 20:24:20.078502 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:24:20.078508    2532 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j29cp\" (UniqueName: \"kubernetes.io/projected/6dcac630-bd97-4a83-83d4-4ef5f299f9d1-kube-api-access-j29cp\") on node \"ip-10-0-130-81.ec2.internal\" DevicePath \"\""
Sep 18 20:24:20.078772 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:24:20.078516    2532 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6dcac630-bd97-4a83-83d4-4ef5f299f9d1-dshm\") on node \"ip-10-0-130-81.ec2.internal\" DevicePath \"\""
Sep 18 20:24:20.078772 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:24:20.078526    2532 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/268e7f4d-7c4e-4f88-8fcf-8d076415de22-dshm\") on node \"ip-10-0-130-81.ec2.internal\" DevicePath \"\""
Sep 18 20:24:20.114007 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:24:20.113953    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:24:20.246047 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:24:20.246012    2532 generic.go:356] "Generic (PLEG): container finished" podID="6dcac630-bd97-4a83-83d4-4ef5f299f9d1" containerID="c2ed48bccfdd55afd9a2a7f50086a63608f9af700c0803c0fc6b9303cf290b0f" exitCode=0
Sep 18 20:24:20.246233 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:24:20.246056    2532 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-b88002/premium-w3-b88002-kserve-688cff555c-tmdhz"
Sep 18 20:24:20.246233 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:24:20.246098    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-b88002/premium-w3-b88002-kserve-688cff555c-tmdhz" event={"ID":"6dcac630-bd97-4a83-83d4-4ef5f299f9d1","Type":"ContainerDied","Data":"c2ed48bccfdd55afd9a2a7f50086a63608f9af700c0803c0fc6b9303cf290b0f"}
Sep 18 20:24:20.246233 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:24:20.246144    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-b88002/premium-w3-b88002-kserve-688cff555c-tmdhz" event={"ID":"6dcac630-bd97-4a83-83d4-4ef5f299f9d1","Type":"ContainerDied","Data":"5905ed8af01ed516ed3d2d3213e511a5b9987f376d65acd020c067d1b096385b"}
Sep 18 20:24:20.246233 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:24:20.246170    2532 scope.go:117] "RemoveContainer" containerID="c2ed48bccfdd55afd9a2a7f50086a63608f9af700c0803c0fc6b9303cf290b0f"
Sep 18 20:24:20.247613 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:24:20.247586    2532 generic.go:356] "Generic (PLEG): container finished" podID="268e7f4d-7c4e-4f88-8fcf-8d076415de22" containerID="0e8721fcbb421d919d6aecebb2b8d49404cc85b0e285866fa7df82c967fe9633" exitCode=0
Sep 18 20:24:20.247751 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:24:20.247632    2532 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-b88002/simulator-w3-b88002-kserve-7f78b4fbdd-w7dzh"
Sep 18 20:24:20.247751 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:24:20.247653    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-b88002/simulator-w3-b88002-kserve-7f78b4fbdd-w7dzh" event={"ID":"268e7f4d-7c4e-4f88-8fcf-8d076415de22","Type":"ContainerDied","Data":"0e8721fcbb421d919d6aecebb2b8d49404cc85b0e285866fa7df82c967fe9633"}
Sep 18 20:24:20.247751 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:24:20.247700    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-b88002/simulator-w3-b88002-kserve-7f78b4fbdd-w7dzh" event={"ID":"268e7f4d-7c4e-4f88-8fcf-8d076415de22","Type":"ContainerDied","Data":"752eb3b1ff8f71c515428d181f7d0fdc6660c8ffe162568b2e315bfcdde7c553"}
Sep 18 20:24:20.255292 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:24:20.255272    2532 scope.go:117] "RemoveContainer" containerID="c2ed48bccfdd55afd9a2a7f50086a63608f9af700c0803c0fc6b9303cf290b0f"
Sep 18 20:24:20.255601 ip-10-0-130-81 kubenswrapper[2532]: E0918 20:24:20.255543    2532 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c2ed48bccfdd55afd9a2a7f50086a63608f9af700c0803c0fc6b9303cf290b0f\": container with ID starting with c2ed48bccfdd55afd9a2a7f50086a63608f9af700c0803c0fc6b9303cf290b0f not found: ID does not exist" containerID="c2ed48bccfdd55afd9a2a7f50086a63608f9af700c0803c0fc6b9303cf290b0f"
Sep 18 20:24:20.255601 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:24:20.255568    2532 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c2ed48bccfdd55afd9a2a7f50086a63608f9af700c0803c0fc6b9303cf290b0f"} err="failed to get container status \"c2ed48bccfdd55afd9a2a7f50086a63608f9af700c0803c0fc6b9303cf290b0f\": rpc error: code = NotFound desc = could not find container \"c2ed48bccfdd55afd9a2a7f50086a63608f9af700c0803c0fc6b9303cf290b0f\": container with ID starting with c2ed48bccfdd55afd9a2a7f50086a63608f9af700c0803c0fc6b9303cf290b0f not found: ID does not exist"
Sep 18 20:24:20.255601 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:24:20.255583    2532 scope.go:117] "RemoveContainer" containerID="0e8721fcbb421d919d6aecebb2b8d49404cc85b0e285866fa7df82c967fe9633"
Sep 18 20:24:20.264308 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:24:20.264297    2532 scope.go:117] "RemoveContainer" containerID="0e8721fcbb421d919d6aecebb2b8d49404cc85b0e285866fa7df82c967fe9633"
Sep 18 20:24:20.264571 ip-10-0-130-81 kubenswrapper[2532]: E0918 20:24:20.264552    2532 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0e8721fcbb421d919d6aecebb2b8d49404cc85b0e285866fa7df82c967fe9633\": container with ID starting with 0e8721fcbb421d919d6aecebb2b8d49404cc85b0e285866fa7df82c967fe9633 not found: ID does not exist" containerID="0e8721fcbb421d919d6aecebb2b8d49404cc85b0e285866fa7df82c967fe9633"
Sep 18 20:24:20.264627 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:24:20.264578    2532 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0e8721fcbb421d919d6aecebb2b8d49404cc85b0e285866fa7df82c967fe9633"} err="failed to get container status \"0e8721fcbb421d919d6aecebb2b8d49404cc85b0e285866fa7df82c967fe9633\": rpc error: code = NotFound desc = could not find container \"0e8721fcbb421d919d6aecebb2b8d49404cc85b0e285866fa7df82c967fe9633\": container with ID starting with 0e8721fcbb421d919d6aecebb2b8d49404cc85b0e285866fa7df82c967fe9633 not found: ID does not exist"
Sep 18 20:24:20.271950 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:24:20.271929    2532 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w3-b88002/simulator-w3-b88002-kserve-7f78b4fbdd-w7dzh"]
Sep 18 20:24:20.275202 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:24:20.275180    2532 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w3-b88002/simulator-w3-b88002-kserve-7f78b4fbdd-w7dzh"]
Sep 18 20:24:20.285401 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:24:20.285380    2532 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w3-b88002/premium-w3-b88002-kserve-688cff555c-tmdhz"]
Sep 18 20:24:20.289048 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:24:20.289028    2532 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w3-b88002/premium-w3-b88002-kserve-688cff555c-tmdhz"]
Sep 18 20:24:20.306850 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:24:20.306825    2532 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="268e7f4d-7c4e-4f88-8fcf-8d076415de22" path="/var/lib/kubelet/pods/268e7f4d-7c4e-4f88-8fcf-8d076415de22/volumes"
Sep 18 20:24:20.307231 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:24:20.307219    2532 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6dcac630-bd97-4a83-83d4-4ef5f299f9d1" path="/var/lib/kubelet/pods/6dcac630-bd97-4a83-83d4-4ef5f299f9d1/volumes"
Sep 18 20:24:21.986220 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:24:21.986183    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:24:22.085888 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:24:22.085855    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:24:22.225835 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:24:22.225802    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:24:28.599632 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:24:28.599596    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-vptdl"]
Sep 18 20:24:31.886326 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:24:31.886286    2532 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-w2-bab50d/auth-test-model-bab50d-kserve-79d56fcd7f-g89z4"]
Sep 18 20:24:31.886769 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:24:31.886663    2532 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="268e7f4d-7c4e-4f88-8fcf-8d076415de22" containerName="main"
Sep 18 20:24:31.886769 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:24:31.886675    2532 state_mem.go:107] "Deleted CPUSet assignment" podUID="268e7f4d-7c4e-4f88-8fcf-8d076415de22" containerName="main"
Sep 18 20:24:31.886769 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:24:31.886693    2532 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6dcac630-bd97-4a83-83d4-4ef5f299f9d1" containerName="main"
Sep 18 20:24:31.886769 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:24:31.886698    2532 state_mem.go:107] "Deleted CPUSet assignment" podUID="6dcac630-bd97-4a83-83d4-4ef5f299f9d1" containerName="main"
Sep 18 20:24:31.886769 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:24:31.886768    2532 memory_manager.go:368] "RemoveStaleState removing state" podUID="6dcac630-bd97-4a83-83d4-4ef5f299f9d1" containerName="main"
Sep 18 20:24:31.886962 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:24:31.886776    2532 memory_manager.go:368] "RemoveStaleState removing state" podUID="268e7f4d-7c4e-4f88-8fcf-8d076415de22" containerName="main"
Sep 18 20:24:31.889693 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:24:31.889677    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w2-bab50d/auth-test-model-bab50d-kserve-79d56fcd7f-g89z4"
Sep 18 20:24:31.894881 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:24:31.894853    2532 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w2-bab50d\"/\"default-dockercfg-tppw2\""
Sep 18 20:24:31.894881 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:24:31.894869    2532 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w2-bab50d\"/\"auth-test-model-bab50d-kserve-self-signed-certs\""
Sep 18 20:24:31.895359 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:24:31.894895    2532 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w2-bab50d\"/\"kube-root-ca.crt\""
Sep 18 20:24:31.895359 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:24:31.894871    2532 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w2-bab50d\"/\"openshift-service-ca.crt\""
Sep 18 20:24:31.899401 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:24:31.899377    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w2-bab50d/auth-test-model-bab50d-kserve-79d56fcd7f-g89z4"]
Sep 18 20:24:31.992940 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:24:31.992908    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a2540b2a-3627-41ea-b2b0-225d80f63662-model-cache\") pod \"auth-test-model-bab50d-kserve-79d56fcd7f-g89z4\" (UID: \"a2540b2a-3627-41ea-b2b0-225d80f63662\") " pod="ai-tenant-e2e-shared-a-w2-bab50d/auth-test-model-bab50d-kserve-79d56fcd7f-g89z4"
Sep 18 20:24:31.993100 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:24:31.992994    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a2540b2a-3627-41ea-b2b0-225d80f63662-tls-certs\") pod \"auth-test-model-bab50d-kserve-79d56fcd7f-g89z4\" (UID: \"a2540b2a-3627-41ea-b2b0-225d80f63662\") " pod="ai-tenant-e2e-shared-a-w2-bab50d/auth-test-model-bab50d-kserve-79d56fcd7f-g89z4"
Sep 18 20:24:31.993100 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:24:31.993050    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bntx7\" (UniqueName: \"kubernetes.io/projected/a2540b2a-3627-41ea-b2b0-225d80f63662-kube-api-access-bntx7\") pod \"auth-test-model-bab50d-kserve-79d56fcd7f-g89z4\" (UID: \"a2540b2a-3627-41ea-b2b0-225d80f63662\") " pod="ai-tenant-e2e-shared-a-w2-bab50d/auth-test-model-bab50d-kserve-79d56fcd7f-g89z4"
Sep 18 20:24:31.993175 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:24:31.993099    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a2540b2a-3627-41ea-b2b0-225d80f63662-tmp-dir\") pod \"auth-test-model-bab50d-kserve-79d56fcd7f-g89z4\" (UID: \"a2540b2a-3627-41ea-b2b0-225d80f63662\") " pod="ai-tenant-e2e-shared-a-w2-bab50d/auth-test-model-bab50d-kserve-79d56fcd7f-g89z4"
Sep 18 20:24:31.993175 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:24:31.993150    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a2540b2a-3627-41ea-b2b0-225d80f63662-home\") pod \"auth-test-model-bab50d-kserve-79d56fcd7f-g89z4\" (UID: \"a2540b2a-3627-41ea-b2b0-225d80f63662\") " pod="ai-tenant-e2e-shared-a-w2-bab50d/auth-test-model-bab50d-kserve-79d56fcd7f-g89z4"
Sep 18 20:24:31.993523 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:24:31.993213    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a2540b2a-3627-41ea-b2b0-225d80f63662-dshm\") pod \"auth-test-model-bab50d-kserve-79d56fcd7f-g89z4\" (UID: \"a2540b2a-3627-41ea-b2b0-225d80f63662\") " pod="ai-tenant-e2e-shared-a-w2-bab50d/auth-test-model-bab50d-kserve-79d56fcd7f-g89z4"
Sep 18 20:24:32.094405 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:24:32.094367    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a2540b2a-3627-41ea-b2b0-225d80f63662-tls-certs\") pod \"auth-test-model-bab50d-kserve-79d56fcd7f-g89z4\" (UID: \"a2540b2a-3627-41ea-b2b0-225d80f63662\") " pod="ai-tenant-e2e-shared-a-w2-bab50d/auth-test-model-bab50d-kserve-79d56fcd7f-g89z4"
Sep 18 20:24:32.094618 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:24:32.094423    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bntx7\" (UniqueName: \"kubernetes.io/projected/a2540b2a-3627-41ea-b2b0-225d80f63662-kube-api-access-bntx7\") pod \"auth-test-model-bab50d-kserve-79d56fcd7f-g89z4\" (UID: \"a2540b2a-3627-41ea-b2b0-225d80f63662\") " pod="ai-tenant-e2e-shared-a-w2-bab50d/auth-test-model-bab50d-kserve-79d56fcd7f-g89z4"
Sep 18 20:24:32.094618 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:24:32.094472    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a2540b2a-3627-41ea-b2b0-225d80f63662-tmp-dir\") pod \"auth-test-model-bab50d-kserve-79d56fcd7f-g89z4\" (UID: \"a2540b2a-3627-41ea-b2b0-225d80f63662\") " pod="ai-tenant-e2e-shared-a-w2-bab50d/auth-test-model-bab50d-kserve-79d56fcd7f-g89z4"
Sep 18 20:24:32.094618 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:24:32.094514    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a2540b2a-3627-41ea-b2b0-225d80f63662-home\") pod \"auth-test-model-bab50d-kserve-79d56fcd7f-g89z4\" (UID: \"a2540b2a-3627-41ea-b2b0-225d80f63662\") " pod="ai-tenant-e2e-shared-a-w2-bab50d/auth-test-model-bab50d-kserve-79d56fcd7f-g89z4"
Sep 18 20:24:32.094618 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:24:32.094529    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a2540b2a-3627-41ea-b2b0-225d80f63662-dshm\") pod \"auth-test-model-bab50d-kserve-79d56fcd7f-g89z4\" (UID: \"a2540b2a-3627-41ea-b2b0-225d80f63662\") " pod="ai-tenant-e2e-shared-a-w2-bab50d/auth-test-model-bab50d-kserve-79d56fcd7f-g89z4"
Sep 18 20:24:32.094618 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:24:32.094549    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a2540b2a-3627-41ea-b2b0-225d80f63662-model-cache\") pod \"auth-test-model-bab50d-kserve-79d56fcd7f-g89z4\" (UID: \"a2540b2a-3627-41ea-b2b0-225d80f63662\") " pod="ai-tenant-e2e-shared-a-w2-bab50d/auth-test-model-bab50d-kserve-79d56fcd7f-g89z4"
Sep 18 20:24:32.094939 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:24:32.094916    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a2540b2a-3627-41ea-b2b0-225d80f63662-tmp-dir\") pod \"auth-test-model-bab50d-kserve-79d56fcd7f-g89z4\" (UID: \"a2540b2a-3627-41ea-b2b0-225d80f63662\") " pod="ai-tenant-e2e-shared-a-w2-bab50d/auth-test-model-bab50d-kserve-79d56fcd7f-g89z4"
Sep 18 20:24:32.095004 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:24:32.094963    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a2540b2a-3627-41ea-b2b0-225d80f63662-home\") pod \"auth-test-model-bab50d-kserve-79d56fcd7f-g89z4\" (UID: \"a2540b2a-3627-41ea-b2b0-225d80f63662\") " pod="ai-tenant-e2e-shared-a-w2-bab50d/auth-test-model-bab50d-kserve-79d56fcd7f-g89z4"
Sep 18 20:24:32.095004 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:24:32.094990    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a2540b2a-3627-41ea-b2b0-225d80f63662-model-cache\") pod \"auth-test-model-bab50d-kserve-79d56fcd7f-g89z4\" (UID: \"a2540b2a-3627-41ea-b2b0-225d80f63662\") " pod="ai-tenant-e2e-shared-a-w2-bab50d/auth-test-model-bab50d-kserve-79d56fcd7f-g89z4"
Sep 18 20:24:32.096825 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:24:32.096806    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a2540b2a-3627-41ea-b2b0-225d80f63662-dshm\") pod \"auth-test-model-bab50d-kserve-79d56fcd7f-g89z4\" (UID: \"a2540b2a-3627-41ea-b2b0-225d80f63662\") " pod="ai-tenant-e2e-shared-a-w2-bab50d/auth-test-model-bab50d-kserve-79d56fcd7f-g89z4"
Sep 18 20:24:32.097001 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:24:32.096982    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a2540b2a-3627-41ea-b2b0-225d80f63662-tls-certs\") pod \"auth-test-model-bab50d-kserve-79d56fcd7f-g89z4\" (UID: \"a2540b2a-3627-41ea-b2b0-225d80f63662\") " pod="ai-tenant-e2e-shared-a-w2-bab50d/auth-test-model-bab50d-kserve-79d56fcd7f-g89z4"
Sep 18 20:24:32.102475 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:24:32.102451    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bntx7\" (UniqueName: \"kubernetes.io/projected/a2540b2a-3627-41ea-b2b0-225d80f63662-kube-api-access-bntx7\") pod \"auth-test-model-bab50d-kserve-79d56fcd7f-g89z4\" (UID: \"a2540b2a-3627-41ea-b2b0-225d80f63662\") " pod="ai-tenant-e2e-shared-a-w2-bab50d/auth-test-model-bab50d-kserve-79d56fcd7f-g89z4"
Sep 18 20:24:32.184124 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:24:32.184043    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:24:32.200114 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:24:32.200090    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w2-bab50d/auth-test-model-bab50d-kserve-79d56fcd7f-g89z4"
Sep 18 20:24:32.472907 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:24:32.472878    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w2-bab50d/auth-test-model-bab50d-kserve-79d56fcd7f-g89z4"]
Sep 18 20:24:32.473884 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:24:32.473857    2532 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda2540b2a_3627_41ea_b2b0_225d80f63662.slice/crio-cae888ea4e64873f5877ef606c3458b0a71d31ec84b4fc1dd435b05e8f793ff7 WatchSource:0}: Error finding container cae888ea4e64873f5877ef606c3458b0a71d31ec84b4fc1dd435b05e8f793ff7: Status 404 returned error can't find the container with id cae888ea4e64873f5877ef606c3458b0a71d31ec84b4fc1dd435b05e8f793ff7
Sep 18 20:24:33.323624 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:24:33.323591    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w2-bab50d/auth-test-model-bab50d-kserve-79d56fcd7f-g89z4" event={"ID":"a2540b2a-3627-41ea-b2b0-225d80f63662","Type":"ContainerStarted","Data":"88b1759ce7cdc3284c6acc7cebcc0aae04fa6cf8211d3d8d623fb0dcdcc32b70"}
Sep 18 20:24:33.323624 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:24:33.323631    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w2-bab50d/auth-test-model-bab50d-kserve-79d56fcd7f-g89z4" event={"ID":"a2540b2a-3627-41ea-b2b0-225d80f63662","Type":"ContainerStarted","Data":"cae888ea4e64873f5877ef606c3458b0a71d31ec84b4fc1dd435b05e8f793ff7"}
Sep 18 20:24:33.340412 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:24:33.340365    2532 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-w2-bab50d/auth-test-model-bab50d-kserve-79d56fcd7f-g89z4" podStartSLOduration=2.340350461 podStartE2EDuration="2.340350461s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 20:24:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 20:24:33.339057767 +0000 UTC m=+1405.104447167" watchObservedRunningTime="2026-09-18 20:24:33.340350461 +0000 UTC m=+1405.105739860"
Sep 18 20:24:41.689328 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:24:41.689290    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:24:41.791416 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:24:41.791384    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:24:42.040666 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:24:42.040627    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:24:42.200536 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:24:42.200476    2532 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-w2-bab50d/auth-test-model-bab50d-kserve-79d56fcd7f-g89z4"
Sep 18 20:24:42.200747 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:24:42.200559    2532 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-w2-bab50d/auth-test-model-bab50d-kserve-79d56fcd7f-g89z4"
Sep 18 20:24:42.213220 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:24:42.213186    2532 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-w2-bab50d/auth-test-model-bab50d-kserve-79d56fcd7f-g89z4"
Sep 18 20:24:42.386430 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:24:42.386346    2532 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-w2-bab50d/auth-test-model-bab50d-kserve-79d56fcd7f-g89z4"
Sep 18 20:24:52.896792 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:24:52.896712    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-vptdl"]
Sep 18 20:25:14.936750 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:25:14.936716    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:25:14.992916 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:25:14.992882    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-vptdl"]
Sep 18 20:25:14.995221 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:25:14.995199    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:25:22.321343 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:25:22.321309    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:25:38.903161 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:25:38.903124    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-vptdl"]
Sep 18 20:25:54.735075 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:25:54.735041    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:25:56.895442 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:25:56.895412    2532 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w0-116778/embedding-w0-116778-kserve-6bd94f5698-n429v"]
Sep 18 20:25:56.899021 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:25:56.899005    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-116778/embedding-w0-116778-kserve-6bd94f5698-n429v"
Sep 18 20:25:56.901596 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:25:56.901572    2532 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-116778\"/\"embedding-w0-116778-kserve-self-signed-certs\""
Sep 18 20:25:56.908259 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:25:56.908237    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-116778/embedding-w0-116778-kserve-6bd94f5698-n429v"]
Sep 18 20:25:57.001619 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:25:57.001579    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:25:57.024891 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:25:57.024849    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9ec8aa37-8e42-432f-860a-aae87a6a7c5f-home\") pod \"embedding-w0-116778-kserve-6bd94f5698-n429v\" (UID: \"9ec8aa37-8e42-432f-860a-aae87a6a7c5f\") " pod="e2e-models-e2e-worker-w0-116778/embedding-w0-116778-kserve-6bd94f5698-n429v"
Sep 18 20:25:57.025060 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:25:57.024918    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9ec8aa37-8e42-432f-860a-aae87a6a7c5f-tls-certs\") pod \"embedding-w0-116778-kserve-6bd94f5698-n429v\" (UID: \"9ec8aa37-8e42-432f-860a-aae87a6a7c5f\") " pod="e2e-models-e2e-worker-w0-116778/embedding-w0-116778-kserve-6bd94f5698-n429v"
Sep 18 20:25:57.025060 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:25:57.024972    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9ec8aa37-8e42-432f-860a-aae87a6a7c5f-tmp-dir\") pod \"embedding-w0-116778-kserve-6bd94f5698-n429v\" (UID: \"9ec8aa37-8e42-432f-860a-aae87a6a7c5f\") " pod="e2e-models-e2e-worker-w0-116778/embedding-w0-116778-kserve-6bd94f5698-n429v"
Sep 18 20:25:57.025060 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:25:57.024993    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m69q4\" (UniqueName: \"kubernetes.io/projected/9ec8aa37-8e42-432f-860a-aae87a6a7c5f-kube-api-access-m69q4\") pod \"embedding-w0-116778-kserve-6bd94f5698-n429v\" (UID: \"9ec8aa37-8e42-432f-860a-aae87a6a7c5f\") " pod="e2e-models-e2e-worker-w0-116778/embedding-w0-116778-kserve-6bd94f5698-n429v"
Sep 18 20:25:57.025060 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:25:57.025031    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9ec8aa37-8e42-432f-860a-aae87a6a7c5f-dshm\") pod \"embedding-w0-116778-kserve-6bd94f5698-n429v\" (UID: \"9ec8aa37-8e42-432f-860a-aae87a6a7c5f\") " pod="e2e-models-e2e-worker-w0-116778/embedding-w0-116778-kserve-6bd94f5698-n429v"
Sep 18 20:25:57.025240 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:25:57.025077    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9ec8aa37-8e42-432f-860a-aae87a6a7c5f-model-cache\") pod \"embedding-w0-116778-kserve-6bd94f5698-n429v\" (UID: \"9ec8aa37-8e42-432f-860a-aae87a6a7c5f\") " pod="e2e-models-e2e-worker-w0-116778/embedding-w0-116778-kserve-6bd94f5698-n429v"
Sep 18 20:25:57.126462 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:25:57.126428    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9ec8aa37-8e42-432f-860a-aae87a6a7c5f-home\") pod \"embedding-w0-116778-kserve-6bd94f5698-n429v\" (UID: \"9ec8aa37-8e42-432f-860a-aae87a6a7c5f\") " pod="e2e-models-e2e-worker-w0-116778/embedding-w0-116778-kserve-6bd94f5698-n429v"
Sep 18 20:25:57.126653 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:25:57.126471    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9ec8aa37-8e42-432f-860a-aae87a6a7c5f-tls-certs\") pod \"embedding-w0-116778-kserve-6bd94f5698-n429v\" (UID: \"9ec8aa37-8e42-432f-860a-aae87a6a7c5f\") " pod="e2e-models-e2e-worker-w0-116778/embedding-w0-116778-kserve-6bd94f5698-n429v"
Sep 18 20:25:57.126653 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:25:57.126526    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9ec8aa37-8e42-432f-860a-aae87a6a7c5f-tmp-dir\") pod \"embedding-w0-116778-kserve-6bd94f5698-n429v\" (UID: \"9ec8aa37-8e42-432f-860a-aae87a6a7c5f\") " pod="e2e-models-e2e-worker-w0-116778/embedding-w0-116778-kserve-6bd94f5698-n429v"
Sep 18 20:25:57.126653 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:25:57.126544    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m69q4\" (UniqueName: \"kubernetes.io/projected/9ec8aa37-8e42-432f-860a-aae87a6a7c5f-kube-api-access-m69q4\") pod \"embedding-w0-116778-kserve-6bd94f5698-n429v\" (UID: \"9ec8aa37-8e42-432f-860a-aae87a6a7c5f\") " pod="e2e-models-e2e-worker-w0-116778/embedding-w0-116778-kserve-6bd94f5698-n429v"
Sep 18 20:25:57.126653 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:25:57.126568    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9ec8aa37-8e42-432f-860a-aae87a6a7c5f-dshm\") pod \"embedding-w0-116778-kserve-6bd94f5698-n429v\" (UID: \"9ec8aa37-8e42-432f-860a-aae87a6a7c5f\") " pod="e2e-models-e2e-worker-w0-116778/embedding-w0-116778-kserve-6bd94f5698-n429v"
Sep 18 20:25:57.126653 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:25:57.126610    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9ec8aa37-8e42-432f-860a-aae87a6a7c5f-model-cache\") pod \"embedding-w0-116778-kserve-6bd94f5698-n429v\" (UID: \"9ec8aa37-8e42-432f-860a-aae87a6a7c5f\") " pod="e2e-models-e2e-worker-w0-116778/embedding-w0-116778-kserve-6bd94f5698-n429v"
Sep 18 20:25:57.126932 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:25:57.126869    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9ec8aa37-8e42-432f-860a-aae87a6a7c5f-home\") pod \"embedding-w0-116778-kserve-6bd94f5698-n429v\" (UID: \"9ec8aa37-8e42-432f-860a-aae87a6a7c5f\") " pod="e2e-models-e2e-worker-w0-116778/embedding-w0-116778-kserve-6bd94f5698-n429v"
Sep 18 20:25:57.126932 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:25:57.126920    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9ec8aa37-8e42-432f-860a-aae87a6a7c5f-tmp-dir\") pod \"embedding-w0-116778-kserve-6bd94f5698-n429v\" (UID: \"9ec8aa37-8e42-432f-860a-aae87a6a7c5f\") " pod="e2e-models-e2e-worker-w0-116778/embedding-w0-116778-kserve-6bd94f5698-n429v"
Sep 18 20:25:57.127016 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:25:57.126987    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9ec8aa37-8e42-432f-860a-aae87a6a7c5f-model-cache\") pod \"embedding-w0-116778-kserve-6bd94f5698-n429v\" (UID: \"9ec8aa37-8e42-432f-860a-aae87a6a7c5f\") " pod="e2e-models-e2e-worker-w0-116778/embedding-w0-116778-kserve-6bd94f5698-n429v"
Sep 18 20:25:57.128792 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:25:57.128765    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9ec8aa37-8e42-432f-860a-aae87a6a7c5f-dshm\") pod \"embedding-w0-116778-kserve-6bd94f5698-n429v\" (UID: \"9ec8aa37-8e42-432f-860a-aae87a6a7c5f\") " pod="e2e-models-e2e-worker-w0-116778/embedding-w0-116778-kserve-6bd94f5698-n429v"
Sep 18 20:25:57.129137 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:25:57.129120    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9ec8aa37-8e42-432f-860a-aae87a6a7c5f-tls-certs\") pod \"embedding-w0-116778-kserve-6bd94f5698-n429v\" (UID: \"9ec8aa37-8e42-432f-860a-aae87a6a7c5f\") " pod="e2e-models-e2e-worker-w0-116778/embedding-w0-116778-kserve-6bd94f5698-n429v"
Sep 18 20:25:57.134591 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:25:57.134570    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m69q4\" (UniqueName: \"kubernetes.io/projected/9ec8aa37-8e42-432f-860a-aae87a6a7c5f-kube-api-access-m69q4\") pod \"embedding-w0-116778-kserve-6bd94f5698-n429v\" (UID: \"9ec8aa37-8e42-432f-860a-aae87a6a7c5f\") " pod="e2e-models-e2e-worker-w0-116778/embedding-w0-116778-kserve-6bd94f5698-n429v"
Sep 18 20:25:57.208069 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:25:57.208041    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-116778/embedding-w0-116778-kserve-6bd94f5698-n429v"
Sep 18 20:25:57.481550 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:25:57.481525    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-116778/embedding-w0-116778-kserve-6bd94f5698-n429v"]
Sep 18 20:25:57.483551 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:25:57.483519    2532 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9ec8aa37_8e42_432f_860a_aae87a6a7c5f.slice/crio-39c19682628ae4d1b542c66fffad9637902c78ed1988481daf1193ae4aebc9aa WatchSource:0}: Error finding container 39c19682628ae4d1b542c66fffad9637902c78ed1988481daf1193ae4aebc9aa: Status 404 returned error can't find the container with id 39c19682628ae4d1b542c66fffad9637902c78ed1988481daf1193ae4aebc9aa
Sep 18 20:25:57.803862 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:25:57.803746    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-116778/embedding-w0-116778-kserve-6bd94f5698-n429v" event={"ID":"9ec8aa37-8e42-432f-860a-aae87a6a7c5f","Type":"ContainerStarted","Data":"ba4f8a04c73b1f1a0cecdeee5d67c590b18134f62221d41af9652f74c76dbbfe"}
Sep 18 20:25:57.803862 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:25:57.803799    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-116778/embedding-w0-116778-kserve-6bd94f5698-n429v" event={"ID":"9ec8aa37-8e42-432f-860a-aae87a6a7c5f","Type":"ContainerStarted","Data":"39c19682628ae4d1b542c66fffad9637902c78ed1988481daf1193ae4aebc9aa"}
Sep 18 20:25:57.821562 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:25:57.821508    2532 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-models-e2e-worker-w0-116778/embedding-w0-116778-kserve-6bd94f5698-n429v" podStartSLOduration=1.821462954 podStartE2EDuration="1.821462954s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 20:25:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 20:25:57.821381853 +0000 UTC m=+1489.586771252" watchObservedRunningTime="2026-09-18 20:25:57.821462954 +0000 UTC m=+1489.586852353"
Sep 18 20:26:03.891314 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:26:03.891278    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-vptdl"]
Sep 18 20:26:07.208887 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:26:07.208851    2532 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-116778/embedding-w0-116778-kserve-6bd94f5698-n429v"
Sep 18 20:26:07.209280 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:26:07.208909    2532 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w0-116778/embedding-w0-116778-kserve-6bd94f5698-n429v"
Sep 18 20:26:07.221374 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:26:07.221348    2532 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w0-116778/embedding-w0-116778-kserve-6bd94f5698-n429v"
Sep 18 20:26:07.877005 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:26:07.876974    2532 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-116778/embedding-w0-116778-kserve-6bd94f5698-n429v"
Sep 18 20:26:09.301189 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:26:09.301152    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:26:10.114649 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:26:10.114613    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:26:10.401061 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:26:10.400979    2532 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-m7wdd_32e33d39-7826-499d-bc01-0487b7580e7a/console-operator/1.log"
Sep 18 20:26:10.401061 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:26:10.400980    2532 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-m7wdd_32e33d39-7826-499d-bc01-0487b7580e7a/console-operator/1.log"
Sep 18 20:26:10.410055 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:26:10.410029    2532 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rtw9n_73fbf16b-3549-4a6c-a959-b633237ebd0c/ovn-acl-logging/0.log"
Sep 18 20:26:10.410189 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:26:10.410082    2532 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rtw9n_73fbf16b-3549-4a6c-a959-b633237ebd0c/ovn-acl-logging/0.log"
Sep 18 20:26:11.023890 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:26:11.023860    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:26:11.632058 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:26:11.632026    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:26:12.307234 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:26:12.307202    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:26:12.924787 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:26:12.924748    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:26:13.530613 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:26:13.530582    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:26:13.938622 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:26:13.938547    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:26:14.331376 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:26:14.331344    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:26:15.028692 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:26:15.028657    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:26:19.311237 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:26:19.311202    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:26:20.265189 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:26:20.265162    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:26:24.846061 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:26:24.845981    2532 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-10f4b77d-85ff88f794-fnpwg"]
Sep 18 20:26:24.846428 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:26:24.846227    2532 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-10f4b77d-85ff88f794-fnpwg" podUID="f719fcf6-cc6a-4e98-883a-00270a9985c0" containerName="maas-api" containerID="cri-o://a42b2df7f95b7bf27bd4d3b1e772e8ad899afa8033f0abf253c52e0e27e36f89" gracePeriod=30
Sep 18 20:26:24.872527 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:26:24.872470    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:26:24.967289 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:26:24.967259    2532 generic.go:356] "Generic (PLEG): container finished" podID="f719fcf6-cc6a-4e98-883a-00270a9985c0" containerID="a42b2df7f95b7bf27bd4d3b1e772e8ad899afa8033f0abf253c52e0e27e36f89" exitCode=0
Sep 18 20:26:24.967412 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:26:24.967294    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-10f4b77d-85ff88f794-fnpwg" event={"ID":"f719fcf6-cc6a-4e98-883a-00270a9985c0","Type":"ContainerDied","Data":"a42b2df7f95b7bf27bd4d3b1e772e8ad899afa8033f0abf253c52e0e27e36f89"}
Sep 18 20:26:24.973133 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:26:24.973104    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:26:25.076988 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:26:25.076959    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:26:25.112060 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:26:25.112000    2532 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-10f4b77d-85ff88f794-fnpwg"
Sep 18 20:26:25.178228 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:26:25.178195    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:26:25.207629 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:26:25.207596    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f719fcf6-cc6a-4e98-883a-00270a9985c0-maas-api-tls\") pod \"f719fcf6-cc6a-4e98-883a-00270a9985c0\" (UID: \"f719fcf6-cc6a-4e98-883a-00270a9985c0\") "
Sep 18 20:26:25.207806 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:26:25.207644    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r8f8z\" (UniqueName: \"kubernetes.io/projected/f719fcf6-cc6a-4e98-883a-00270a9985c0-kube-api-access-r8f8z\") pod \"f719fcf6-cc6a-4e98-883a-00270a9985c0\" (UID: \"f719fcf6-cc6a-4e98-883a-00270a9985c0\") "
Sep 18 20:26:25.207806 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:26:25.207668    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f719fcf6-cc6a-4e98-883a-00270a9985c0-openshift-service-ca\") pod \"f719fcf6-cc6a-4e98-883a-00270a9985c0\" (UID: \"f719fcf6-cc6a-4e98-883a-00270a9985c0\") "
Sep 18 20:26:25.207806 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:26:25.207713    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f719fcf6-cc6a-4e98-883a-00270a9985c0-metrics-certs\") pod \"f719fcf6-cc6a-4e98-883a-00270a9985c0\" (UID: \"f719fcf6-cc6a-4e98-883a-00270a9985c0\") "
Sep 18 20:26:25.208131 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:26:25.208107    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f719fcf6-cc6a-4e98-883a-00270a9985c0-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "f719fcf6-cc6a-4e98-883a-00270a9985c0" (UID: "f719fcf6-cc6a-4e98-883a-00270a9985c0"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 20:26:25.209936 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:26:25.209901    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f719fcf6-cc6a-4e98-883a-00270a9985c0-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "f719fcf6-cc6a-4e98-883a-00270a9985c0" (UID: "f719fcf6-cc6a-4e98-883a-00270a9985c0"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 20:26:25.210038 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:26:25.209914    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f719fcf6-cc6a-4e98-883a-00270a9985c0-kube-api-access-r8f8z" (OuterVolumeSpecName: "kube-api-access-r8f8z") pod "f719fcf6-cc6a-4e98-883a-00270a9985c0" (UID: "f719fcf6-cc6a-4e98-883a-00270a9985c0"). InnerVolumeSpecName "kube-api-access-r8f8z". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 20:26:25.210038 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:26:25.209998    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f719fcf6-cc6a-4e98-883a-00270a9985c0-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "f719fcf6-cc6a-4e98-883a-00270a9985c0" (UID: "f719fcf6-cc6a-4e98-883a-00270a9985c0"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 20:26:25.309334 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:26:25.309292    2532 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f719fcf6-cc6a-4e98-883a-00270a9985c0-metrics-certs\") on node \"ip-10-0-130-81.ec2.internal\" DevicePath \"\""
Sep 18 20:26:25.309334 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:26:25.309326    2532 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f719fcf6-cc6a-4e98-883a-00270a9985c0-maas-api-tls\") on node \"ip-10-0-130-81.ec2.internal\" DevicePath \"\""
Sep 18 20:26:25.309334 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:26:25.309335    2532 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r8f8z\" (UniqueName: \"kubernetes.io/projected/f719fcf6-cc6a-4e98-883a-00270a9985c0-kube-api-access-r8f8z\") on node \"ip-10-0-130-81.ec2.internal\" DevicePath \"\""
Sep 18 20:26:25.309334 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:26:25.309345    2532 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f719fcf6-cc6a-4e98-883a-00270a9985c0-openshift-service-ca\") on node \"ip-10-0-130-81.ec2.internal\" DevicePath \"\""
Sep 18 20:26:25.505416 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:26:25.505381    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:26:25.975002 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:26:25.974972    2532 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-10f4b77d-85ff88f794-fnpwg"
Sep 18 20:26:25.975449 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:26:25.975018    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-10f4b77d-85ff88f794-fnpwg" event={"ID":"f719fcf6-cc6a-4e98-883a-00270a9985c0","Type":"ContainerDied","Data":"a4b31a0386642cc11545d5c7389d7612efb12a9b704de072df1ba283dab2d1cc"}
Sep 18 20:26:25.975449 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:26:25.975078    2532 scope.go:117] "RemoveContainer" containerID="a42b2df7f95b7bf27bd4d3b1e772e8ad899afa8033f0abf253c52e0e27e36f89"
Sep 18 20:26:26.000310 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:26:26.000282    2532 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-10f4b77d-85ff88f794-fnpwg"]
Sep 18 20:26:26.004997 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:26:26.004975    2532 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-10f4b77d-85ff88f794-fnpwg"]
Sep 18 20:26:26.149237 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:26:26.149187    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:26:26.306237 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:26:26.306152    2532 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f719fcf6-cc6a-4e98-883a-00270a9985c0" path="/var/lib/kubelet/pods/f719fcf6-cc6a-4e98-883a-00270a9985c0/volumes"
Sep 18 20:26:26.777913 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:26:26.777878    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:26:27.525106 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:26:27.525075    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:26:28.154551 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:26:28.154513    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:26:28.254069 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:26:28.254037    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:26:28.911085 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:26:28.911053    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:26:33.204185 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:26:33.204149    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:26:39.491886 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:26:39.491848    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-vptdl"]
Sep 18 20:27:05.891898 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:27:05.891861    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-vptdl"]
Sep 18 20:27:09.700625 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:27:09.700593    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:27:10.319777 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:27:10.319743    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:27:13.453790 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:27:13.453754    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:27:14.077757 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:27:14.077727    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:27:14.935013 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:27:14.934962    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:27:15.608851 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:27:15.608809    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:27:16.313632 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:27:16.313599    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:27:16.415105 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:27:16.415064    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:27:16.600211 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:27:16.600132    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:27:17.040304 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:27:17.040277    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:27:17.790372 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:27:17.790332    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:27:17.875368 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:27:17.875337    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:27:17.977406 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:27:17.977375    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:27:18.444864 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:27:18.444828    2532 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-116778/distinct-w0-116778-kserve-5f6955bd97-qf5vt"]
Sep 18 20:27:18.455458 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:27:18.455430    2532 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-116778/embedding-w0-116778-kserve-6bd94f5698-n429v"]
Sep 18 20:27:18.472959 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:27:18.472928    2532 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-116778/premium-w0-116778-kserve-9b5787766-tth7r"]
Sep 18 20:27:18.484780 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:27:18.484749    2532 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-116778/simulator-w0-116778-kserve-8f94f4bdc-4wbtx"]
Sep 18 20:27:18.540598 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:27:18.540545    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:27:18.634757 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:27:18.634726    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:27:18.732261 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:27:18.732228    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:27:18.833010 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:27:18.832970    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:27:19.920257 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:27:19.920224    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:27:25.742236 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:27:25.742204    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:27:25.842331 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:27:25.842302    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:27:26.257973 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:27:26.257933    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:27:26.374728 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:27:26.374695    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:27:29.033545 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:27:29.033514    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:27:29.133735 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:27:29.133703    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:27:29.234790 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:27:29.234742    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:27:29.334209 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:27:29.334118    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:27:29.576193 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:27:29.576161    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:27:29.678668 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:27:29.678586    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:27:30.314813 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:27:30.314776    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:27:30.981448 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:27:30.981406    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:27:31.828040 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:27:31.828006    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:27:32.348853 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:27:32.348819    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:27:32.451650 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:27:32.451608    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:27:32.549474 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:27:32.549445    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:27:33.200509 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:27:33.200457    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-vptdl"]
Sep 18 20:27:33.465179 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:27:33.465127    2532 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-models-e2e-worker-w0-116778/distinct-w0-116778-kserve-5f6955bd97-qf5vt" podUID="f1270a9f-d699-4dc7-a101-df212a401b81" containerName="main" containerID="cri-o://89925065f14a18f99d444f2c9e2b34d146713b2c14bee410b170f55b874eebe9" gracePeriod=45
Sep 18 20:27:33.476863 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:27:33.476836    2532 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-models-e2e-worker-w0-116778/embedding-w0-116778-kserve-6bd94f5698-n429v" podUID="9ec8aa37-8e42-432f-860a-aae87a6a7c5f" containerName="main" containerID="cri-o://ba4f8a04c73b1f1a0cecdeee5d67c590b18134f62221d41af9652f74c76dbbfe" gracePeriod=45
Sep 18 20:27:33.491674 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:27:33.491646    2532 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-models-e2e-worker-w0-116778/premium-w0-116778-kserve-9b5787766-tth7r" podUID="53c9b26c-7ae0-418a-9296-08fc8ad7b84e" containerName="main" containerID="cri-o://3070bd4e0384dae04a9a7b9df2da95f3c98c15597f4482ea27e79f795dd8d2bc" gracePeriod=45
Sep 18 20:27:33.503755 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:27:33.503723    2532 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-models-e2e-worker-w0-116778/simulator-w0-116778-kserve-8f94f4bdc-4wbtx" podUID="d839bc6f-e7f5-496d-9e64-eacee63a95fd" containerName="main" containerID="cri-o://0eb85f11fccbb6c2ed97ecd27fdf6120fdd7c602d3eec60dae71e112ff48d0c0" gracePeriod=45
Sep 18 20:27:33.762972 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:27:33.762947    2532 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-116778/distinct-w0-116778-kserve-5f6955bd97-qf5vt"
Sep 18 20:27:33.791049 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:27:33.791020    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-97rsf\" (UniqueName: \"kubernetes.io/projected/f1270a9f-d699-4dc7-a101-df212a401b81-kube-api-access-97rsf\") pod \"f1270a9f-d699-4dc7-a101-df212a401b81\" (UID: \"f1270a9f-d699-4dc7-a101-df212a401b81\") "
Sep 18 20:27:33.791049 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:27:33.791061    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f1270a9f-d699-4dc7-a101-df212a401b81-dshm\") pod \"f1270a9f-d699-4dc7-a101-df212a401b81\" (UID: \"f1270a9f-d699-4dc7-a101-df212a401b81\") "
Sep 18 20:27:33.791291 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:27:33.791106    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f1270a9f-d699-4dc7-a101-df212a401b81-home\") pod \"f1270a9f-d699-4dc7-a101-df212a401b81\" (UID: \"f1270a9f-d699-4dc7-a101-df212a401b81\") "
Sep 18 20:27:33.791291 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:27:33.791149    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f1270a9f-d699-4dc7-a101-df212a401b81-tmp-dir\") pod \"f1270a9f-d699-4dc7-a101-df212a401b81\" (UID: \"f1270a9f-d699-4dc7-a101-df212a401b81\") "
Sep 18 20:27:33.791291 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:27:33.791180    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f1270a9f-d699-4dc7-a101-df212a401b81-model-cache\") pod \"f1270a9f-d699-4dc7-a101-df212a401b81\" (UID: \"f1270a9f-d699-4dc7-a101-df212a401b81\") "
Sep 18 20:27:33.791291 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:27:33.791250    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f1270a9f-d699-4dc7-a101-df212a401b81-tls-certs\") pod \"f1270a9f-d699-4dc7-a101-df212a401b81\" (UID: \"f1270a9f-d699-4dc7-a101-df212a401b81\") "
Sep 18 20:27:33.794297 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:27:33.792117    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f1270a9f-d699-4dc7-a101-df212a401b81-model-cache" (OuterVolumeSpecName: "model-cache") pod "f1270a9f-d699-4dc7-a101-df212a401b81" (UID: "f1270a9f-d699-4dc7-a101-df212a401b81"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 20:27:33.794297 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:27:33.792130    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f1270a9f-d699-4dc7-a101-df212a401b81-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "f1270a9f-d699-4dc7-a101-df212a401b81" (UID: "f1270a9f-d699-4dc7-a101-df212a401b81"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 20:27:33.794297 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:27:33.792252    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f1270a9f-d699-4dc7-a101-df212a401b81-home" (OuterVolumeSpecName: "home") pod "f1270a9f-d699-4dc7-a101-df212a401b81" (UID: "f1270a9f-d699-4dc7-a101-df212a401b81"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 20:27:33.798400 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:27:33.798372    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f1270a9f-d699-4dc7-a101-df212a401b81-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "f1270a9f-d699-4dc7-a101-df212a401b81" (UID: "f1270a9f-d699-4dc7-a101-df212a401b81"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 20:27:33.800198 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:27:33.800168    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f1270a9f-d699-4dc7-a101-df212a401b81-dshm" (OuterVolumeSpecName: "dshm") pod "f1270a9f-d699-4dc7-a101-df212a401b81" (UID: "f1270a9f-d699-4dc7-a101-df212a401b81"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 20:27:33.800461 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:27:33.800438    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f1270a9f-d699-4dc7-a101-df212a401b81-kube-api-access-97rsf" (OuterVolumeSpecName: "kube-api-access-97rsf") pod "f1270a9f-d699-4dc7-a101-df212a401b81" (UID: "f1270a9f-d699-4dc7-a101-df212a401b81"). InnerVolumeSpecName "kube-api-access-97rsf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 20:27:33.823529 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:27:33.823473    2532 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-116778/embedding-w0-116778-kserve-6bd94f5698-n429v"
Sep 18 20:27:33.826998 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:27:33.826977    2532 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-116778/premium-w0-116778-kserve-9b5787766-tth7r"
Sep 18 20:27:33.835908 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:27:33.835889    2532 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-116778/simulator-w0-116778-kserve-8f94f4bdc-4wbtx"
Sep 18 20:27:33.892336 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:27:33.892300    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lxjxv\" (UniqueName: \"kubernetes.io/projected/d839bc6f-e7f5-496d-9e64-eacee63a95fd-kube-api-access-lxjxv\") pod \"d839bc6f-e7f5-496d-9e64-eacee63a95fd\" (UID: \"d839bc6f-e7f5-496d-9e64-eacee63a95fd\") "
Sep 18 20:27:33.892539 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:27:33.892360    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9ec8aa37-8e42-432f-860a-aae87a6a7c5f-home\") pod \"9ec8aa37-8e42-432f-860a-aae87a6a7c5f\" (UID: \"9ec8aa37-8e42-432f-860a-aae87a6a7c5f\") "
Sep 18 20:27:33.892539 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:27:33.892388    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d839bc6f-e7f5-496d-9e64-eacee63a95fd-tls-certs\") pod \"d839bc6f-e7f5-496d-9e64-eacee63a95fd\" (UID: \"d839bc6f-e7f5-496d-9e64-eacee63a95fd\") "
Sep 18 20:27:33.892539 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:27:33.892417    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vz4fx\" (UniqueName: \"kubernetes.io/projected/53c9b26c-7ae0-418a-9296-08fc8ad7b84e-kube-api-access-vz4fx\") pod \"53c9b26c-7ae0-418a-9296-08fc8ad7b84e\" (UID: \"53c9b26c-7ae0-418a-9296-08fc8ad7b84e\") "
Sep 18 20:27:33.892539 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:27:33.892438    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/53c9b26c-7ae0-418a-9296-08fc8ad7b84e-home\") pod \"53c9b26c-7ae0-418a-9296-08fc8ad7b84e\" (UID: \"53c9b26c-7ae0-418a-9296-08fc8ad7b84e\") "
Sep 18 20:27:33.892770 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:27:33.892579    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/53c9b26c-7ae0-418a-9296-08fc8ad7b84e-tmp-dir\") pod \"53c9b26c-7ae0-418a-9296-08fc8ad7b84e\" (UID: \"53c9b26c-7ae0-418a-9296-08fc8ad7b84e\") "
Sep 18 20:27:33.892770 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:27:33.892608    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9ec8aa37-8e42-432f-860a-aae87a6a7c5f-dshm\") pod \"9ec8aa37-8e42-432f-860a-aae87a6a7c5f\" (UID: \"9ec8aa37-8e42-432f-860a-aae87a6a7c5f\") "
Sep 18 20:27:33.892770 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:27:33.892646    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d839bc6f-e7f5-496d-9e64-eacee63a95fd-dshm\") pod \"d839bc6f-e7f5-496d-9e64-eacee63a95fd\" (UID: \"d839bc6f-e7f5-496d-9e64-eacee63a95fd\") "
Sep 18 20:27:33.892770 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:27:33.892687    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/53c9b26c-7ae0-418a-9296-08fc8ad7b84e-dshm\") pod \"53c9b26c-7ae0-418a-9296-08fc8ad7b84e\" (UID: \"53c9b26c-7ae0-418a-9296-08fc8ad7b84e\") "
Sep 18 20:27:33.892770 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:27:33.892750    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9ec8aa37-8e42-432f-860a-aae87a6a7c5f-home" (OuterVolumeSpecName: "home") pod "9ec8aa37-8e42-432f-860a-aae87a6a7c5f" (UID: "9ec8aa37-8e42-432f-860a-aae87a6a7c5f"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 20:27:33.892770 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:27:33.892757    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-m69q4\" (UniqueName: \"kubernetes.io/projected/9ec8aa37-8e42-432f-860a-aae87a6a7c5f-kube-api-access-m69q4\") pod \"9ec8aa37-8e42-432f-860a-aae87a6a7c5f\" (UID: \"9ec8aa37-8e42-432f-860a-aae87a6a7c5f\") "
Sep 18 20:27:33.893070 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:27:33.892764    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/53c9b26c-7ae0-418a-9296-08fc8ad7b84e-home" (OuterVolumeSpecName: "home") pod "53c9b26c-7ae0-418a-9296-08fc8ad7b84e" (UID: "53c9b26c-7ae0-418a-9296-08fc8ad7b84e"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 20:27:33.893070 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:27:33.892782    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d839bc6f-e7f5-496d-9e64-eacee63a95fd-home\") pod \"d839bc6f-e7f5-496d-9e64-eacee63a95fd\" (UID: \"d839bc6f-e7f5-496d-9e64-eacee63a95fd\") "
Sep 18 20:27:33.893070 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:27:33.892809    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d839bc6f-e7f5-496d-9e64-eacee63a95fd-tmp-dir\") pod \"d839bc6f-e7f5-496d-9e64-eacee63a95fd\" (UID: \"d839bc6f-e7f5-496d-9e64-eacee63a95fd\") "
Sep 18 20:27:33.893070 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:27:33.892835    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9ec8aa37-8e42-432f-860a-aae87a6a7c5f-model-cache\") pod \"9ec8aa37-8e42-432f-860a-aae87a6a7c5f\" (UID: \"9ec8aa37-8e42-432f-860a-aae87a6a7c5f\") "
Sep 18 20:27:33.893070 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:27:33.892876    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d839bc6f-e7f5-496d-9e64-eacee63a95fd-model-cache\") pod \"d839bc6f-e7f5-496d-9e64-eacee63a95fd\" (UID: \"d839bc6f-e7f5-496d-9e64-eacee63a95fd\") "
Sep 18 20:27:33.893070 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:27:33.892913    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9ec8aa37-8e42-432f-860a-aae87a6a7c5f-tls-certs\") pod \"9ec8aa37-8e42-432f-860a-aae87a6a7c5f\" (UID: \"9ec8aa37-8e42-432f-860a-aae87a6a7c5f\") "
Sep 18 20:27:33.893070 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:27:33.892950    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9ec8aa37-8e42-432f-860a-aae87a6a7c5f-tmp-dir\") pod \"9ec8aa37-8e42-432f-860a-aae87a6a7c5f\" (UID: \"9ec8aa37-8e42-432f-860a-aae87a6a7c5f\") "
Sep 18 20:27:33.893070 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:27:33.892973    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/53c9b26c-7ae0-418a-9296-08fc8ad7b84e-tls-certs\") pod \"53c9b26c-7ae0-418a-9296-08fc8ad7b84e\" (UID: \"53c9b26c-7ae0-418a-9296-08fc8ad7b84e\") "
Sep 18 20:27:33.893070 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:27:33.892974    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/53c9b26c-7ae0-418a-9296-08fc8ad7b84e-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "53c9b26c-7ae0-418a-9296-08fc8ad7b84e" (UID: "53c9b26c-7ae0-418a-9296-08fc8ad7b84e"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 20:27:33.893070 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:27:33.893002    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/53c9b26c-7ae0-418a-9296-08fc8ad7b84e-model-cache\") pod \"53c9b26c-7ae0-418a-9296-08fc8ad7b84e\" (UID: \"53c9b26c-7ae0-418a-9296-08fc8ad7b84e\") "
Sep 18 20:27:33.893575 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:27:33.893147    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d839bc6f-e7f5-496d-9e64-eacee63a95fd-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "d839bc6f-e7f5-496d-9e64-eacee63a95fd" (UID: "d839bc6f-e7f5-496d-9e64-eacee63a95fd"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 20:27:33.893575 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:27:33.893357    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/53c9b26c-7ae0-418a-9296-08fc8ad7b84e-model-cache" (OuterVolumeSpecName: "model-cache") pod "53c9b26c-7ae0-418a-9296-08fc8ad7b84e" (UID: "53c9b26c-7ae0-418a-9296-08fc8ad7b84e"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 20:27:33.893575 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:27:33.893439    2532 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9ec8aa37-8e42-432f-860a-aae87a6a7c5f-home\") on node \"ip-10-0-130-81.ec2.internal\" DevicePath \"\""
Sep 18 20:27:33.893575 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:27:33.893457    2532 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/53c9b26c-7ae0-418a-9296-08fc8ad7b84e-home\") on node \"ip-10-0-130-81.ec2.internal\" DevicePath \"\""
Sep 18 20:27:33.893575 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:27:33.893471    2532 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/53c9b26c-7ae0-418a-9296-08fc8ad7b84e-tmp-dir\") on node \"ip-10-0-130-81.ec2.internal\" DevicePath \"\""
Sep 18 20:27:33.893575 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:27:33.893501    2532 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-97rsf\" (UniqueName: \"kubernetes.io/projected/f1270a9f-d699-4dc7-a101-df212a401b81-kube-api-access-97rsf\") on node \"ip-10-0-130-81.ec2.internal\" DevicePath \"\""
Sep 18 20:27:33.893575 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:27:33.893516    2532 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f1270a9f-d699-4dc7-a101-df212a401b81-dshm\") on node \"ip-10-0-130-81.ec2.internal\" DevicePath \"\""
Sep 18 20:27:33.893575 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:27:33.893531    2532 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f1270a9f-d699-4dc7-a101-df212a401b81-home\") on node \"ip-10-0-130-81.ec2.internal\" DevicePath \"\""
Sep 18 20:27:33.893575 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:27:33.893543    2532 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d839bc6f-e7f5-496d-9e64-eacee63a95fd-tmp-dir\") on node \"ip-10-0-130-81.ec2.internal\" DevicePath \"\""
Sep 18 20:27:33.893575 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:27:33.893555    2532 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f1270a9f-d699-4dc7-a101-df212a401b81-tmp-dir\") on node \"ip-10-0-130-81.ec2.internal\" DevicePath \"\""
Sep 18 20:27:33.893575 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:27:33.893567    2532 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f1270a9f-d699-4dc7-a101-df212a401b81-model-cache\") on node \"ip-10-0-130-81.ec2.internal\" DevicePath \"\""
Sep 18 20:27:33.893575 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:27:33.893579    2532 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/53c9b26c-7ae0-418a-9296-08fc8ad7b84e-model-cache\") on node \"ip-10-0-130-81.ec2.internal\" DevicePath \"\""
Sep 18 20:27:33.894176 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:27:33.893592    2532 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f1270a9f-d699-4dc7-a101-df212a401b81-tls-certs\") on node \"ip-10-0-130-81.ec2.internal\" DevicePath \"\""
Sep 18 20:27:33.894176 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:27:33.893610    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9ec8aa37-8e42-432f-860a-aae87a6a7c5f-model-cache" (OuterVolumeSpecName: "model-cache") pod "9ec8aa37-8e42-432f-860a-aae87a6a7c5f" (UID: "9ec8aa37-8e42-432f-860a-aae87a6a7c5f"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 20:27:33.894176 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:27:33.893805    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d839bc6f-e7f5-496d-9e64-eacee63a95fd-home" (OuterVolumeSpecName: "home") pod "d839bc6f-e7f5-496d-9e64-eacee63a95fd" (UID: "d839bc6f-e7f5-496d-9e64-eacee63a95fd"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 20:27:33.894176 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:27:33.893976    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d839bc6f-e7f5-496d-9e64-eacee63a95fd-model-cache" (OuterVolumeSpecName: "model-cache") pod "d839bc6f-e7f5-496d-9e64-eacee63a95fd" (UID: "d839bc6f-e7f5-496d-9e64-eacee63a95fd"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 20:27:33.894176 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:27:33.894121    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9ec8aa37-8e42-432f-860a-aae87a6a7c5f-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "9ec8aa37-8e42-432f-860a-aae87a6a7c5f" (UID: "9ec8aa37-8e42-432f-860a-aae87a6a7c5f"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 20:27:33.896422 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:27:33.896395    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9ec8aa37-8e42-432f-860a-aae87a6a7c5f-kube-api-access-m69q4" (OuterVolumeSpecName: "kube-api-access-m69q4") pod "9ec8aa37-8e42-432f-860a-aae87a6a7c5f" (UID: "9ec8aa37-8e42-432f-860a-aae87a6a7c5f"). InnerVolumeSpecName "kube-api-access-m69q4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 20:27:33.896812 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:27:33.896790    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9ec8aa37-8e42-432f-860a-aae87a6a7c5f-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "9ec8aa37-8e42-432f-860a-aae87a6a7c5f" (UID: "9ec8aa37-8e42-432f-860a-aae87a6a7c5f"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 20:27:33.896913 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:27:33.896832    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/53c9b26c-7ae0-418a-9296-08fc8ad7b84e-dshm" (OuterVolumeSpecName: "dshm") pod "53c9b26c-7ae0-418a-9296-08fc8ad7b84e" (UID: "53c9b26c-7ae0-418a-9296-08fc8ad7b84e"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 20:27:33.896913 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:27:33.896887    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d839bc6f-e7f5-496d-9e64-eacee63a95fd-dshm" (OuterVolumeSpecName: "dshm") pod "d839bc6f-e7f5-496d-9e64-eacee63a95fd" (UID: "d839bc6f-e7f5-496d-9e64-eacee63a95fd"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 20:27:33.897034 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:27:33.896908    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/53c9b26c-7ae0-418a-9296-08fc8ad7b84e-kube-api-access-vz4fx" (OuterVolumeSpecName: "kube-api-access-vz4fx") pod "53c9b26c-7ae0-418a-9296-08fc8ad7b84e" (UID: "53c9b26c-7ae0-418a-9296-08fc8ad7b84e"). InnerVolumeSpecName "kube-api-access-vz4fx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 20:27:33.897607 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:27:33.897579    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/53c9b26c-7ae0-418a-9296-08fc8ad7b84e-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "53c9b26c-7ae0-418a-9296-08fc8ad7b84e" (UID: "53c9b26c-7ae0-418a-9296-08fc8ad7b84e"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 20:27:33.897673 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:27:33.897611    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9ec8aa37-8e42-432f-860a-aae87a6a7c5f-dshm" (OuterVolumeSpecName: "dshm") pod "9ec8aa37-8e42-432f-860a-aae87a6a7c5f" (UID: "9ec8aa37-8e42-432f-860a-aae87a6a7c5f"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 20:27:33.897839 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:27:33.897806    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d839bc6f-e7f5-496d-9e64-eacee63a95fd-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "d839bc6f-e7f5-496d-9e64-eacee63a95fd" (UID: "d839bc6f-e7f5-496d-9e64-eacee63a95fd"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 20:27:33.898243 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:27:33.898225    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d839bc6f-e7f5-496d-9e64-eacee63a95fd-kube-api-access-lxjxv" (OuterVolumeSpecName: "kube-api-access-lxjxv") pod "d839bc6f-e7f5-496d-9e64-eacee63a95fd" (UID: "d839bc6f-e7f5-496d-9e64-eacee63a95fd"). InnerVolumeSpecName "kube-api-access-lxjxv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 20:27:33.994695 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:27:33.994600    2532 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9ec8aa37-8e42-432f-860a-aae87a6a7c5f-tls-certs\") on node \"ip-10-0-130-81.ec2.internal\" DevicePath \"\""
Sep 18 20:27:33.994695 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:27:33.994633    2532 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9ec8aa37-8e42-432f-860a-aae87a6a7c5f-tmp-dir\") on node \"ip-10-0-130-81.ec2.internal\" DevicePath \"\""
Sep 18 20:27:33.994695 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:27:33.994645    2532 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/53c9b26c-7ae0-418a-9296-08fc8ad7b84e-tls-certs\") on node \"ip-10-0-130-81.ec2.internal\" DevicePath \"\""
Sep 18 20:27:33.994695 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:27:33.994656    2532 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lxjxv\" (UniqueName: \"kubernetes.io/projected/d839bc6f-e7f5-496d-9e64-eacee63a95fd-kube-api-access-lxjxv\") on node \"ip-10-0-130-81.ec2.internal\" DevicePath \"\""
Sep 18 20:27:33.994695 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:27:33.994668    2532 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d839bc6f-e7f5-496d-9e64-eacee63a95fd-tls-certs\") on node \"ip-10-0-130-81.ec2.internal\" DevicePath \"\""
Sep 18 20:27:33.994695 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:27:33.994682    2532 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vz4fx\" (UniqueName: \"kubernetes.io/projected/53c9b26c-7ae0-418a-9296-08fc8ad7b84e-kube-api-access-vz4fx\") on node \"ip-10-0-130-81.ec2.internal\" DevicePath \"\""
Sep 18 20:27:33.994695 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:27:33.994693    2532 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9ec8aa37-8e42-432f-860a-aae87a6a7c5f-dshm\") on node \"ip-10-0-130-81.ec2.internal\" DevicePath \"\""
Sep 18 20:27:33.995032 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:27:33.994704    2532 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d839bc6f-e7f5-496d-9e64-eacee63a95fd-dshm\") on node \"ip-10-0-130-81.ec2.internal\" DevicePath \"\""
Sep 18 20:27:33.995032 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:27:33.994714    2532 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/53c9b26c-7ae0-418a-9296-08fc8ad7b84e-dshm\") on node \"ip-10-0-130-81.ec2.internal\" DevicePath \"\""
Sep 18 20:27:33.995032 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:27:33.994725    2532 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m69q4\" (UniqueName: \"kubernetes.io/projected/9ec8aa37-8e42-432f-860a-aae87a6a7c5f-kube-api-access-m69q4\") on node \"ip-10-0-130-81.ec2.internal\" DevicePath \"\""
Sep 18 20:27:33.995032 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:27:33.994735    2532 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d839bc6f-e7f5-496d-9e64-eacee63a95fd-home\") on node \"ip-10-0-130-81.ec2.internal\" DevicePath \"\""
Sep 18 20:27:33.995032 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:27:33.994745    2532 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9ec8aa37-8e42-432f-860a-aae87a6a7c5f-model-cache\") on node \"ip-10-0-130-81.ec2.internal\" DevicePath \"\""
Sep 18 20:27:33.995032 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:27:33.994755    2532 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d839bc6f-e7f5-496d-9e64-eacee63a95fd-model-cache\") on node \"ip-10-0-130-81.ec2.internal\" DevicePath \"\""
Sep 18 20:27:34.356857 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:27:34.356825    2532 generic.go:356] "Generic (PLEG): container finished" podID="53c9b26c-7ae0-418a-9296-08fc8ad7b84e" containerID="3070bd4e0384dae04a9a7b9df2da95f3c98c15597f4482ea27e79f795dd8d2bc" exitCode=0
Sep 18 20:27:34.357293 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:27:34.356866    2532 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-116778/premium-w0-116778-kserve-9b5787766-tth7r"
Sep 18 20:27:34.357293 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:27:34.356912    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-116778/premium-w0-116778-kserve-9b5787766-tth7r" event={"ID":"53c9b26c-7ae0-418a-9296-08fc8ad7b84e","Type":"ContainerDied","Data":"3070bd4e0384dae04a9a7b9df2da95f3c98c15597f4482ea27e79f795dd8d2bc"}
Sep 18 20:27:34.357293 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:27:34.356957    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-116778/premium-w0-116778-kserve-9b5787766-tth7r" event={"ID":"53c9b26c-7ae0-418a-9296-08fc8ad7b84e","Type":"ContainerDied","Data":"e516d8e81f78c5dff4d6f26bd9fa3dae5d8c628f4071de983122155fac9c2f8d"}
Sep 18 20:27:34.357293 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:27:34.356974    2532 scope.go:117] "RemoveContainer" containerID="3070bd4e0384dae04a9a7b9df2da95f3c98c15597f4482ea27e79f795dd8d2bc"
Sep 18 20:27:34.358456 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:27:34.358427    2532 generic.go:356] "Generic (PLEG): container finished" podID="d839bc6f-e7f5-496d-9e64-eacee63a95fd" containerID="0eb85f11fccbb6c2ed97ecd27fdf6120fdd7c602d3eec60dae71e112ff48d0c0" exitCode=0
Sep 18 20:27:34.358456 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:27:34.358473    2532 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-116778/simulator-w0-116778-kserve-8f94f4bdc-4wbtx"
Sep 18 20:27:34.358654 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:27:34.358528    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-116778/simulator-w0-116778-kserve-8f94f4bdc-4wbtx" event={"ID":"d839bc6f-e7f5-496d-9e64-eacee63a95fd","Type":"ContainerDied","Data":"0eb85f11fccbb6c2ed97ecd27fdf6120fdd7c602d3eec60dae71e112ff48d0c0"}
Sep 18 20:27:34.358654 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:27:34.358564    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-116778/simulator-w0-116778-kserve-8f94f4bdc-4wbtx" event={"ID":"d839bc6f-e7f5-496d-9e64-eacee63a95fd","Type":"ContainerDied","Data":"8a4b4e8eac984c72c2ba948b489707433aef556e31d67e74df2d29200aefc04f"}
Sep 18 20:27:34.360081 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:27:34.360062    2532 generic.go:356] "Generic (PLEG): container finished" podID="f1270a9f-d699-4dc7-a101-df212a401b81" containerID="89925065f14a18f99d444f2c9e2b34d146713b2c14bee410b170f55b874eebe9" exitCode=0
Sep 18 20:27:34.360182 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:27:34.360103    2532 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-116778/distinct-w0-116778-kserve-5f6955bd97-qf5vt"
Sep 18 20:27:34.360182 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:27:34.360141    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-116778/distinct-w0-116778-kserve-5f6955bd97-qf5vt" event={"ID":"f1270a9f-d699-4dc7-a101-df212a401b81","Type":"ContainerDied","Data":"89925065f14a18f99d444f2c9e2b34d146713b2c14bee410b170f55b874eebe9"}
Sep 18 20:27:34.360182 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:27:34.360172    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-116778/distinct-w0-116778-kserve-5f6955bd97-qf5vt" event={"ID":"f1270a9f-d699-4dc7-a101-df212a401b81","Type":"ContainerDied","Data":"eb16e34a15c943bf4acbea4ca42796ecf0dc5762f234f87079461083389b0b63"}
Sep 18 20:27:34.361640 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:27:34.361619    2532 generic.go:356] "Generic (PLEG): container finished" podID="9ec8aa37-8e42-432f-860a-aae87a6a7c5f" containerID="ba4f8a04c73b1f1a0cecdeee5d67c590b18134f62221d41af9652f74c76dbbfe" exitCode=0
Sep 18 20:27:34.361733 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:27:34.361653    2532 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-116778/embedding-w0-116778-kserve-6bd94f5698-n429v"
Sep 18 20:27:34.361733 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:27:34.361662    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-116778/embedding-w0-116778-kserve-6bd94f5698-n429v" event={"ID":"9ec8aa37-8e42-432f-860a-aae87a6a7c5f","Type":"ContainerDied","Data":"ba4f8a04c73b1f1a0cecdeee5d67c590b18134f62221d41af9652f74c76dbbfe"}
Sep 18 20:27:34.361733 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:27:34.361685    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-116778/embedding-w0-116778-kserve-6bd94f5698-n429v" event={"ID":"9ec8aa37-8e42-432f-860a-aae87a6a7c5f","Type":"ContainerDied","Data":"39c19682628ae4d1b542c66fffad9637902c78ed1988481daf1193ae4aebc9aa"}
Sep 18 20:27:34.369826 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:27:34.369790    2532 scope.go:117] "RemoveContainer" containerID="3070bd4e0384dae04a9a7b9df2da95f3c98c15597f4482ea27e79f795dd8d2bc"
Sep 18 20:27:34.370096 ip-10-0-130-81 kubenswrapper[2532]: E0918 20:27:34.370074    2532 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3070bd4e0384dae04a9a7b9df2da95f3c98c15597f4482ea27e79f795dd8d2bc\": container with ID starting with 3070bd4e0384dae04a9a7b9df2da95f3c98c15597f4482ea27e79f795dd8d2bc not found: ID does not exist" containerID="3070bd4e0384dae04a9a7b9df2da95f3c98c15597f4482ea27e79f795dd8d2bc"
Sep 18 20:27:34.370168 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:27:34.370104    2532 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3070bd4e0384dae04a9a7b9df2da95f3c98c15597f4482ea27e79f795dd8d2bc"} err="failed to get container status \"3070bd4e0384dae04a9a7b9df2da95f3c98c15597f4482ea27e79f795dd8d2bc\": rpc error: code = NotFound desc = could not find container \"3070bd4e0384dae04a9a7b9df2da95f3c98c15597f4482ea27e79f795dd8d2bc\": container with ID starting with 3070bd4e0384dae04a9a7b9df2da95f3c98c15597f4482ea27e79f795dd8d2bc not found: ID does not exist"
Sep 18 20:27:34.370168 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:27:34.370122    2532 scope.go:117] "RemoveContainer" containerID="0eb85f11fccbb6c2ed97ecd27fdf6120fdd7c602d3eec60dae71e112ff48d0c0"
Sep 18 20:27:34.378374 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:27:34.378354    2532 scope.go:117] "RemoveContainer" containerID="0eb85f11fccbb6c2ed97ecd27fdf6120fdd7c602d3eec60dae71e112ff48d0c0"
Sep 18 20:27:34.378690 ip-10-0-130-81 kubenswrapper[2532]: E0918 20:27:34.378663    2532 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0eb85f11fccbb6c2ed97ecd27fdf6120fdd7c602d3eec60dae71e112ff48d0c0\": container with ID starting with 0eb85f11fccbb6c2ed97ecd27fdf6120fdd7c602d3eec60dae71e112ff48d0c0 not found: ID does not exist" containerID="0eb85f11fccbb6c2ed97ecd27fdf6120fdd7c602d3eec60dae71e112ff48d0c0"
Sep 18 20:27:34.378791 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:27:34.378695    2532 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0eb85f11fccbb6c2ed97ecd27fdf6120fdd7c602d3eec60dae71e112ff48d0c0"} err="failed to get container status \"0eb85f11fccbb6c2ed97ecd27fdf6120fdd7c602d3eec60dae71e112ff48d0c0\": rpc error: code = NotFound desc = could not find container \"0eb85f11fccbb6c2ed97ecd27fdf6120fdd7c602d3eec60dae71e112ff48d0c0\": container with ID starting with 0eb85f11fccbb6c2ed97ecd27fdf6120fdd7c602d3eec60dae71e112ff48d0c0 not found: ID does not exist"
Sep 18 20:27:34.378791 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:27:34.378711    2532 scope.go:117] "RemoveContainer" containerID="89925065f14a18f99d444f2c9e2b34d146713b2c14bee410b170f55b874eebe9"
Sep 18 20:27:34.383122 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:27:34.383102    2532 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-116778/premium-w0-116778-kserve-9b5787766-tth7r"]
Sep 18 20:27:34.386805 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:27:34.386780    2532 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w0-116778/premium-w0-116778-kserve-9b5787766-tth7r"]
Sep 18 20:27:34.388763 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:27:34.388745    2532 scope.go:117] "RemoveContainer" containerID="89925065f14a18f99d444f2c9e2b34d146713b2c14bee410b170f55b874eebe9"
Sep 18 20:27:34.389020 ip-10-0-130-81 kubenswrapper[2532]: E0918 20:27:34.389001    2532 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"89925065f14a18f99d444f2c9e2b34d146713b2c14bee410b170f55b874eebe9\": container with ID starting with 89925065f14a18f99d444f2c9e2b34d146713b2c14bee410b170f55b874eebe9 not found: ID does not exist" containerID="89925065f14a18f99d444f2c9e2b34d146713b2c14bee410b170f55b874eebe9"
Sep 18 20:27:34.389094 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:27:34.389029    2532 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"89925065f14a18f99d444f2c9e2b34d146713b2c14bee410b170f55b874eebe9"} err="failed to get container status \"89925065f14a18f99d444f2c9e2b34d146713b2c14bee410b170f55b874eebe9\": rpc error: code = NotFound desc = could not find container \"89925065f14a18f99d444f2c9e2b34d146713b2c14bee410b170f55b874eebe9\": container with ID starting with 89925065f14a18f99d444f2c9e2b34d146713b2c14bee410b170f55b874eebe9 not found: ID does not exist"
Sep 18 20:27:34.389094 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:27:34.389051    2532 scope.go:117] "RemoveContainer" containerID="ba4f8a04c73b1f1a0cecdeee5d67c590b18134f62221d41af9652f74c76dbbfe"
Sep 18 20:27:34.396918 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:27:34.396901    2532 scope.go:117] "RemoveContainer" containerID="ba4f8a04c73b1f1a0cecdeee5d67c590b18134f62221d41af9652f74c76dbbfe"
Sep 18 20:27:34.397173 ip-10-0-130-81 kubenswrapper[2532]: E0918 20:27:34.397155    2532 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ba4f8a04c73b1f1a0cecdeee5d67c590b18134f62221d41af9652f74c76dbbfe\": container with ID starting with ba4f8a04c73b1f1a0cecdeee5d67c590b18134f62221d41af9652f74c76dbbfe not found: ID does not exist" containerID="ba4f8a04c73b1f1a0cecdeee5d67c590b18134f62221d41af9652f74c76dbbfe"
Sep 18 20:27:34.397246 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:27:34.397180    2532 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ba4f8a04c73b1f1a0cecdeee5d67c590b18134f62221d41af9652f74c76dbbfe"} err="failed to get container status \"ba4f8a04c73b1f1a0cecdeee5d67c590b18134f62221d41af9652f74c76dbbfe\": rpc error: code = NotFound desc = could not find container \"ba4f8a04c73b1f1a0cecdeee5d67c590b18134f62221d41af9652f74c76dbbfe\": container with ID starting with ba4f8a04c73b1f1a0cecdeee5d67c590b18134f62221d41af9652f74c76dbbfe not found: ID does not exist"
Sep 18 20:27:34.404343 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:27:34.404325    2532 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-116778/embedding-w0-116778-kserve-6bd94f5698-n429v"]
Sep 18 20:27:34.406429 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:27:34.406412    2532 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w0-116778/embedding-w0-116778-kserve-6bd94f5698-n429v"]
Sep 18 20:27:34.416950 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:27:34.416931    2532 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-116778/simulator-w0-116778-kserve-8f94f4bdc-4wbtx"]
Sep 18 20:27:34.424056 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:27:34.424026    2532 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w0-116778/simulator-w0-116778-kserve-8f94f4bdc-4wbtx"]
Sep 18 20:27:34.431406 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:27:34.431381    2532 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-116778/distinct-w0-116778-kserve-5f6955bd97-qf5vt"]
Sep 18 20:27:34.434941 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:27:34.434919    2532 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w0-116778/distinct-w0-116778-kserve-5f6955bd97-qf5vt"]
Sep 18 20:27:35.274580 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:27:35.274545    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:27:35.373675 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:27:35.373649    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:27:35.515298 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:27:35.515268    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:27:36.306630 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:27:36.306597    2532 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="53c9b26c-7ae0-418a-9296-08fc8ad7b84e" path="/var/lib/kubelet/pods/53c9b26c-7ae0-418a-9296-08fc8ad7b84e/volumes"
Sep 18 20:27:36.307009 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:27:36.306996    2532 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9ec8aa37-8e42-432f-860a-aae87a6a7c5f" path="/var/lib/kubelet/pods/9ec8aa37-8e42-432f-860a-aae87a6a7c5f/volumes"
Sep 18 20:27:36.307359 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:27:36.307347    2532 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d839bc6f-e7f5-496d-9e64-eacee63a95fd" path="/var/lib/kubelet/pods/d839bc6f-e7f5-496d-9e64-eacee63a95fd/volumes"
Sep 18 20:27:36.308096 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:27:36.308079    2532 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f1270a9f-d699-4dc7-a101-df212a401b81" path="/var/lib/kubelet/pods/f1270a9f-d699-4dc7-a101-df212a401b81/volumes"
Sep 18 20:27:43.008199 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:27:43.008165    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:27:43.228420 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:27:43.228384    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:27:47.005312 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:27:47.005278    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:27:54.805567 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:27:54.805482    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:27:55.196905 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:27:55.196823    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:27:55.608412 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:27:55.608377    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:27:55.723917 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:27:55.723876    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:27:55.822466 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:27:55.822440    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:27:55.922690 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:27:55.922609    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:27:56.170937 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:27:56.170908    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:27:56.990788 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:27:56.990756    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-vptdl"]
Sep 18 20:28:01.664807 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:28:01.664772    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:28:04.119470 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:28:04.119436    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:28:06.771942 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:28:06.771909    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:28:06.871631 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:28:06.871599    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:28:07.946509 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:28:07.946451    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:28:09.840138 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:28:09.840106    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:28:09.940710 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:28:09.940679    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:28:10.169705 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:28:10.169614    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:28:10.458830 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:28:10.458802    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:28:10.708646 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:28:10.708611    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:28:12.691692 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:28:12.691660    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:28:13.201715 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:28:13.201679    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-vptdl"]
Sep 18 20:28:13.345568 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:28:13.345512    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:28:13.586586 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:28:13.586550    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:28:13.689084 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:28:13.689049    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:28:14.262027 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:28:14.261994    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:28:14.912367 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:28:14.912335    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:28:15.432405 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:28:15.432356    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:28:16.218002 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:28:16.217970    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:28:16.833145 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:28:16.833112    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:28:16.891320 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:28:16.891277    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:28:17.251369 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:28:17.251331    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:28:17.353309 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:28:17.353275    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:28:29.012532 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:28:29.012497    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-vptdl"]
Sep 18 20:28:34.140814 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:28:34.140776    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:28:34.199316 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:28:34.199284    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:28:38.049513 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:28:38.049461    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:28:38.805545 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:28:38.805471    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:28:40.094399 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:28:40.094368    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:28:40.805917 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:28:40.805886    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:28:41.495755 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:28:41.495719    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:28:41.754714 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:28:41.754633    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:28:42.089744 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:28:42.089663    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:28:42.749213 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:28:42.749179    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:28:43.508195 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:28:43.508165    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:28:43.626120 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:28:43.626088    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:28:44.050292 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:28:44.050259    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:28:44.365158 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:28:44.365077    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:28:44.456478 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:28:44.456444    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:28:47.534862 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:28:47.534811    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:28:47.845958 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:28:47.845874    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:28:47.937247 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:28:47.937206    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:28:48.343349 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:28:48.343321    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:28:49.135602 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:28:49.135561    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:28:49.819713 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:28:49.819677    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:28:50.200543 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:28:50.200450    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-vptdl"]
Sep 18 20:28:50.537179 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:28:50.537145    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:28:51.316124 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:28:51.316091    2532 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-102c83-7c4d97c84b-jlrwh"]
Sep 18 20:28:51.316540 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:28:51.316525    2532 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="53c9b26c-7ae0-418a-9296-08fc8ad7b84e" containerName="main"
Sep 18 20:28:51.316597 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:28:51.316542    2532 state_mem.go:107] "Deleted CPUSet assignment" podUID="53c9b26c-7ae0-418a-9296-08fc8ad7b84e" containerName="main"
Sep 18 20:28:51.316597 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:28:51.316552    2532 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f1270a9f-d699-4dc7-a101-df212a401b81" containerName="main"
Sep 18 20:28:51.316597 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:28:51.316557    2532 state_mem.go:107] "Deleted CPUSet assignment" podUID="f1270a9f-d699-4dc7-a101-df212a401b81" containerName="main"
Sep 18 20:28:51.316597 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:28:51.316572    2532 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f719fcf6-cc6a-4e98-883a-00270a9985c0" containerName="maas-api"
Sep 18 20:28:51.316597 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:28:51.316578    2532 state_mem.go:107] "Deleted CPUSet assignment" podUID="f719fcf6-cc6a-4e98-883a-00270a9985c0" containerName="maas-api"
Sep 18 20:28:51.316597 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:28:51.316585    2532 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9ec8aa37-8e42-432f-860a-aae87a6a7c5f" containerName="main"
Sep 18 20:28:51.316597 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:28:51.316590    2532 state_mem.go:107] "Deleted CPUSet assignment" podUID="9ec8aa37-8e42-432f-860a-aae87a6a7c5f" containerName="main"
Sep 18 20:28:51.316597 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:28:51.316600    2532 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d839bc6f-e7f5-496d-9e64-eacee63a95fd" containerName="main"
Sep 18 20:28:51.316868 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:28:51.316605    2532 state_mem.go:107] "Deleted CPUSet assignment" podUID="d839bc6f-e7f5-496d-9e64-eacee63a95fd" containerName="main"
Sep 18 20:28:51.316868 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:28:51.316667    2532 memory_manager.go:368] "RemoveStaleState removing state" podUID="d839bc6f-e7f5-496d-9e64-eacee63a95fd" containerName="main"
Sep 18 20:28:51.316868 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:28:51.316677    2532 memory_manager.go:368] "RemoveStaleState removing state" podUID="53c9b26c-7ae0-418a-9296-08fc8ad7b84e" containerName="main"
Sep 18 20:28:51.316868 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:28:51.316684    2532 memory_manager.go:368] "RemoveStaleState removing state" podUID="f719fcf6-cc6a-4e98-883a-00270a9985c0" containerName="maas-api"
Sep 18 20:28:51.316868 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:28:51.316690    2532 memory_manager.go:368] "RemoveStaleState removing state" podUID="9ec8aa37-8e42-432f-860a-aae87a6a7c5f" containerName="main"
Sep 18 20:28:51.316868 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:28:51.316696    2532 memory_manager.go:368] "RemoveStaleState removing state" podUID="f1270a9f-d699-4dc7-a101-df212a401b81" containerName="main"
Sep 18 20:28:51.319746 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:28:51.319729    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-102c83-7c4d97c84b-jlrwh"
Sep 18 20:28:51.322612 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:28:51.322588    2532 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-b-102c83-serving-cert\""
Sep 18 20:28:51.330304 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:28:51.330282    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c43f0b3f-b95f-4e4a-81dc-2d6c248a41fa-maas-api-tls\") pod \"maas-api-e2e-api-b-102c83-7c4d97c84b-jlrwh\" (UID: \"c43f0b3f-b95f-4e4a-81dc-2d6c248a41fa\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-102c83-7c4d97c84b-jlrwh"
Sep 18 20:28:51.330399 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:28:51.330331    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c43f0b3f-b95f-4e4a-81dc-2d6c248a41fa-openshift-service-ca\") pod \"maas-api-e2e-api-b-102c83-7c4d97c84b-jlrwh\" (UID: \"c43f0b3f-b95f-4e4a-81dc-2d6c248a41fa\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-102c83-7c4d97c84b-jlrwh"
Sep 18 20:28:51.330455 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:28:51.330410    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rbpp2\" (UniqueName: \"kubernetes.io/projected/c43f0b3f-b95f-4e4a-81dc-2d6c248a41fa-kube-api-access-rbpp2\") pod \"maas-api-e2e-api-b-102c83-7c4d97c84b-jlrwh\" (UID: \"c43f0b3f-b95f-4e4a-81dc-2d6c248a41fa\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-102c83-7c4d97c84b-jlrwh"
Sep 18 20:28:51.330455 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:28:51.330438    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c43f0b3f-b95f-4e4a-81dc-2d6c248a41fa-metrics-certs\") pod \"maas-api-e2e-api-b-102c83-7c4d97c84b-jlrwh\" (UID: \"c43f0b3f-b95f-4e4a-81dc-2d6c248a41fa\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-102c83-7c4d97c84b-jlrwh"
Sep 18 20:28:51.335261 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:28:51.335238    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-102c83-7c4d97c84b-jlrwh"]
Sep 18 20:28:51.361692 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:28:51.361665    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:28:51.431826 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:28:51.431789    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c43f0b3f-b95f-4e4a-81dc-2d6c248a41fa-openshift-service-ca\") pod \"maas-api-e2e-api-b-102c83-7c4d97c84b-jlrwh\" (UID: \"c43f0b3f-b95f-4e4a-81dc-2d6c248a41fa\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-102c83-7c4d97c84b-jlrwh"
Sep 18 20:28:51.432013 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:28:51.431843    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rbpp2\" (UniqueName: \"kubernetes.io/projected/c43f0b3f-b95f-4e4a-81dc-2d6c248a41fa-kube-api-access-rbpp2\") pod \"maas-api-e2e-api-b-102c83-7c4d97c84b-jlrwh\" (UID: \"c43f0b3f-b95f-4e4a-81dc-2d6c248a41fa\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-102c83-7c4d97c84b-jlrwh"
Sep 18 20:28:51.432013 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:28:51.431964    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c43f0b3f-b95f-4e4a-81dc-2d6c248a41fa-metrics-certs\") pod \"maas-api-e2e-api-b-102c83-7c4d97c84b-jlrwh\" (UID: \"c43f0b3f-b95f-4e4a-81dc-2d6c248a41fa\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-102c83-7c4d97c84b-jlrwh"
Sep 18 20:28:51.432117 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:28:51.432102    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c43f0b3f-b95f-4e4a-81dc-2d6c248a41fa-maas-api-tls\") pod \"maas-api-e2e-api-b-102c83-7c4d97c84b-jlrwh\" (UID: \"c43f0b3f-b95f-4e4a-81dc-2d6c248a41fa\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-102c83-7c4d97c84b-jlrwh"
Sep 18 20:28:51.432568 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:28:51.432532    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c43f0b3f-b95f-4e4a-81dc-2d6c248a41fa-openshift-service-ca\") pod \"maas-api-e2e-api-b-102c83-7c4d97c84b-jlrwh\" (UID: \"c43f0b3f-b95f-4e4a-81dc-2d6c248a41fa\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-102c83-7c4d97c84b-jlrwh"
Sep 18 20:28:51.434438 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:28:51.434418    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c43f0b3f-b95f-4e4a-81dc-2d6c248a41fa-maas-api-tls\") pod \"maas-api-e2e-api-b-102c83-7c4d97c84b-jlrwh\" (UID: \"c43f0b3f-b95f-4e4a-81dc-2d6c248a41fa\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-102c83-7c4d97c84b-jlrwh"
Sep 18 20:28:51.434609 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:28:51.434590    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c43f0b3f-b95f-4e4a-81dc-2d6c248a41fa-metrics-certs\") pod \"maas-api-e2e-api-b-102c83-7c4d97c84b-jlrwh\" (UID: \"c43f0b3f-b95f-4e4a-81dc-2d6c248a41fa\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-102c83-7c4d97c84b-jlrwh"
Sep 18 20:28:51.441672 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:28:51.441647    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rbpp2\" (UniqueName: \"kubernetes.io/projected/c43f0b3f-b95f-4e4a-81dc-2d6c248a41fa-kube-api-access-rbpp2\") pod \"maas-api-e2e-api-b-102c83-7c4d97c84b-jlrwh\" (UID: \"c43f0b3f-b95f-4e4a-81dc-2d6c248a41fa\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-102c83-7c4d97c84b-jlrwh"
Sep 18 20:28:51.462422 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:28:51.462396    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:28:51.630931 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:28:51.630829    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-102c83-7c4d97c84b-jlrwh"
Sep 18 20:28:51.875089 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:28:51.875063    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:28:51.903957 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:28:51.903929    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-102c83-7c4d97c84b-jlrwh"]
Sep 18 20:28:51.906169 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:28:51.906140    2532 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc43f0b3f_b95f_4e4a_81dc_2d6c248a41fa.slice/crio-adff5afddd978ae168268dde05da45002046a33849cdfdf9b1d531a17183671e WatchSource:0}: Error finding container adff5afddd978ae168268dde05da45002046a33849cdfdf9b1d531a17183671e: Status 404 returned error can't find the container with id adff5afddd978ae168268dde05da45002046a33849cdfdf9b1d531a17183671e
Sep 18 20:28:51.907702 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:28:51.907685    2532 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 18 20:28:51.958831 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:28:51.958801    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:28:52.806779 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:28:52.806743    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-102c83-7c4d97c84b-jlrwh" event={"ID":"c43f0b3f-b95f-4e4a-81dc-2d6c248a41fa","Type":"ContainerStarted","Data":"69cd719fa7c93d6b4e36a06787aff366720cc7acb4dd363570fb99634f0d45f7"}
Sep 18 20:28:52.807171 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:28:52.806798    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-102c83-7c4d97c84b-jlrwh" event={"ID":"c43f0b3f-b95f-4e4a-81dc-2d6c248a41fa","Type":"ContainerStarted","Data":"adff5afddd978ae168268dde05da45002046a33849cdfdf9b1d531a17183671e"}
Sep 18 20:28:52.807171 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:28:52.806826    2532 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-102c83-7c4d97c84b-jlrwh"
Sep 18 20:28:52.825826 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:28:52.825780    2532 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-102c83-7c4d97c84b-jlrwh" podStartSLOduration=1.53804244 podStartE2EDuration="1.825765129s" totalImagesPullingTime="287.722689ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 20:28:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 20:28:52.822531684 +0000 UTC m=+1664.587921085" watchObservedRunningTime="2026-09-18 20:28:52.825765129 +0000 UTC m=+1664.591154527"
Sep 18 20:28:58.818110 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:28:58.818078    2532 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-102c83-7c4d97c84b-jlrwh"
Sep 18 20:29:03.374331 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:29:03.374291    2532 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-8719-56a598"]
Sep 18 20:29:03.377709 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:29:03.377690    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-8719-56a598"
Sep 18 20:29:03.380869 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:29:03.380842    2532 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-pqc57\""
Sep 18 20:29:03.384736 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:29:03.384713    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-8719-56a598"]
Sep 18 20:29:03.450956 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:29:03.450907    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-52jlc\" (UniqueName: \"kubernetes.io/projected/a49843a5-8098-45c8-926b-f39e46182606-kube-api-access-52jlc\") pod \"test-curl-8719-56a598\" (UID: \"a49843a5-8098-45c8-926b-f39e46182606\") " pod="opendatahub/test-curl-8719-56a598"
Sep 18 20:29:03.551755 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:29:03.551714    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-52jlc\" (UniqueName: \"kubernetes.io/projected/a49843a5-8098-45c8-926b-f39e46182606-kube-api-access-52jlc\") pod \"test-curl-8719-56a598\" (UID: \"a49843a5-8098-45c8-926b-f39e46182606\") " pod="opendatahub/test-curl-8719-56a598"
Sep 18 20:29:03.560805 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:29:03.560780    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-52jlc\" (UniqueName: \"kubernetes.io/projected/a49843a5-8098-45c8-926b-f39e46182606-kube-api-access-52jlc\") pod \"test-curl-8719-56a598\" (UID: \"a49843a5-8098-45c8-926b-f39e46182606\") " pod="opendatahub/test-curl-8719-56a598"
Sep 18 20:29:03.688277 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:29:03.688197    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-8719-56a598"
Sep 18 20:29:03.960615 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:29:03.960590    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-8719-56a598"]
Sep 18 20:29:03.962717 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:29:03.962688    2532 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda49843a5_8098_45c8_926b_f39e46182606.slice/crio-4603d2a6ff8b97a92a75b90bde148096177e6da9a9985c3f6b069d6818de34e4 WatchSource:0}: Error finding container 4603d2a6ff8b97a92a75b90bde148096177e6da9a9985c3f6b069d6818de34e4: Status 404 returned error can't find the container with id 4603d2a6ff8b97a92a75b90bde148096177e6da9a9985c3f6b069d6818de34e4
Sep 18 20:29:04.871600 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:29:04.871561    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-8719-56a598" event={"ID":"a49843a5-8098-45c8-926b-f39e46182606","Type":"ContainerStarted","Data":"46e297a59195330d7908a4bdcf01ea650a40f21abb05088d7988454b5c657bfb"}
Sep 18 20:29:04.872000 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:29:04.871613    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-8719-56a598" event={"ID":"a49843a5-8098-45c8-926b-f39e46182606","Type":"ContainerStarted","Data":"4603d2a6ff8b97a92a75b90bde148096177e6da9a9985c3f6b069d6818de34e4"}
Sep 18 20:29:04.887927 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:29:04.887876    2532 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-8719-56a598" podStartSLOduration=1.88786013 podStartE2EDuration="1.88786013s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 20:29:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 20:29:04.885570764 +0000 UTC m=+1676.650960165" watchObservedRunningTime="2026-09-18 20:29:04.88786013 +0000 UTC m=+1676.653249586"
Sep 18 20:29:05.440970 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:29:05.440894    2532 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-8719-56a598"]
Sep 18 20:29:05.443472 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:29:05.443443    2532 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-8719-56a598"]
Sep 18 20:29:05.876831 ip-10-0-130-81 kubenswrapper[2532]: E0918 20:29:05.876800    2532 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-8719-56a598" container="test-curl-8719-56a598"
Sep 18 20:29:06.742216 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:29:06.742185    2532 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w2-bab50d/auth-test-model-bab50d-kserve-79d56fcd7f-g89z4"]
Sep 18 20:29:06.765671 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:29:06.765638    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:29:06.882434 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:29:06.882387    2532 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-8719-56a598" podUID="a49843a5-8098-45c8-926b-f39e46182606" containerName="test-curl-8719-56a598" containerID="cri-o://46e297a59195330d7908a4bdcf01ea650a40f21abb05088d7988454b5c657bfb" gracePeriod=2
Sep 18 20:29:07.899362 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:29:07.899323    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:29:09.076574 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:29:09.076549    2532 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-8719-56a598"
Sep 18 20:29:09.211915 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:29:09.211889    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-52jlc\" (UniqueName: \"kubernetes.io/projected/a49843a5-8098-45c8-926b-f39e46182606-kube-api-access-52jlc\") pod \"a49843a5-8098-45c8-926b-f39e46182606\" (UID: \"a49843a5-8098-45c8-926b-f39e46182606\") "
Sep 18 20:29:09.213958 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:29:09.213929    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a49843a5-8098-45c8-926b-f39e46182606-kube-api-access-52jlc" (OuterVolumeSpecName: "kube-api-access-52jlc") pod "a49843a5-8098-45c8-926b-f39e46182606" (UID: "a49843a5-8098-45c8-926b-f39e46182606"). InnerVolumeSpecName "kube-api-access-52jlc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 20:29:09.313180 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:29:09.313146    2532 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-52jlc\" (UniqueName: \"kubernetes.io/projected/a49843a5-8098-45c8-926b-f39e46182606-kube-api-access-52jlc\") on node \"ip-10-0-130-81.ec2.internal\" DevicePath \"\""
Sep 18 20:29:09.901673 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:29:09.901634    2532 generic.go:356] "Generic (PLEG): container finished" podID="a49843a5-8098-45c8-926b-f39e46182606" containerID="46e297a59195330d7908a4bdcf01ea650a40f21abb05088d7988454b5c657bfb" exitCode=137
Sep 18 20:29:09.901673 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:29:09.901672    2532 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-8719-56a598"
Sep 18 20:29:09.901923 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:29:09.901734    2532 scope.go:117] "RemoveContainer" containerID="46e297a59195330d7908a4bdcf01ea650a40f21abb05088d7988454b5c657bfb"
Sep 18 20:29:09.910948 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:29:09.910930    2532 scope.go:117] "RemoveContainer" containerID="46e297a59195330d7908a4bdcf01ea650a40f21abb05088d7988454b5c657bfb"
Sep 18 20:29:09.911229 ip-10-0-130-81 kubenswrapper[2532]: E0918 20:29:09.911212    2532 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"46e297a59195330d7908a4bdcf01ea650a40f21abb05088d7988454b5c657bfb\": container with ID starting with 46e297a59195330d7908a4bdcf01ea650a40f21abb05088d7988454b5c657bfb not found: ID does not exist" containerID="46e297a59195330d7908a4bdcf01ea650a40f21abb05088d7988454b5c657bfb"
Sep 18 20:29:09.911273 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:29:09.911238    2532 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"46e297a59195330d7908a4bdcf01ea650a40f21abb05088d7988454b5c657bfb"} err="failed to get container status \"46e297a59195330d7908a4bdcf01ea650a40f21abb05088d7988454b5c657bfb\": rpc error: code = NotFound desc = could not find container \"46e297a59195330d7908a4bdcf01ea650a40f21abb05088d7988454b5c657bfb\": container with ID starting with 46e297a59195330d7908a4bdcf01ea650a40f21abb05088d7988454b5c657bfb not found: ID does not exist"
Sep 18 20:29:09.912547 ip-10-0-130-81 kubenswrapper[2532]: E0918 20:29:09.912526    2532 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-8719-56a598\" is forbidden: User \"system:node:ip-10-0-130-81.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-130-81.ec2.internal' and this object" podUID="a49843a5-8098-45c8-926b-f39e46182606" pod="opendatahub/test-curl-8719-56a598"
Sep 18 20:29:10.306337 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:29:10.306301    2532 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a49843a5-8098-45c8-926b-f39e46182606" path="/var/lib/kubelet/pods/a49843a5-8098-45c8-926b-f39e46182606/volumes"
Sep 18 20:29:10.306716 ip-10-0-130-81 kubenswrapper[2532]: E0918 20:29:10.306395    2532 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-8719-56a598\" is forbidden: User \"system:node:ip-10-0-130-81.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-130-81.ec2.internal' and this object" podUID="a49843a5-8098-45c8-926b-f39e46182606" pod="opendatahub/test-curl-8719-56a598"
Sep 18 20:29:11.600128 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:29:11.600093    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-vptdl"]
Sep 18 20:29:13.395300 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:29:13.395271    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:29:14.067877 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:29:14.067844    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:29:14.814333 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:29:14.814294    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:29:15.443820 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:29:15.443791    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:29:16.113010 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:29:16.112980    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:29:16.919793 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:29:16.919763    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:29:17.550165 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:29:17.550136    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:29:17.927456 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:29:17.927373    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:29:21.025775 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:29:21.025740    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:29:21.761350 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:29:21.761313    2532 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-w2-bab50d/auth-test-model-bab50d-kserve-79d56fcd7f-g89z4" podUID="a2540b2a-3627-41ea-b2b0-225d80f63662" containerName="main" containerID="cri-o://88b1759ce7cdc3284c6acc7cebcc0aae04fa6cf8211d3d8d623fb0dcdcc32b70" gracePeriod=45
Sep 18 20:29:21.933399 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:29:21.933372    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:29:21.969062 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:29:21.969030    2532 generic.go:356] "Generic (PLEG): container finished" podID="a2540b2a-3627-41ea-b2b0-225d80f63662" containerID="88b1759ce7cdc3284c6acc7cebcc0aae04fa6cf8211d3d8d623fb0dcdcc32b70" exitCode=0
Sep 18 20:29:21.969208 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:29:21.969105    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w2-bab50d/auth-test-model-bab50d-kserve-79d56fcd7f-g89z4" event={"ID":"a2540b2a-3627-41ea-b2b0-225d80f63662","Type":"ContainerDied","Data":"88b1759ce7cdc3284c6acc7cebcc0aae04fa6cf8211d3d8d623fb0dcdcc32b70"}
Sep 18 20:29:22.010056 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:29:22.010034    2532 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w2-bab50d/auth-test-model-bab50d-kserve-79d56fcd7f-g89z4"
Sep 18 20:29:22.134595 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:29:22.134510    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a2540b2a-3627-41ea-b2b0-225d80f63662-tmp-dir\") pod \"a2540b2a-3627-41ea-b2b0-225d80f63662\" (UID: \"a2540b2a-3627-41ea-b2b0-225d80f63662\") "
Sep 18 20:29:22.134595 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:29:22.134591    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a2540b2a-3627-41ea-b2b0-225d80f63662-tls-certs\") pod \"a2540b2a-3627-41ea-b2b0-225d80f63662\" (UID: \"a2540b2a-3627-41ea-b2b0-225d80f63662\") "
Sep 18 20:29:22.135078 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:29:22.134626    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a2540b2a-3627-41ea-b2b0-225d80f63662-model-cache\") pod \"a2540b2a-3627-41ea-b2b0-225d80f63662\" (UID: \"a2540b2a-3627-41ea-b2b0-225d80f63662\") "
Sep 18 20:29:22.135078 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:29:22.134657    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a2540b2a-3627-41ea-b2b0-225d80f63662-dshm\") pod \"a2540b2a-3627-41ea-b2b0-225d80f63662\" (UID: \"a2540b2a-3627-41ea-b2b0-225d80f63662\") "
Sep 18 20:29:22.135078 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:29:22.134729    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bntx7\" (UniqueName: \"kubernetes.io/projected/a2540b2a-3627-41ea-b2b0-225d80f63662-kube-api-access-bntx7\") pod \"a2540b2a-3627-41ea-b2b0-225d80f63662\" (UID: \"a2540b2a-3627-41ea-b2b0-225d80f63662\") "
Sep 18 20:29:22.135078 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:29:22.134778    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a2540b2a-3627-41ea-b2b0-225d80f63662-home\") pod \"a2540b2a-3627-41ea-b2b0-225d80f63662\" (UID: \"a2540b2a-3627-41ea-b2b0-225d80f63662\") "
Sep 18 20:29:22.135078 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:29:22.134834    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a2540b2a-3627-41ea-b2b0-225d80f63662-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "a2540b2a-3627-41ea-b2b0-225d80f63662" (UID: "a2540b2a-3627-41ea-b2b0-225d80f63662"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 20:29:22.135078 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:29:22.134925    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a2540b2a-3627-41ea-b2b0-225d80f63662-model-cache" (OuterVolumeSpecName: "model-cache") pod "a2540b2a-3627-41ea-b2b0-225d80f63662" (UID: "a2540b2a-3627-41ea-b2b0-225d80f63662"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 20:29:22.135401 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:29:22.135088    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a2540b2a-3627-41ea-b2b0-225d80f63662-home" (OuterVolumeSpecName: "home") pod "a2540b2a-3627-41ea-b2b0-225d80f63662" (UID: "a2540b2a-3627-41ea-b2b0-225d80f63662"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 20:29:22.135401 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:29:22.135140    2532 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a2540b2a-3627-41ea-b2b0-225d80f63662-model-cache\") on node \"ip-10-0-130-81.ec2.internal\" DevicePath \"\""
Sep 18 20:29:22.135401 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:29:22.135175    2532 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a2540b2a-3627-41ea-b2b0-225d80f63662-tmp-dir\") on node \"ip-10-0-130-81.ec2.internal\" DevicePath \"\""
Sep 18 20:29:22.136938 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:29:22.136911    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a2540b2a-3627-41ea-b2b0-225d80f63662-dshm" (OuterVolumeSpecName: "dshm") pod "a2540b2a-3627-41ea-b2b0-225d80f63662" (UID: "a2540b2a-3627-41ea-b2b0-225d80f63662"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 20:29:22.137064 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:29:22.136977    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a2540b2a-3627-41ea-b2b0-225d80f63662-kube-api-access-bntx7" (OuterVolumeSpecName: "kube-api-access-bntx7") pod "a2540b2a-3627-41ea-b2b0-225d80f63662" (UID: "a2540b2a-3627-41ea-b2b0-225d80f63662"). InnerVolumeSpecName "kube-api-access-bntx7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 20:29:22.137064 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:29:22.136995    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a2540b2a-3627-41ea-b2b0-225d80f63662-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "a2540b2a-3627-41ea-b2b0-225d80f63662" (UID: "a2540b2a-3627-41ea-b2b0-225d80f63662"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 20:29:22.235945 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:29:22.235909    2532 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a2540b2a-3627-41ea-b2b0-225d80f63662-home\") on node \"ip-10-0-130-81.ec2.internal\" DevicePath \"\""
Sep 18 20:29:22.235945 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:29:22.235938    2532 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a2540b2a-3627-41ea-b2b0-225d80f63662-tls-certs\") on node \"ip-10-0-130-81.ec2.internal\" DevicePath \"\""
Sep 18 20:29:22.235945 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:29:22.235948    2532 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a2540b2a-3627-41ea-b2b0-225d80f63662-dshm\") on node \"ip-10-0-130-81.ec2.internal\" DevicePath \"\""
Sep 18 20:29:22.235945 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:29:22.235956    2532 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bntx7\" (UniqueName: \"kubernetes.io/projected/a2540b2a-3627-41ea-b2b0-225d80f63662-kube-api-access-bntx7\") on node \"ip-10-0-130-81.ec2.internal\" DevicePath \"\""
Sep 18 20:29:22.585098 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:29:22.585068    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:29:22.975867 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:29:22.975843    2532 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w2-bab50d/auth-test-model-bab50d-kserve-79d56fcd7f-g89z4"
Sep 18 20:29:22.976031 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:29:22.975872    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w2-bab50d/auth-test-model-bab50d-kserve-79d56fcd7f-g89z4" event={"ID":"a2540b2a-3627-41ea-b2b0-225d80f63662","Type":"ContainerDied","Data":"cae888ea4e64873f5877ef606c3458b0a71d31ec84b4fc1dd435b05e8f793ff7"}
Sep 18 20:29:22.976031 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:29:22.975916    2532 scope.go:117] "RemoveContainer" containerID="88b1759ce7cdc3284c6acc7cebcc0aae04fa6cf8211d3d8d623fb0dcdcc32b70"
Sep 18 20:29:22.992791 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:29:22.992769    2532 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w2-bab50d/auth-test-model-bab50d-kserve-79d56fcd7f-g89z4"]
Sep 18 20:29:22.996624 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:29:22.996604    2532 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-w2-bab50d/auth-test-model-bab50d-kserve-79d56fcd7f-g89z4"]
Sep 18 20:29:23.309816 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:29:23.309735    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:29:24.007191 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:29:24.007143    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:29:24.306938 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:29:24.306859    2532 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a2540b2a-3627-41ea-b2b0-225d80f63662" path="/var/lib/kubelet/pods/a2540b2a-3627-41ea-b2b0-225d80f63662/volumes"
Sep 18 20:29:24.562170 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:29:24.562082    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:29:25.327458 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:29:25.327425    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:29:25.977351 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:29:25.977308    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:29:27.006519 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:29:27.006475    2532 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-w2-bab50d/sub-test-model-bab50d-kserve-546f9db6c-9zflz"]
Sep 18 20:29:27.006893 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:29:27.006879    2532 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a2540b2a-3627-41ea-b2b0-225d80f63662" containerName="main"
Sep 18 20:29:27.006934 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:29:27.006894    2532 state_mem.go:107] "Deleted CPUSet assignment" podUID="a2540b2a-3627-41ea-b2b0-225d80f63662" containerName="main"
Sep 18 20:29:27.006934 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:29:27.006904    2532 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a49843a5-8098-45c8-926b-f39e46182606" containerName="test-curl-8719-56a598"
Sep 18 20:29:27.006934 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:29:27.006909    2532 state_mem.go:107] "Deleted CPUSet assignment" podUID="a49843a5-8098-45c8-926b-f39e46182606" containerName="test-curl-8719-56a598"
Sep 18 20:29:27.007059 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:29:27.007000    2532 memory_manager.go:368] "RemoveStaleState removing state" podUID="a49843a5-8098-45c8-926b-f39e46182606" containerName="test-curl-8719-56a598"
Sep 18 20:29:27.007059 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:29:27.007012    2532 memory_manager.go:368] "RemoveStaleState removing state" podUID="a2540b2a-3627-41ea-b2b0-225d80f63662" containerName="main"
Sep 18 20:29:27.011188 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:29:27.011171    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w2-bab50d/sub-test-model-bab50d-kserve-546f9db6c-9zflz"
Sep 18 20:29:27.020375 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:29:27.020351    2532 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w2-bab50d\"/\"sub-test-model-bab50d-kserve-self-signed-certs\""
Sep 18 20:29:27.020535 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:29:27.020399    2532 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w2-bab50d\"/\"openshift-service-ca.crt\""
Sep 18 20:29:27.020535 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:29:27.020429    2532 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w2-bab50d\"/\"kube-root-ca.crt\""
Sep 18 20:29:27.020678 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:29:27.020544    2532 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w2-bab50d\"/\"default-dockercfg-tppw2\""
Sep 18 20:29:27.023100 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:29:27.023075    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w2-bab50d/sub-test-model-bab50d-kserve-546f9db6c-9zflz"]
Sep 18 20:29:27.035827 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:29:27.035798    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:29:27.074999 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:29:27.074963    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/904d4141-71a8-47a3-8678-a2fd2e729c18-home\") pod \"sub-test-model-bab50d-kserve-546f9db6c-9zflz\" (UID: \"904d4141-71a8-47a3-8678-a2fd2e729c18\") " pod="ai-tenant-e2e-shared-a-w2-bab50d/sub-test-model-bab50d-kserve-546f9db6c-9zflz"
Sep 18 20:29:27.074999 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:29:27.074999    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/904d4141-71a8-47a3-8678-a2fd2e729c18-tmp-dir\") pod \"sub-test-model-bab50d-kserve-546f9db6c-9zflz\" (UID: \"904d4141-71a8-47a3-8678-a2fd2e729c18\") " pod="ai-tenant-e2e-shared-a-w2-bab50d/sub-test-model-bab50d-kserve-546f9db6c-9zflz"
Sep 18 20:29:27.075207 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:29:27.075031    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/904d4141-71a8-47a3-8678-a2fd2e729c18-dshm\") pod \"sub-test-model-bab50d-kserve-546f9db6c-9zflz\" (UID: \"904d4141-71a8-47a3-8678-a2fd2e729c18\") " pod="ai-tenant-e2e-shared-a-w2-bab50d/sub-test-model-bab50d-kserve-546f9db6c-9zflz"
Sep 18 20:29:27.075207 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:29:27.075079    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/904d4141-71a8-47a3-8678-a2fd2e729c18-tls-certs\") pod \"sub-test-model-bab50d-kserve-546f9db6c-9zflz\" (UID: \"904d4141-71a8-47a3-8678-a2fd2e729c18\") " pod="ai-tenant-e2e-shared-a-w2-bab50d/sub-test-model-bab50d-kserve-546f9db6c-9zflz"
Sep 18 20:29:27.075207 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:29:27.075128    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/904d4141-71a8-47a3-8678-a2fd2e729c18-model-cache\") pod \"sub-test-model-bab50d-kserve-546f9db6c-9zflz\" (UID: \"904d4141-71a8-47a3-8678-a2fd2e729c18\") " pod="ai-tenant-e2e-shared-a-w2-bab50d/sub-test-model-bab50d-kserve-546f9db6c-9zflz"
Sep 18 20:29:27.075327 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:29:27.075202    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p8wpv\" (UniqueName: \"kubernetes.io/projected/904d4141-71a8-47a3-8678-a2fd2e729c18-kube-api-access-p8wpv\") pod \"sub-test-model-bab50d-kserve-546f9db6c-9zflz\" (UID: \"904d4141-71a8-47a3-8678-a2fd2e729c18\") " pod="ai-tenant-e2e-shared-a-w2-bab50d/sub-test-model-bab50d-kserve-546f9db6c-9zflz"
Sep 18 20:29:27.175717 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:29:27.175683    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p8wpv\" (UniqueName: \"kubernetes.io/projected/904d4141-71a8-47a3-8678-a2fd2e729c18-kube-api-access-p8wpv\") pod \"sub-test-model-bab50d-kserve-546f9db6c-9zflz\" (UID: \"904d4141-71a8-47a3-8678-a2fd2e729c18\") " pod="ai-tenant-e2e-shared-a-w2-bab50d/sub-test-model-bab50d-kserve-546f9db6c-9zflz"
Sep 18 20:29:27.175913 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:29:27.175728    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/904d4141-71a8-47a3-8678-a2fd2e729c18-home\") pod \"sub-test-model-bab50d-kserve-546f9db6c-9zflz\" (UID: \"904d4141-71a8-47a3-8678-a2fd2e729c18\") " pod="ai-tenant-e2e-shared-a-w2-bab50d/sub-test-model-bab50d-kserve-546f9db6c-9zflz"
Sep 18 20:29:27.175913 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:29:27.175745    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/904d4141-71a8-47a3-8678-a2fd2e729c18-tmp-dir\") pod \"sub-test-model-bab50d-kserve-546f9db6c-9zflz\" (UID: \"904d4141-71a8-47a3-8678-a2fd2e729c18\") " pod="ai-tenant-e2e-shared-a-w2-bab50d/sub-test-model-bab50d-kserve-546f9db6c-9zflz"
Sep 18 20:29:27.175913 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:29:27.175786    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/904d4141-71a8-47a3-8678-a2fd2e729c18-dshm\") pod \"sub-test-model-bab50d-kserve-546f9db6c-9zflz\" (UID: \"904d4141-71a8-47a3-8678-a2fd2e729c18\") " pod="ai-tenant-e2e-shared-a-w2-bab50d/sub-test-model-bab50d-kserve-546f9db6c-9zflz"
Sep 18 20:29:27.175913 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:29:27.175806    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/904d4141-71a8-47a3-8678-a2fd2e729c18-tls-certs\") pod \"sub-test-model-bab50d-kserve-546f9db6c-9zflz\" (UID: \"904d4141-71a8-47a3-8678-a2fd2e729c18\") " pod="ai-tenant-e2e-shared-a-w2-bab50d/sub-test-model-bab50d-kserve-546f9db6c-9zflz"
Sep 18 20:29:27.176121 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:29:27.175965    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/904d4141-71a8-47a3-8678-a2fd2e729c18-model-cache\") pod \"sub-test-model-bab50d-kserve-546f9db6c-9zflz\" (UID: \"904d4141-71a8-47a3-8678-a2fd2e729c18\") " pod="ai-tenant-e2e-shared-a-w2-bab50d/sub-test-model-bab50d-kserve-546f9db6c-9zflz"
Sep 18 20:29:27.176181 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:29:27.176153    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/904d4141-71a8-47a3-8678-a2fd2e729c18-home\") pod \"sub-test-model-bab50d-kserve-546f9db6c-9zflz\" (UID: \"904d4141-71a8-47a3-8678-a2fd2e729c18\") " pod="ai-tenant-e2e-shared-a-w2-bab50d/sub-test-model-bab50d-kserve-546f9db6c-9zflz"
Sep 18 20:29:27.176241 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:29:27.176186    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/904d4141-71a8-47a3-8678-a2fd2e729c18-tmp-dir\") pod \"sub-test-model-bab50d-kserve-546f9db6c-9zflz\" (UID: \"904d4141-71a8-47a3-8678-a2fd2e729c18\") " pod="ai-tenant-e2e-shared-a-w2-bab50d/sub-test-model-bab50d-kserve-546f9db6c-9zflz"
Sep 18 20:29:27.176362 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:29:27.176340    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/904d4141-71a8-47a3-8678-a2fd2e729c18-model-cache\") pod \"sub-test-model-bab50d-kserve-546f9db6c-9zflz\" (UID: \"904d4141-71a8-47a3-8678-a2fd2e729c18\") " pod="ai-tenant-e2e-shared-a-w2-bab50d/sub-test-model-bab50d-kserve-546f9db6c-9zflz"
Sep 18 20:29:27.178135 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:29:27.178114    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/904d4141-71a8-47a3-8678-a2fd2e729c18-dshm\") pod \"sub-test-model-bab50d-kserve-546f9db6c-9zflz\" (UID: \"904d4141-71a8-47a3-8678-a2fd2e729c18\") " pod="ai-tenant-e2e-shared-a-w2-bab50d/sub-test-model-bab50d-kserve-546f9db6c-9zflz"
Sep 18 20:29:27.178304 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:29:27.178287    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/904d4141-71a8-47a3-8678-a2fd2e729c18-tls-certs\") pod \"sub-test-model-bab50d-kserve-546f9db6c-9zflz\" (UID: \"904d4141-71a8-47a3-8678-a2fd2e729c18\") " pod="ai-tenant-e2e-shared-a-w2-bab50d/sub-test-model-bab50d-kserve-546f9db6c-9zflz"
Sep 18 20:29:27.203502 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:29:27.203464    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p8wpv\" (UniqueName: \"kubernetes.io/projected/904d4141-71a8-47a3-8678-a2fd2e729c18-kube-api-access-p8wpv\") pod \"sub-test-model-bab50d-kserve-546f9db6c-9zflz\" (UID: \"904d4141-71a8-47a3-8678-a2fd2e729c18\") " pod="ai-tenant-e2e-shared-a-w2-bab50d/sub-test-model-bab50d-kserve-546f9db6c-9zflz"
Sep 18 20:29:27.320871 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:29:27.320787    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w2-bab50d/sub-test-model-bab50d-kserve-546f9db6c-9zflz"
Sep 18 20:29:27.593136 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:29:27.593111    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w2-bab50d/sub-test-model-bab50d-kserve-546f9db6c-9zflz"]
Sep 18 20:29:27.594075 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:29:27.594052    2532 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod904d4141_71a8_47a3_8678_a2fd2e729c18.slice/crio-c240674e3631c260b450987d2f0e23289b5c0c4d2e9f0368ad0f67b65529d3c5 WatchSource:0}: Error finding container c240674e3631c260b450987d2f0e23289b5c0c4d2e9f0368ad0f67b65529d3c5: Status 404 returned error can't find the container with id c240674e3631c260b450987d2f0e23289b5c0c4d2e9f0368ad0f67b65529d3c5
Sep 18 20:29:28.006008 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:29:28.005971    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w2-bab50d/sub-test-model-bab50d-kserve-546f9db6c-9zflz" event={"ID":"904d4141-71a8-47a3-8678-a2fd2e729c18","Type":"ContainerStarted","Data":"171010b82ab79d499e6d5f7df5c045b7fada7ee70cb0286c4474c06f6748957e"}
Sep 18 20:29:28.006211 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:29:28.006022    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w2-bab50d/sub-test-model-bab50d-kserve-546f9db6c-9zflz" event={"ID":"904d4141-71a8-47a3-8678-a2fd2e729c18","Type":"ContainerStarted","Data":"c240674e3631c260b450987d2f0e23289b5c0c4d2e9f0368ad0f67b65529d3c5"}
Sep 18 20:29:28.024433 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:29:28.024378    2532 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-w2-bab50d/sub-test-model-bab50d-kserve-546f9db6c-9zflz" podStartSLOduration=2.024361649 podStartE2EDuration="2.024361649s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 20:29:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 20:29:28.023146025 +0000 UTC m=+1699.788535426" watchObservedRunningTime="2026-09-18 20:29:28.024361649 +0000 UTC m=+1699.789751045"
Sep 18 20:29:30.798598 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:29:30.798568    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-vptdl"]
Sep 18 20:29:36.211258 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:29:36.211219    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:29:36.280302 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:29:36.280270    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:29:36.380677 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:29:36.380643    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:29:36.481150 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:29:36.481070    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:29:36.774447 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:29:36.774363    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:29:36.854792 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:29:36.854750    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:29:37.321140 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:29:37.321108    2532 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-w2-bab50d/sub-test-model-bab50d-kserve-546f9db6c-9zflz"
Sep 18 20:29:37.321551 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:29:37.321178    2532 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-w2-bab50d/sub-test-model-bab50d-kserve-546f9db6c-9zflz"
Sep 18 20:29:37.334077 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:29:37.334051    2532 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-w2-bab50d/sub-test-model-bab50d-kserve-546f9db6c-9zflz"
Sep 18 20:29:37.619167 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:29:37.619079    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:29:38.070499 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:29:38.070465    2532 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-w2-bab50d/sub-test-model-bab50d-kserve-546f9db6c-9zflz"
Sep 18 20:29:38.310234 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:29:38.310206    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:29:39.278557 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:29:39.278523    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:29:39.914350 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:29:39.914306    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:29:40.526174 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:29:40.526138    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:29:42.617968 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:29:42.617939    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:29:42.685889 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:29:42.685858    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:29:42.869076 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:29:42.868998    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:29:43.502575 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:29:43.502526    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:29:44.175159 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:29:44.175128    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:29:44.939156 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:29:44.939125    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:29:45.594902 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:29:45.594867    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:29:46.096228 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:29:46.096198    2532 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-w2-35558b/sub-test-model-35558b-kserve-6fd4fb4567-9zj2l"]
Sep 18 20:29:46.099867 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:29:46.099845    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w2-35558b/sub-test-model-35558b-kserve-6fd4fb4567-9zj2l"
Sep 18 20:29:46.105002 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:29:46.104970    2532 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w2-35558b\"/\"openshift-service-ca.crt\""
Sep 18 20:29:46.105002 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:29:46.104992    2532 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w2-35558b\"/\"default-dockercfg-lcngt\""
Sep 18 20:29:46.105242 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:29:46.104995    2532 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w2-35558b\"/\"sub-test-model-35558b-kserve-self-signed-certs\""
Sep 18 20:29:46.105474 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:29:46.105458    2532 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w2-35558b\"/\"kube-root-ca.crt\""
Sep 18 20:29:46.113740 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:29:46.113716    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w2-35558b/sub-test-model-35558b-kserve-6fd4fb4567-9zj2l"]
Sep 18 20:29:46.131831 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:29:46.131799    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:29:46.155932 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:29:46.155899    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f0a6570f-5955-4f14-a162-8ff27ca89629-home\") pod \"sub-test-model-35558b-kserve-6fd4fb4567-9zj2l\" (UID: \"f0a6570f-5955-4f14-a162-8ff27ca89629\") " pod="ai-tenant-e2e-shared-b-w2-35558b/sub-test-model-35558b-kserve-6fd4fb4567-9zj2l"
Sep 18 20:29:46.156051 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:29:46.155937    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f0a6570f-5955-4f14-a162-8ff27ca89629-tls-certs\") pod \"sub-test-model-35558b-kserve-6fd4fb4567-9zj2l\" (UID: \"f0a6570f-5955-4f14-a162-8ff27ca89629\") " pod="ai-tenant-e2e-shared-b-w2-35558b/sub-test-model-35558b-kserve-6fd4fb4567-9zj2l"
Sep 18 20:29:46.156051 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:29:46.155958    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f0a6570f-5955-4f14-a162-8ff27ca89629-dshm\") pod \"sub-test-model-35558b-kserve-6fd4fb4567-9zj2l\" (UID: \"f0a6570f-5955-4f14-a162-8ff27ca89629\") " pod="ai-tenant-e2e-shared-b-w2-35558b/sub-test-model-35558b-kserve-6fd4fb4567-9zj2l"
Sep 18 20:29:46.156051 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:29:46.155983    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wpwbr\" (UniqueName: \"kubernetes.io/projected/f0a6570f-5955-4f14-a162-8ff27ca89629-kube-api-access-wpwbr\") pod \"sub-test-model-35558b-kserve-6fd4fb4567-9zj2l\" (UID: \"f0a6570f-5955-4f14-a162-8ff27ca89629\") " pod="ai-tenant-e2e-shared-b-w2-35558b/sub-test-model-35558b-kserve-6fd4fb4567-9zj2l"
Sep 18 20:29:46.156051 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:29:46.156041    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f0a6570f-5955-4f14-a162-8ff27ca89629-model-cache\") pod \"sub-test-model-35558b-kserve-6fd4fb4567-9zj2l\" (UID: \"f0a6570f-5955-4f14-a162-8ff27ca89629\") " pod="ai-tenant-e2e-shared-b-w2-35558b/sub-test-model-35558b-kserve-6fd4fb4567-9zj2l"
Sep 18 20:29:46.156225 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:29:46.156088    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f0a6570f-5955-4f14-a162-8ff27ca89629-tmp-dir\") pod \"sub-test-model-35558b-kserve-6fd4fb4567-9zj2l\" (UID: \"f0a6570f-5955-4f14-a162-8ff27ca89629\") " pod="ai-tenant-e2e-shared-b-w2-35558b/sub-test-model-35558b-kserve-6fd4fb4567-9zj2l"
Sep 18 20:29:46.257160 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:29:46.257122    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f0a6570f-5955-4f14-a162-8ff27ca89629-home\") pod \"sub-test-model-35558b-kserve-6fd4fb4567-9zj2l\" (UID: \"f0a6570f-5955-4f14-a162-8ff27ca89629\") " pod="ai-tenant-e2e-shared-b-w2-35558b/sub-test-model-35558b-kserve-6fd4fb4567-9zj2l"
Sep 18 20:29:46.257160 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:29:46.257165    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f0a6570f-5955-4f14-a162-8ff27ca89629-tls-certs\") pod \"sub-test-model-35558b-kserve-6fd4fb4567-9zj2l\" (UID: \"f0a6570f-5955-4f14-a162-8ff27ca89629\") " pod="ai-tenant-e2e-shared-b-w2-35558b/sub-test-model-35558b-kserve-6fd4fb4567-9zj2l"
Sep 18 20:29:46.257404 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:29:46.257184    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f0a6570f-5955-4f14-a162-8ff27ca89629-dshm\") pod \"sub-test-model-35558b-kserve-6fd4fb4567-9zj2l\" (UID: \"f0a6570f-5955-4f14-a162-8ff27ca89629\") " pod="ai-tenant-e2e-shared-b-w2-35558b/sub-test-model-35558b-kserve-6fd4fb4567-9zj2l"
Sep 18 20:29:46.257404 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:29:46.257210    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wpwbr\" (UniqueName: \"kubernetes.io/projected/f0a6570f-5955-4f14-a162-8ff27ca89629-kube-api-access-wpwbr\") pod \"sub-test-model-35558b-kserve-6fd4fb4567-9zj2l\" (UID: \"f0a6570f-5955-4f14-a162-8ff27ca89629\") " pod="ai-tenant-e2e-shared-b-w2-35558b/sub-test-model-35558b-kserve-6fd4fb4567-9zj2l"
Sep 18 20:29:46.257404 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:29:46.257246    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f0a6570f-5955-4f14-a162-8ff27ca89629-model-cache\") pod \"sub-test-model-35558b-kserve-6fd4fb4567-9zj2l\" (UID: \"f0a6570f-5955-4f14-a162-8ff27ca89629\") " pod="ai-tenant-e2e-shared-b-w2-35558b/sub-test-model-35558b-kserve-6fd4fb4567-9zj2l"
Sep 18 20:29:46.257404 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:29:46.257281    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f0a6570f-5955-4f14-a162-8ff27ca89629-tmp-dir\") pod \"sub-test-model-35558b-kserve-6fd4fb4567-9zj2l\" (UID: \"f0a6570f-5955-4f14-a162-8ff27ca89629\") " pod="ai-tenant-e2e-shared-b-w2-35558b/sub-test-model-35558b-kserve-6fd4fb4567-9zj2l"
Sep 18 20:29:46.257669 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:29:46.257641    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f0a6570f-5955-4f14-a162-8ff27ca89629-home\") pod \"sub-test-model-35558b-kserve-6fd4fb4567-9zj2l\" (UID: \"f0a6570f-5955-4f14-a162-8ff27ca89629\") " pod="ai-tenant-e2e-shared-b-w2-35558b/sub-test-model-35558b-kserve-6fd4fb4567-9zj2l"
Sep 18 20:29:46.257744 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:29:46.257692    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f0a6570f-5955-4f14-a162-8ff27ca89629-tmp-dir\") pod \"sub-test-model-35558b-kserve-6fd4fb4567-9zj2l\" (UID: \"f0a6570f-5955-4f14-a162-8ff27ca89629\") " pod="ai-tenant-e2e-shared-b-w2-35558b/sub-test-model-35558b-kserve-6fd4fb4567-9zj2l"
Sep 18 20:29:46.257827 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:29:46.257798    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f0a6570f-5955-4f14-a162-8ff27ca89629-model-cache\") pod \"sub-test-model-35558b-kserve-6fd4fb4567-9zj2l\" (UID: \"f0a6570f-5955-4f14-a162-8ff27ca89629\") " pod="ai-tenant-e2e-shared-b-w2-35558b/sub-test-model-35558b-kserve-6fd4fb4567-9zj2l"
Sep 18 20:29:46.259804 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:29:46.259765    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f0a6570f-5955-4f14-a162-8ff27ca89629-dshm\") pod \"sub-test-model-35558b-kserve-6fd4fb4567-9zj2l\" (UID: \"f0a6570f-5955-4f14-a162-8ff27ca89629\") " pod="ai-tenant-e2e-shared-b-w2-35558b/sub-test-model-35558b-kserve-6fd4fb4567-9zj2l"
Sep 18 20:29:46.259991 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:29:46.259976    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f0a6570f-5955-4f14-a162-8ff27ca89629-tls-certs\") pod \"sub-test-model-35558b-kserve-6fd4fb4567-9zj2l\" (UID: \"f0a6570f-5955-4f14-a162-8ff27ca89629\") " pod="ai-tenant-e2e-shared-b-w2-35558b/sub-test-model-35558b-kserve-6fd4fb4567-9zj2l"
Sep 18 20:29:46.266795 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:29:46.266774    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wpwbr\" (UniqueName: \"kubernetes.io/projected/f0a6570f-5955-4f14-a162-8ff27ca89629-kube-api-access-wpwbr\") pod \"sub-test-model-35558b-kserve-6fd4fb4567-9zj2l\" (UID: \"f0a6570f-5955-4f14-a162-8ff27ca89629\") " pod="ai-tenant-e2e-shared-b-w2-35558b/sub-test-model-35558b-kserve-6fd4fb4567-9zj2l"
Sep 18 20:29:46.409656 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:29:46.409574    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w2-35558b/sub-test-model-35558b-kserve-6fd4fb4567-9zj2l"
Sep 18 20:29:46.430363 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:29:46.430338    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:29:46.681782 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:29:46.681754    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w2-35558b/sub-test-model-35558b-kserve-6fd4fb4567-9zj2l"]
Sep 18 20:29:46.683250 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:29:46.683224    2532 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf0a6570f_5955_4f14_a162_8ff27ca89629.slice/crio-883db7505160222ce852c351303060eac7ac88474bc0b85c2f6b4ce96e9e4d51 WatchSource:0}: Error finding container 883db7505160222ce852c351303060eac7ac88474bc0b85c2f6b4ce96e9e4d51: Status 404 returned error can't find the container with id 883db7505160222ce852c351303060eac7ac88474bc0b85c2f6b4ce96e9e4d51
Sep 18 20:29:47.115266 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:29:47.115227    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:29:47.118673 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:29:47.118639    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w2-35558b/sub-test-model-35558b-kserve-6fd4fb4567-9zj2l" event={"ID":"f0a6570f-5955-4f14-a162-8ff27ca89629","Type":"ContainerStarted","Data":"201ce52f97f1a0646a6cf4e574b20e048eb7b5f5ccccf13aa524645e875c9623"}
Sep 18 20:29:47.118853 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:29:47.118683    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w2-35558b/sub-test-model-35558b-kserve-6fd4fb4567-9zj2l" event={"ID":"f0a6570f-5955-4f14-a162-8ff27ca89629","Type":"ContainerStarted","Data":"883db7505160222ce852c351303060eac7ac88474bc0b85c2f6b4ce96e9e4d51"}
Sep 18 20:29:47.137236 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:29:47.137184    2532 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-w2-35558b/sub-test-model-35558b-kserve-6fd4fb4567-9zj2l" podStartSLOduration=1.137169635 podStartE2EDuration="1.137169635s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 20:29:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 20:29:47.135508649 +0000 UTC m=+1718.900898040" watchObservedRunningTime="2026-09-18 20:29:47.137169635 +0000 UTC m=+1718.902559034"
Sep 18 20:29:47.808438 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:29:47.808401    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:29:50.502878 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:29:50.502842    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-vptdl"]
Sep 18 20:29:56.410621 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:29:56.410586    2532 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-w2-35558b/sub-test-model-35558b-kserve-6fd4fb4567-9zj2l"
Sep 18 20:29:56.411020 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:29:56.410856    2532 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-w2-35558b/sub-test-model-35558b-kserve-6fd4fb4567-9zj2l"
Sep 18 20:29:56.423343 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:29:56.423320    2532 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-w2-35558b/sub-test-model-35558b-kserve-6fd4fb4567-9zj2l"
Sep 18 20:29:57.185991 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:29:57.185960    2532 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-w2-35558b/sub-test-model-35558b-kserve-6fd4fb4567-9zj2l"
Sep 18 20:29:58.604265 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:29:58.604233    2532 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-102c83-7c4d97c84b-jlrwh"]
Sep 18 20:29:58.604795 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:29:58.604472    2532 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-102c83-7c4d97c84b-jlrwh" podUID="c43f0b3f-b95f-4e4a-81dc-2d6c248a41fa" containerName="maas-api" containerID="cri-o://69cd719fa7c93d6b4e36a06787aff366720cc7acb4dd363570fb99634f0d45f7" gracePeriod=30
Sep 18 20:29:58.641947 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:29:58.641907    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:29:58.740597 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:29:58.740563    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:29:58.825165 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:29:58.825137    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:29:58.858445 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:29:58.858390    2532 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-102c83-7c4d97c84b-jlrwh"
Sep 18 20:29:58.907774 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:29:58.907744    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:29:58.980054 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:29:58.980023    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c43f0b3f-b95f-4e4a-81dc-2d6c248a41fa-maas-api-tls\") pod \"c43f0b3f-b95f-4e4a-81dc-2d6c248a41fa\" (UID: \"c43f0b3f-b95f-4e4a-81dc-2d6c248a41fa\") "
Sep 18 20:29:58.980247 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:29:58.980198    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c43f0b3f-b95f-4e4a-81dc-2d6c248a41fa-openshift-service-ca\") pod \"c43f0b3f-b95f-4e4a-81dc-2d6c248a41fa\" (UID: \"c43f0b3f-b95f-4e4a-81dc-2d6c248a41fa\") "
Sep 18 20:29:58.980326 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:29:58.980249    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c43f0b3f-b95f-4e4a-81dc-2d6c248a41fa-metrics-certs\") pod \"c43f0b3f-b95f-4e4a-81dc-2d6c248a41fa\" (UID: \"c43f0b3f-b95f-4e4a-81dc-2d6c248a41fa\") "
Sep 18 20:29:58.980326 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:29:58.980304    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rbpp2\" (UniqueName: \"kubernetes.io/projected/c43f0b3f-b95f-4e4a-81dc-2d6c248a41fa-kube-api-access-rbpp2\") pod \"c43f0b3f-b95f-4e4a-81dc-2d6c248a41fa\" (UID: \"c43f0b3f-b95f-4e4a-81dc-2d6c248a41fa\") "
Sep 18 20:29:58.980639 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:29:58.980615    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c43f0b3f-b95f-4e4a-81dc-2d6c248a41fa-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "c43f0b3f-b95f-4e4a-81dc-2d6c248a41fa" (UID: "c43f0b3f-b95f-4e4a-81dc-2d6c248a41fa"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 20:29:58.982437 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:29:58.982406    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c43f0b3f-b95f-4e4a-81dc-2d6c248a41fa-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "c43f0b3f-b95f-4e4a-81dc-2d6c248a41fa" (UID: "c43f0b3f-b95f-4e4a-81dc-2d6c248a41fa"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 20:29:58.982568 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:29:58.982535    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c43f0b3f-b95f-4e4a-81dc-2d6c248a41fa-kube-api-access-rbpp2" (OuterVolumeSpecName: "kube-api-access-rbpp2") pod "c43f0b3f-b95f-4e4a-81dc-2d6c248a41fa" (UID: "c43f0b3f-b95f-4e4a-81dc-2d6c248a41fa"). InnerVolumeSpecName "kube-api-access-rbpp2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 20:29:58.982812 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:29:58.982786    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c43f0b3f-b95f-4e4a-81dc-2d6c248a41fa-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "c43f0b3f-b95f-4e4a-81dc-2d6c248a41fa" (UID: "c43f0b3f-b95f-4e4a-81dc-2d6c248a41fa"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 20:29:59.081747 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:29:59.081710    2532 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c43f0b3f-b95f-4e4a-81dc-2d6c248a41fa-openshift-service-ca\") on node \"ip-10-0-130-81.ec2.internal\" DevicePath \"\""
Sep 18 20:29:59.081747 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:29:59.081740    2532 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c43f0b3f-b95f-4e4a-81dc-2d6c248a41fa-metrics-certs\") on node \"ip-10-0-130-81.ec2.internal\" DevicePath \"\""
Sep 18 20:29:59.081747 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:29:59.081750    2532 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rbpp2\" (UniqueName: \"kubernetes.io/projected/c43f0b3f-b95f-4e4a-81dc-2d6c248a41fa-kube-api-access-rbpp2\") on node \"ip-10-0-130-81.ec2.internal\" DevicePath \"\""
Sep 18 20:29:59.081986 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:29:59.081759    2532 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c43f0b3f-b95f-4e4a-81dc-2d6c248a41fa-maas-api-tls\") on node \"ip-10-0-130-81.ec2.internal\" DevicePath \"\""
Sep 18 20:29:59.185566 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:29:59.184954    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:29:59.192446 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:29:59.192416    2532 generic.go:356] "Generic (PLEG): container finished" podID="c43f0b3f-b95f-4e4a-81dc-2d6c248a41fa" containerID="69cd719fa7c93d6b4e36a06787aff366720cc7acb4dd363570fb99634f0d45f7" exitCode=0
Sep 18 20:29:59.192610 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:29:59.192495    2532 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-102c83-7c4d97c84b-jlrwh"
Sep 18 20:29:59.192610 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:29:59.192524    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-102c83-7c4d97c84b-jlrwh" event={"ID":"c43f0b3f-b95f-4e4a-81dc-2d6c248a41fa","Type":"ContainerDied","Data":"69cd719fa7c93d6b4e36a06787aff366720cc7acb4dd363570fb99634f0d45f7"}
Sep 18 20:29:59.192737 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:29:59.192612    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-102c83-7c4d97c84b-jlrwh" event={"ID":"c43f0b3f-b95f-4e4a-81dc-2d6c248a41fa","Type":"ContainerDied","Data":"adff5afddd978ae168268dde05da45002046a33849cdfdf9b1d531a17183671e"}
Sep 18 20:29:59.192737 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:29:59.192640    2532 scope.go:117] "RemoveContainer" containerID="69cd719fa7c93d6b4e36a06787aff366720cc7acb4dd363570fb99634f0d45f7"
Sep 18 20:29:59.202762 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:29:59.202747    2532 scope.go:117] "RemoveContainer" containerID="69cd719fa7c93d6b4e36a06787aff366720cc7acb4dd363570fb99634f0d45f7"
Sep 18 20:29:59.203008 ip-10-0-130-81 kubenswrapper[2532]: E0918 20:29:59.202988    2532 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"69cd719fa7c93d6b4e36a06787aff366720cc7acb4dd363570fb99634f0d45f7\": container with ID starting with 69cd719fa7c93d6b4e36a06787aff366720cc7acb4dd363570fb99634f0d45f7 not found: ID does not exist" containerID="69cd719fa7c93d6b4e36a06787aff366720cc7acb4dd363570fb99634f0d45f7"
Sep 18 20:29:59.203079 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:29:59.203014    2532 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"69cd719fa7c93d6b4e36a06787aff366720cc7acb4dd363570fb99634f0d45f7"} err="failed to get container status \"69cd719fa7c93d6b4e36a06787aff366720cc7acb4dd363570fb99634f0d45f7\": rpc error: code = NotFound desc = could not find container \"69cd719fa7c93d6b4e36a06787aff366720cc7acb4dd363570fb99634f0d45f7\": container with ID starting with 69cd719fa7c93d6b4e36a06787aff366720cc7acb4dd363570fb99634f0d45f7 not found: ID does not exist"
Sep 18 20:29:59.215592 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:29:59.215565    2532 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-102c83-7c4d97c84b-jlrwh"]
Sep 18 20:29:59.219174 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:29:59.219149    2532 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-102c83-7c4d97c84b-jlrwh"]
Sep 18 20:29:59.346576 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:29:59.346544    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:30:00.010772 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:30:00.010739    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:30:00.306969 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:30:00.306886    2532 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c43f0b3f-b95f-4e4a-81dc-2d6c248a41fa" path="/var/lib/kubelet/pods/c43f0b3f-b95f-4e4a-81dc-2d6c248a41fa/volumes"
Sep 18 20:30:00.820210 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:30:00.820183    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:30:01.532234 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:30:01.532206    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:30:01.813674 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:30:01.813586    2532 prober.go:120] "Probe failed" probeType="Readiness" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-102c83-7c4d97c84b-jlrwh" podUID="c43f0b3f-b95f-4e4a-81dc-2d6c248a41fa" containerName="maas-api" probeResult="failure" output="Get \"https://10.133.0.66:8443/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Sep 18 20:30:02.243799 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:30:02.243765    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:30:05.124224 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:30:05.124191    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:30:05.225150 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:30:05.225108    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:30:05.370151 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:30:05.370121    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:30:05.830772 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:30:05.830739    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:30:06.580034 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:30:06.580005    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:30:07.218814 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:30:07.218777    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:30:07.995341 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:30:07.995305    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:30:08.774563 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:30:08.774531    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:30:09.521374 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:30:09.521341    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:30:11.415889 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:30:11.415854    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-vptdl"]
Sep 18 20:30:16.213847 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:30:16.213814    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:30:16.281845 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:30:16.281817    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:30:16.381744 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:30:16.381715    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:30:16.439672 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:30:16.439643    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:30:16.754537 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:30:16.754477    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:30:16.856002 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:30:16.855966    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:30:17.525766 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:30:17.525732    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:30:18.205298 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:30:18.205261    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:30:18.898409 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:30:18.898375    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:30:22.497835 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:30:22.497797    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:30:22.600069 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:30:22.600040    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:30:22.743029 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:30:22.742995    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:30:23.317920 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:30:23.317881    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:30:24.015067 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:30:24.015039    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:30:24.614787 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:30:24.614757    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:30:24.892899 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:30:24.892823    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-vptdl"]
Sep 18 20:30:25.254065 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:30:25.254032    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:30:26.025321 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:30:26.025290    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:30:33.610443 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:30:33.610407    2532 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-9a5190-5ccb744cb6-drw2b"]
Sep 18 20:30:33.610897 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:30:33.610648    2532 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-9a5190-5ccb744cb6-drw2b" podUID="ae7e4693-ee7e-4cf7-a545-c43fa542d35f" containerName="maas-api" containerID="cri-o://093a51b40286172019d8f08e674aabcc943f362ef7161074cbf8f6aed610e26d" gracePeriod=30
Sep 18 20:30:33.650725 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:30:33.650688    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:30:33.687822 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:30:33.687784    2532 prober.go:120] "Probe failed" probeType="Readiness" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-9a5190-5ccb744cb6-drw2b" podUID="ae7e4693-ee7e-4cf7-a545-c43fa542d35f" containerName="maas-api" probeResult="failure" output="Get \"https://10.133.0.56:8443/health\": dial tcp 10.133.0.56:8443: connect: connection refused"
Sep 18 20:30:33.719855 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:30:33.719818    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:30:33.819748 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:30:33.819713    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:30:33.860187 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:30:33.860163    2532 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-9a5190-5ccb744cb6-drw2b"
Sep 18 20:30:33.882342 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:30:33.882234    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:30:34.001440 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:30:34.001411    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ae7e4693-ee7e-4cf7-a545-c43fa542d35f-maas-api-tls\") pod \"ae7e4693-ee7e-4cf7-a545-c43fa542d35f\" (UID: \"ae7e4693-ee7e-4cf7-a545-c43fa542d35f\") "
Sep 18 20:30:34.001640 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:30:34.001463    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-86km4\" (UniqueName: \"kubernetes.io/projected/ae7e4693-ee7e-4cf7-a545-c43fa542d35f-kube-api-access-86km4\") pod \"ae7e4693-ee7e-4cf7-a545-c43fa542d35f\" (UID: \"ae7e4693-ee7e-4cf7-a545-c43fa542d35f\") "
Sep 18 20:30:34.001640 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:30:34.001562    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ae7e4693-ee7e-4cf7-a545-c43fa542d35f-openshift-service-ca\") pod \"ae7e4693-ee7e-4cf7-a545-c43fa542d35f\" (UID: \"ae7e4693-ee7e-4cf7-a545-c43fa542d35f\") "
Sep 18 20:30:34.001640 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:30:34.001598    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ae7e4693-ee7e-4cf7-a545-c43fa542d35f-metrics-certs\") pod \"ae7e4693-ee7e-4cf7-a545-c43fa542d35f\" (UID: \"ae7e4693-ee7e-4cf7-a545-c43fa542d35f\") "
Sep 18 20:30:34.001982 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:30:34.001956    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ae7e4693-ee7e-4cf7-a545-c43fa542d35f-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "ae7e4693-ee7e-4cf7-a545-c43fa542d35f" (UID: "ae7e4693-ee7e-4cf7-a545-c43fa542d35f"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 20:30:34.003878 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:30:34.003843    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ae7e4693-ee7e-4cf7-a545-c43fa542d35f-kube-api-access-86km4" (OuterVolumeSpecName: "kube-api-access-86km4") pod "ae7e4693-ee7e-4cf7-a545-c43fa542d35f" (UID: "ae7e4693-ee7e-4cf7-a545-c43fa542d35f"). InnerVolumeSpecName "kube-api-access-86km4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 20:30:34.003878 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:30:34.003871    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ae7e4693-ee7e-4cf7-a545-c43fa542d35f-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "ae7e4693-ee7e-4cf7-a545-c43fa542d35f" (UID: "ae7e4693-ee7e-4cf7-a545-c43fa542d35f"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 20:30:34.004016 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:30:34.003858    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ae7e4693-ee7e-4cf7-a545-c43fa542d35f-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "ae7e4693-ee7e-4cf7-a545-c43fa542d35f" (UID: "ae7e4693-ee7e-4cf7-a545-c43fa542d35f"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 20:30:34.102431 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:30:34.102394    2532 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ae7e4693-ee7e-4cf7-a545-c43fa542d35f-openshift-service-ca\") on node \"ip-10-0-130-81.ec2.internal\" DevicePath \"\""
Sep 18 20:30:34.102431 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:30:34.102425    2532 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ae7e4693-ee7e-4cf7-a545-c43fa542d35f-metrics-certs\") on node \"ip-10-0-130-81.ec2.internal\" DevicePath \"\""
Sep 18 20:30:34.102431 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:30:34.102437    2532 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ae7e4693-ee7e-4cf7-a545-c43fa542d35f-maas-api-tls\") on node \"ip-10-0-130-81.ec2.internal\" DevicePath \"\""
Sep 18 20:30:34.102701 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:30:34.102449    2532 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-86km4\" (UniqueName: \"kubernetes.io/projected/ae7e4693-ee7e-4cf7-a545-c43fa542d35f-kube-api-access-86km4\") on node \"ip-10-0-130-81.ec2.internal\" DevicePath \"\""
Sep 18 20:30:34.269050 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:30:34.269009    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:30:34.364153 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:30:34.364118    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:30:34.387273 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:30:34.387240    2532 generic.go:356] "Generic (PLEG): container finished" podID="ae7e4693-ee7e-4cf7-a545-c43fa542d35f" containerID="093a51b40286172019d8f08e674aabcc943f362ef7161074cbf8f6aed610e26d" exitCode=0
Sep 18 20:30:34.387464 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:30:34.387294    2532 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-9a5190-5ccb744cb6-drw2b"
Sep 18 20:30:34.387464 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:30:34.387300    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-9a5190-5ccb744cb6-drw2b" event={"ID":"ae7e4693-ee7e-4cf7-a545-c43fa542d35f","Type":"ContainerDied","Data":"093a51b40286172019d8f08e674aabcc943f362ef7161074cbf8f6aed610e26d"}
Sep 18 20:30:34.387464 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:30:34.387341    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-9a5190-5ccb744cb6-drw2b" event={"ID":"ae7e4693-ee7e-4cf7-a545-c43fa542d35f","Type":"ContainerDied","Data":"10167baf2402a593d5327db4ae00ea3aaef8811605e1cc5e7a79d4e473122cdc"}
Sep 18 20:30:34.387464 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:30:34.387363    2532 scope.go:117] "RemoveContainer" containerID="093a51b40286172019d8f08e674aabcc943f362ef7161074cbf8f6aed610e26d"
Sep 18 20:30:34.396443 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:30:34.396429    2532 scope.go:117] "RemoveContainer" containerID="093a51b40286172019d8f08e674aabcc943f362ef7161074cbf8f6aed610e26d"
Sep 18 20:30:34.396753 ip-10-0-130-81 kubenswrapper[2532]: E0918 20:30:34.396734    2532 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"093a51b40286172019d8f08e674aabcc943f362ef7161074cbf8f6aed610e26d\": container with ID starting with 093a51b40286172019d8f08e674aabcc943f362ef7161074cbf8f6aed610e26d not found: ID does not exist" containerID="093a51b40286172019d8f08e674aabcc943f362ef7161074cbf8f6aed610e26d"
Sep 18 20:30:34.396817 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:30:34.396764    2532 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"093a51b40286172019d8f08e674aabcc943f362ef7161074cbf8f6aed610e26d"} err="failed to get container status \"093a51b40286172019d8f08e674aabcc943f362ef7161074cbf8f6aed610e26d\": rpc error: code = NotFound desc = could not find container \"093a51b40286172019d8f08e674aabcc943f362ef7161074cbf8f6aed610e26d\": container with ID starting with 093a51b40286172019d8f08e674aabcc943f362ef7161074cbf8f6aed610e26d not found: ID does not exist"
Sep 18 20:30:34.404574 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:30:34.404550    2532 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-9a5190-5ccb744cb6-drw2b"]
Sep 18 20:30:34.408768 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:30:34.408747    2532 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-9a5190-5ccb744cb6-drw2b"]
Sep 18 20:30:34.900729 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:30:34.900690    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:30:35.790822 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:30:35.790790    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:30:36.306456 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:30:36.306407    2532 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ae7e4693-ee7e-4cf7-a545-c43fa542d35f" path="/var/lib/kubelet/pods/ae7e4693-ee7e-4cf7-a545-c43fa542d35f/volumes"
Sep 18 20:30:38.908704 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:30:38.908672    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-vptdl"]
Sep 18 20:30:40.183701 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:30:40.183664    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:30:40.373154 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:30:40.373123    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:30:40.506786 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:30:40.506749    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:30:43.440910 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:30:43.440881    2532 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w2-bab50d/sub-test-model-bab50d-kserve-546f9db6c-9zflz"]
Sep 18 20:30:43.467372 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:30:43.461432    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:30:44.516957 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:30:44.516916    2532 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w2-35558b/sub-test-model-35558b-kserve-6fd4fb4567-9zj2l"]
Sep 18 20:30:44.538874 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:30:44.538830    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:30:51.502955 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:30:51.502920    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-vptdl"]
Sep 18 20:30:53.428439 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:30:53.428367    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:30:58.459272 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:30:58.459233    2532 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-w2-bab50d/sub-test-model-bab50d-kserve-546f9db6c-9zflz" podUID="904d4141-71a8-47a3-8678-a2fd2e729c18" containerName="main" containerID="cri-o://171010b82ab79d499e6d5f7df5c045b7fada7ee70cb0286c4474c06f6748957e" gracePeriod=45
Sep 18 20:30:58.718537 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:30:58.718457    2532 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w2-bab50d/sub-test-model-bab50d-kserve-546f9db6c-9zflz"
Sep 18 20:30:58.729903 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:30:58.729882    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/904d4141-71a8-47a3-8678-a2fd2e729c18-model-cache\") pod \"904d4141-71a8-47a3-8678-a2fd2e729c18\" (UID: \"904d4141-71a8-47a3-8678-a2fd2e729c18\") "
Sep 18 20:30:58.730000 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:30:58.729916    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/904d4141-71a8-47a3-8678-a2fd2e729c18-tmp-dir\") pod \"904d4141-71a8-47a3-8678-a2fd2e729c18\" (UID: \"904d4141-71a8-47a3-8678-a2fd2e729c18\") "
Sep 18 20:30:58.730000 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:30:58.729936    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/904d4141-71a8-47a3-8678-a2fd2e729c18-tls-certs\") pod \"904d4141-71a8-47a3-8678-a2fd2e729c18\" (UID: \"904d4141-71a8-47a3-8678-a2fd2e729c18\") "
Sep 18 20:30:58.730000 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:30:58.729981    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p8wpv\" (UniqueName: \"kubernetes.io/projected/904d4141-71a8-47a3-8678-a2fd2e729c18-kube-api-access-p8wpv\") pod \"904d4141-71a8-47a3-8678-a2fd2e729c18\" (UID: \"904d4141-71a8-47a3-8678-a2fd2e729c18\") "
Sep 18 20:30:58.730138 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:30:58.730015    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/904d4141-71a8-47a3-8678-a2fd2e729c18-home\") pod \"904d4141-71a8-47a3-8678-a2fd2e729c18\" (UID: \"904d4141-71a8-47a3-8678-a2fd2e729c18\") "
Sep 18 20:30:58.730138 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:30:58.730043    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/904d4141-71a8-47a3-8678-a2fd2e729c18-dshm\") pod \"904d4141-71a8-47a3-8678-a2fd2e729c18\" (UID: \"904d4141-71a8-47a3-8678-a2fd2e729c18\") "
Sep 18 20:30:58.730243 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:30:58.730146    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/904d4141-71a8-47a3-8678-a2fd2e729c18-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "904d4141-71a8-47a3-8678-a2fd2e729c18" (UID: "904d4141-71a8-47a3-8678-a2fd2e729c18"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 20:30:58.730243 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:30:58.730165    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/904d4141-71a8-47a3-8678-a2fd2e729c18-model-cache" (OuterVolumeSpecName: "model-cache") pod "904d4141-71a8-47a3-8678-a2fd2e729c18" (UID: "904d4141-71a8-47a3-8678-a2fd2e729c18"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 20:30:58.730347 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:30:58.730305    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/904d4141-71a8-47a3-8678-a2fd2e729c18-home" (OuterVolumeSpecName: "home") pod "904d4141-71a8-47a3-8678-a2fd2e729c18" (UID: "904d4141-71a8-47a3-8678-a2fd2e729c18"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 20:30:58.730411 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:30:58.730394    2532 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/904d4141-71a8-47a3-8678-a2fd2e729c18-model-cache\") on node \"ip-10-0-130-81.ec2.internal\" DevicePath \"\""
Sep 18 20:30:58.730472 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:30:58.730415    2532 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/904d4141-71a8-47a3-8678-a2fd2e729c18-tmp-dir\") on node \"ip-10-0-130-81.ec2.internal\" DevicePath \"\""
Sep 18 20:30:58.730472 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:30:58.730423    2532 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/904d4141-71a8-47a3-8678-a2fd2e729c18-home\") on node \"ip-10-0-130-81.ec2.internal\" DevicePath \"\""
Sep 18 20:30:58.732104 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:30:58.732067    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/904d4141-71a8-47a3-8678-a2fd2e729c18-dshm" (OuterVolumeSpecName: "dshm") pod "904d4141-71a8-47a3-8678-a2fd2e729c18" (UID: "904d4141-71a8-47a3-8678-a2fd2e729c18"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 20:30:58.732213 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:30:58.732133    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/904d4141-71a8-47a3-8678-a2fd2e729c18-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "904d4141-71a8-47a3-8678-a2fd2e729c18" (UID: "904d4141-71a8-47a3-8678-a2fd2e729c18"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 20:30:58.732213 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:30:58.732147    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/904d4141-71a8-47a3-8678-a2fd2e729c18-kube-api-access-p8wpv" (OuterVolumeSpecName: "kube-api-access-p8wpv") pod "904d4141-71a8-47a3-8678-a2fd2e729c18" (UID: "904d4141-71a8-47a3-8678-a2fd2e729c18"). InnerVolumeSpecName "kube-api-access-p8wpv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 20:30:58.831446 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:30:58.831411    2532 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/904d4141-71a8-47a3-8678-a2fd2e729c18-tls-certs\") on node \"ip-10-0-130-81.ec2.internal\" DevicePath \"\""
Sep 18 20:30:58.831446 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:30:58.831440    2532 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p8wpv\" (UniqueName: \"kubernetes.io/projected/904d4141-71a8-47a3-8678-a2fd2e729c18-kube-api-access-p8wpv\") on node \"ip-10-0-130-81.ec2.internal\" DevicePath \"\""
Sep 18 20:30:58.831446 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:30:58.831455    2532 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/904d4141-71a8-47a3-8678-a2fd2e729c18-dshm\") on node \"ip-10-0-130-81.ec2.internal\" DevicePath \"\""
Sep 18 20:30:59.530954 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:30:59.530921    2532 generic.go:356] "Generic (PLEG): container finished" podID="904d4141-71a8-47a3-8678-a2fd2e729c18" containerID="171010b82ab79d499e6d5f7df5c045b7fada7ee70cb0286c4474c06f6748957e" exitCode=0
Sep 18 20:30:59.531366 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:30:59.530962    2532 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w2-bab50d/sub-test-model-bab50d-kserve-546f9db6c-9zflz"
Sep 18 20:30:59.531366 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:30:59.531006    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w2-bab50d/sub-test-model-bab50d-kserve-546f9db6c-9zflz" event={"ID":"904d4141-71a8-47a3-8678-a2fd2e729c18","Type":"ContainerDied","Data":"171010b82ab79d499e6d5f7df5c045b7fada7ee70cb0286c4474c06f6748957e"}
Sep 18 20:30:59.531366 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:30:59.531045    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w2-bab50d/sub-test-model-bab50d-kserve-546f9db6c-9zflz" event={"ID":"904d4141-71a8-47a3-8678-a2fd2e729c18","Type":"ContainerDied","Data":"c240674e3631c260b450987d2f0e23289b5c0c4d2e9f0368ad0f67b65529d3c5"}
Sep 18 20:30:59.531366 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:30:59.531061    2532 scope.go:117] "RemoveContainer" containerID="171010b82ab79d499e6d5f7df5c045b7fada7ee70cb0286c4474c06f6748957e"
Sep 18 20:30:59.535849 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:30:59.535803    2532 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-w2-35558b/sub-test-model-35558b-kserve-6fd4fb4567-9zj2l" podUID="f0a6570f-5955-4f14-a162-8ff27ca89629" containerName="main" containerID="cri-o://201ce52f97f1a0646a6cf4e574b20e048eb7b5f5ccccf13aa524645e875c9623" gracePeriod=45
Sep 18 20:30:59.540913 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:30:59.540897    2532 scope.go:117] "RemoveContainer" containerID="171010b82ab79d499e6d5f7df5c045b7fada7ee70cb0286c4474c06f6748957e"
Sep 18 20:30:59.541185 ip-10-0-130-81 kubenswrapper[2532]: E0918 20:30:59.541166    2532 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"171010b82ab79d499e6d5f7df5c045b7fada7ee70cb0286c4474c06f6748957e\": container with ID starting with 171010b82ab79d499e6d5f7df5c045b7fada7ee70cb0286c4474c06f6748957e not found: ID does not exist" containerID="171010b82ab79d499e6d5f7df5c045b7fada7ee70cb0286c4474c06f6748957e"
Sep 18 20:30:59.541260 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:30:59.541194    2532 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"171010b82ab79d499e6d5f7df5c045b7fada7ee70cb0286c4474c06f6748957e"} err="failed to get container status \"171010b82ab79d499e6d5f7df5c045b7fada7ee70cb0286c4474c06f6748957e\": rpc error: code = NotFound desc = could not find container \"171010b82ab79d499e6d5f7df5c045b7fada7ee70cb0286c4474c06f6748957e\": container with ID starting with 171010b82ab79d499e6d5f7df5c045b7fada7ee70cb0286c4474c06f6748957e not found: ID does not exist"
Sep 18 20:30:59.553949 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:30:59.553902    2532 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w2-bab50d/sub-test-model-bab50d-kserve-546f9db6c-9zflz"]
Sep 18 20:30:59.556562 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:30:59.556542    2532 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-w2-bab50d/sub-test-model-bab50d-kserve-546f9db6c-9zflz"]
Sep 18 20:30:59.782795 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:30:59.782735    2532 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w2-35558b/sub-test-model-35558b-kserve-6fd4fb4567-9zj2l"
Sep 18 20:30:59.841370 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:30:59.841341    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f0a6570f-5955-4f14-a162-8ff27ca89629-home\") pod \"f0a6570f-5955-4f14-a162-8ff27ca89629\" (UID: \"f0a6570f-5955-4f14-a162-8ff27ca89629\") "
Sep 18 20:30:59.841565 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:30:59.841436    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wpwbr\" (UniqueName: \"kubernetes.io/projected/f0a6570f-5955-4f14-a162-8ff27ca89629-kube-api-access-wpwbr\") pod \"f0a6570f-5955-4f14-a162-8ff27ca89629\" (UID: \"f0a6570f-5955-4f14-a162-8ff27ca89629\") "
Sep 18 20:30:59.841565 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:30:59.841461    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f0a6570f-5955-4f14-a162-8ff27ca89629-tmp-dir\") pod \"f0a6570f-5955-4f14-a162-8ff27ca89629\" (UID: \"f0a6570f-5955-4f14-a162-8ff27ca89629\") "
Sep 18 20:30:59.841565 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:30:59.841477    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f0a6570f-5955-4f14-a162-8ff27ca89629-dshm\") pod \"f0a6570f-5955-4f14-a162-8ff27ca89629\" (UID: \"f0a6570f-5955-4f14-a162-8ff27ca89629\") "
Sep 18 20:30:59.841565 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:30:59.841549    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f0a6570f-5955-4f14-a162-8ff27ca89629-tls-certs\") pod \"f0a6570f-5955-4f14-a162-8ff27ca89629\" (UID: \"f0a6570f-5955-4f14-a162-8ff27ca89629\") "
Sep 18 20:30:59.841797 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:30:59.841601    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f0a6570f-5955-4f14-a162-8ff27ca89629-model-cache\") pod \"f0a6570f-5955-4f14-a162-8ff27ca89629\" (UID: \"f0a6570f-5955-4f14-a162-8ff27ca89629\") "
Sep 18 20:30:59.841797 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:30:59.841652    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f0a6570f-5955-4f14-a162-8ff27ca89629-home" (OuterVolumeSpecName: "home") pod "f0a6570f-5955-4f14-a162-8ff27ca89629" (UID: "f0a6570f-5955-4f14-a162-8ff27ca89629"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 20:30:59.841797 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:30:59.841748    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f0a6570f-5955-4f14-a162-8ff27ca89629-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "f0a6570f-5955-4f14-a162-8ff27ca89629" (UID: "f0a6570f-5955-4f14-a162-8ff27ca89629"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 20:30:59.841933 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:30:59.841889    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f0a6570f-5955-4f14-a162-8ff27ca89629-model-cache" (OuterVolumeSpecName: "model-cache") pod "f0a6570f-5955-4f14-a162-8ff27ca89629" (UID: "f0a6570f-5955-4f14-a162-8ff27ca89629"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 20:30:59.841977 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:30:59.841954    2532 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f0a6570f-5955-4f14-a162-8ff27ca89629-home\") on node \"ip-10-0-130-81.ec2.internal\" DevicePath \"\""
Sep 18 20:30:59.841977 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:30:59.841968    2532 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f0a6570f-5955-4f14-a162-8ff27ca89629-tmp-dir\") on node \"ip-10-0-130-81.ec2.internal\" DevicePath \"\""
Sep 18 20:30:59.843904 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:30:59.843887    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f0a6570f-5955-4f14-a162-8ff27ca89629-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "f0a6570f-5955-4f14-a162-8ff27ca89629" (UID: "f0a6570f-5955-4f14-a162-8ff27ca89629"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 20:30:59.844089 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:30:59.844061    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f0a6570f-5955-4f14-a162-8ff27ca89629-kube-api-access-wpwbr" (OuterVolumeSpecName: "kube-api-access-wpwbr") pod "f0a6570f-5955-4f14-a162-8ff27ca89629" (UID: "f0a6570f-5955-4f14-a162-8ff27ca89629"). InnerVolumeSpecName "kube-api-access-wpwbr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 20:30:59.844089 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:30:59.844069    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f0a6570f-5955-4f14-a162-8ff27ca89629-dshm" (OuterVolumeSpecName: "dshm") pod "f0a6570f-5955-4f14-a162-8ff27ca89629" (UID: "f0a6570f-5955-4f14-a162-8ff27ca89629"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 20:30:59.942586 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:30:59.942553    2532 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wpwbr\" (UniqueName: \"kubernetes.io/projected/f0a6570f-5955-4f14-a162-8ff27ca89629-kube-api-access-wpwbr\") on node \"ip-10-0-130-81.ec2.internal\" DevicePath \"\""
Sep 18 20:30:59.942586 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:30:59.942579    2532 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f0a6570f-5955-4f14-a162-8ff27ca89629-dshm\") on node \"ip-10-0-130-81.ec2.internal\" DevicePath \"\""
Sep 18 20:30:59.942586 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:30:59.942589    2532 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f0a6570f-5955-4f14-a162-8ff27ca89629-tls-certs\") on node \"ip-10-0-130-81.ec2.internal\" DevicePath \"\""
Sep 18 20:30:59.942586 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:30:59.942597    2532 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f0a6570f-5955-4f14-a162-8ff27ca89629-model-cache\") on node \"ip-10-0-130-81.ec2.internal\" DevicePath \"\""
Sep 18 20:31:00.306964 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:31:00.306924    2532 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="904d4141-71a8-47a3-8678-a2fd2e729c18" path="/var/lib/kubelet/pods/904d4141-71a8-47a3-8678-a2fd2e729c18/volumes"
Sep 18 20:31:00.539336 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:31:00.539298    2532 generic.go:356] "Generic (PLEG): container finished" podID="f0a6570f-5955-4f14-a162-8ff27ca89629" containerID="201ce52f97f1a0646a6cf4e574b20e048eb7b5f5ccccf13aa524645e875c9623" exitCode=0
Sep 18 20:31:00.539786 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:31:00.539347    2532 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w2-35558b/sub-test-model-35558b-kserve-6fd4fb4567-9zj2l"
Sep 18 20:31:00.539786 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:31:00.539382    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w2-35558b/sub-test-model-35558b-kserve-6fd4fb4567-9zj2l" event={"ID":"f0a6570f-5955-4f14-a162-8ff27ca89629","Type":"ContainerDied","Data":"201ce52f97f1a0646a6cf4e574b20e048eb7b5f5ccccf13aa524645e875c9623"}
Sep 18 20:31:00.539786 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:31:00.539428    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w2-35558b/sub-test-model-35558b-kserve-6fd4fb4567-9zj2l" event={"ID":"f0a6570f-5955-4f14-a162-8ff27ca89629","Type":"ContainerDied","Data":"883db7505160222ce852c351303060eac7ac88474bc0b85c2f6b4ce96e9e4d51"}
Sep 18 20:31:00.539786 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:31:00.539444    2532 scope.go:117] "RemoveContainer" containerID="201ce52f97f1a0646a6cf4e574b20e048eb7b5f5ccccf13aa524645e875c9623"
Sep 18 20:31:00.555442 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:31:00.555420    2532 scope.go:117] "RemoveContainer" containerID="201ce52f97f1a0646a6cf4e574b20e048eb7b5f5ccccf13aa524645e875c9623"
Sep 18 20:31:00.555755 ip-10-0-130-81 kubenswrapper[2532]: E0918 20:31:00.555736    2532 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"201ce52f97f1a0646a6cf4e574b20e048eb7b5f5ccccf13aa524645e875c9623\": container with ID starting with 201ce52f97f1a0646a6cf4e574b20e048eb7b5f5ccccf13aa524645e875c9623 not found: ID does not exist" containerID="201ce52f97f1a0646a6cf4e574b20e048eb7b5f5ccccf13aa524645e875c9623"
Sep 18 20:31:00.555810 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:31:00.555764    2532 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"201ce52f97f1a0646a6cf4e574b20e048eb7b5f5ccccf13aa524645e875c9623"} err="failed to get container status \"201ce52f97f1a0646a6cf4e574b20e048eb7b5f5ccccf13aa524645e875c9623\": rpc error: code = NotFound desc = could not find container \"201ce52f97f1a0646a6cf4e574b20e048eb7b5f5ccccf13aa524645e875c9623\": container with ID starting with 201ce52f97f1a0646a6cf4e574b20e048eb7b5f5ccccf13aa524645e875c9623 not found: ID does not exist"
Sep 18 20:31:00.562645 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:31:00.562593    2532 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w2-35558b/sub-test-model-35558b-kserve-6fd4fb4567-9zj2l"]
Sep 18 20:31:00.567084 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:31:00.567061    2532 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-w2-35558b/sub-test-model-35558b-kserve-6fd4fb4567-9zj2l"]
Sep 18 20:31:01.988807 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:31:01.988775    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-vptdl"]
Sep 18 20:31:02.306463 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:31:02.306427    2532 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f0a6570f-5955-4f14-a162-8ff27ca89629" path="/var/lib/kubelet/pods/f0a6570f-5955-4f14-a162-8ff27ca89629/volumes"
Sep 18 20:31:10.438328 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:31:10.438301    2532 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-m7wdd_32e33d39-7826-499d-bc01-0487b7580e7a/console-operator/1.log"
Sep 18 20:31:10.440016 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:31:10.439997    2532 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-m7wdd_32e33d39-7826-499d-bc01-0487b7580e7a/console-operator/1.log"
Sep 18 20:31:10.447044 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:31:10.447025    2532 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rtw9n_73fbf16b-3549-4a6c-a959-b633237ebd0c/ovn-acl-logging/0.log"
Sep 18 20:31:10.448795 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:31:10.448780    2532 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rtw9n_73fbf16b-3549-4a6c-a959-b633237ebd0c/ovn-acl-logging/0.log"
Sep 18 20:31:15.546341 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:31:15.546301    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:31:35.796000 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:31:35.795965    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:31:39.595189 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:31:39.595151    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-vptdl"]
Sep 18 20:31:57.597444 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:31:57.597409    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-vptdl"]
Sep 18 20:32:06.597744 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:32:06.597698    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-vptdl"]
Sep 18 20:32:13.283315 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:32:13.283280    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:32:14.121702 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:32:14.121671    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:32:14.556344 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:32:14.556310    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:32:14.825901 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:32:14.825823    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:32:15.271784 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:32:15.271748    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:32:17.290801 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:32:17.290762    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-vptdl"]
Sep 18 20:32:17.867515 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:32:17.867469    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:32:18.208597 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:32:18.208567    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:32:18.398070 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:32:18.398032    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:32:18.750758 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:32:18.750724    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:32:19.590868 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:32:19.590833    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:32:19.691461 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:32:19.691415    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:32:20.013280 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:32:20.013240    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:32:20.112068 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:32:20.112040    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:32:20.210806 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:32:20.210773    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:32:31.495634 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:32:31.495544    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-vptdl"]
Sep 18 20:32:37.855590 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:32:37.855554    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:32:37.955859 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:32:37.955829    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:32:42.215811 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:32:42.215773    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:32:42.723659 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:32:42.723624    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:32:43.108590 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:32:43.108469    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:32:43.345915 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:32:43.345882    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:32:43.746196 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:32:43.746168    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:32:44.049078 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:32:44.048997    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:32:44.148811 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:32:44.148778    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:32:44.290697 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:32:44.290662    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:32:46.698527 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:32:46.698471    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:32:47.206857 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:32:47.206825    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:32:47.412032 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:32:47.412000    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:32:47.508286 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:32:47.508204    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:32:48.196116 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:32:48.196083    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:32:48.838409 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:32:48.838374    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:32:49.515738 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:32:49.515709    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:32:49.616107 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:32:49.616080    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:32:49.719095 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:32:49.719057    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:32:49.956198 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:32:49.956172    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:32:50.063120 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:32:50.063091    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:32:54.903443 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:32:54.903401    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-vptdl"]
Sep 18 20:32:55.323962 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:32:55.323931    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:33:08.635331 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:33:08.635298    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:33:10.314982 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:33:10.314954    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:33:11.491710 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:33:11.491642    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:33:12.110764 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:33:12.110730    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:33:12.335720 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:33:12.335673    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:33:12.729427 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:33:12.729383    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:33:13.611569 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:33:13.611508    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:33:14.238438 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:33:14.238399    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:33:25.492221 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:33:25.492188    2532 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-ipp-a-8cdaea/ipp-route-8cdaea-kserve-849b5f488c-wwzff"]
Sep 18 20:33:25.492651 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:33:25.492578    2532 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ae7e4693-ee7e-4cf7-a545-c43fa542d35f" containerName="maas-api"
Sep 18 20:33:25.492651 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:33:25.492594    2532 state_mem.go:107] "Deleted CPUSet assignment" podUID="ae7e4693-ee7e-4cf7-a545-c43fa542d35f" containerName="maas-api"
Sep 18 20:33:25.492651 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:33:25.492606    2532 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f0a6570f-5955-4f14-a162-8ff27ca89629" containerName="main"
Sep 18 20:33:25.492651 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:33:25.492612    2532 state_mem.go:107] "Deleted CPUSet assignment" podUID="f0a6570f-5955-4f14-a162-8ff27ca89629" containerName="main"
Sep 18 20:33:25.492651 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:33:25.492622    2532 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="904d4141-71a8-47a3-8678-a2fd2e729c18" containerName="main"
Sep 18 20:33:25.492651 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:33:25.492627    2532 state_mem.go:107] "Deleted CPUSet assignment" podUID="904d4141-71a8-47a3-8678-a2fd2e729c18" containerName="main"
Sep 18 20:33:25.492651 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:33:25.492638    2532 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c43f0b3f-b95f-4e4a-81dc-2d6c248a41fa" containerName="maas-api"
Sep 18 20:33:25.492651 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:33:25.492644    2532 state_mem.go:107] "Deleted CPUSet assignment" podUID="c43f0b3f-b95f-4e4a-81dc-2d6c248a41fa" containerName="maas-api"
Sep 18 20:33:25.493000 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:33:25.492699    2532 memory_manager.go:368] "RemoveStaleState removing state" podUID="ae7e4693-ee7e-4cf7-a545-c43fa542d35f" containerName="maas-api"
Sep 18 20:33:25.493000 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:33:25.492707    2532 memory_manager.go:368] "RemoveStaleState removing state" podUID="904d4141-71a8-47a3-8678-a2fd2e729c18" containerName="main"
Sep 18 20:33:25.493000 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:33:25.492714    2532 memory_manager.go:368] "RemoveStaleState removing state" podUID="f0a6570f-5955-4f14-a162-8ff27ca89629" containerName="main"
Sep 18 20:33:25.493000 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:33:25.492720    2532 memory_manager.go:368] "RemoveStaleState removing state" podUID="c43f0b3f-b95f-4e4a-81dc-2d6c248a41fa" containerName="maas-api"
Sep 18 20:33:25.495294 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:33:25.495277    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-8cdaea/ipp-route-8cdaea-kserve-849b5f488c-wwzff"
Sep 18 20:33:25.498064 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:33:25.498036    2532 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-ipp-a-8cdaea\"/\"kube-root-ca.crt\""
Sep 18 20:33:25.498064 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:33:25.498057    2532 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-ipp-a-8cdaea\"/\"openshift-service-ca.crt\""
Sep 18 20:33:25.499464 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:33:25.499444    2532 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-ipp-a-8cdaea\"/\"default-dockercfg-mhlr5\""
Sep 18 20:33:25.499591 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:33:25.499523    2532 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-ipp-a-8cdaea\"/\"ipp-route-8cdaea-kserve-self-signed-certs\""
Sep 18 20:33:25.506632 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:33:25.506609    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-ipp-a-8cdaea/ipp-route-8cdaea-kserve-849b5f488c-wwzff"]
Sep 18 20:33:25.526549 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:33:25.526512    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:33:25.558025 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:33:25.557996    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f3969483-1cdd-4132-9344-6748326b8a81-tls-certs\") pod \"ipp-route-8cdaea-kserve-849b5f488c-wwzff\" (UID: \"f3969483-1cdd-4132-9344-6748326b8a81\") " pod="ai-tenant-e2e-ipp-a-8cdaea/ipp-route-8cdaea-kserve-849b5f488c-wwzff"
Sep 18 20:33:25.558195 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:33:25.558065    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f3969483-1cdd-4132-9344-6748326b8a81-home\") pod \"ipp-route-8cdaea-kserve-849b5f488c-wwzff\" (UID: \"f3969483-1cdd-4132-9344-6748326b8a81\") " pod="ai-tenant-e2e-ipp-a-8cdaea/ipp-route-8cdaea-kserve-849b5f488c-wwzff"
Sep 18 20:33:25.558195 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:33:25.558098    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f3969483-1cdd-4132-9344-6748326b8a81-dshm\") pod \"ipp-route-8cdaea-kserve-849b5f488c-wwzff\" (UID: \"f3969483-1cdd-4132-9344-6748326b8a81\") " pod="ai-tenant-e2e-ipp-a-8cdaea/ipp-route-8cdaea-kserve-849b5f488c-wwzff"
Sep 18 20:33:25.558195 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:33:25.558125    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bdkxx\" (UniqueName: \"kubernetes.io/projected/f3969483-1cdd-4132-9344-6748326b8a81-kube-api-access-bdkxx\") pod \"ipp-route-8cdaea-kserve-849b5f488c-wwzff\" (UID: \"f3969483-1cdd-4132-9344-6748326b8a81\") " pod="ai-tenant-e2e-ipp-a-8cdaea/ipp-route-8cdaea-kserve-849b5f488c-wwzff"
Sep 18 20:33:25.558330 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:33:25.558200    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f3969483-1cdd-4132-9344-6748326b8a81-tmp-dir\") pod \"ipp-route-8cdaea-kserve-849b5f488c-wwzff\" (UID: \"f3969483-1cdd-4132-9344-6748326b8a81\") " pod="ai-tenant-e2e-ipp-a-8cdaea/ipp-route-8cdaea-kserve-849b5f488c-wwzff"
Sep 18 20:33:25.558330 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:33:25.558228    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f3969483-1cdd-4132-9344-6748326b8a81-model-cache\") pod \"ipp-route-8cdaea-kserve-849b5f488c-wwzff\" (UID: \"f3969483-1cdd-4132-9344-6748326b8a81\") " pod="ai-tenant-e2e-ipp-a-8cdaea/ipp-route-8cdaea-kserve-849b5f488c-wwzff"
Sep 18 20:33:25.659217 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:33:25.659175    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f3969483-1cdd-4132-9344-6748326b8a81-home\") pod \"ipp-route-8cdaea-kserve-849b5f488c-wwzff\" (UID: \"f3969483-1cdd-4132-9344-6748326b8a81\") " pod="ai-tenant-e2e-ipp-a-8cdaea/ipp-route-8cdaea-kserve-849b5f488c-wwzff"
Sep 18 20:33:25.659217 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:33:25.659222    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f3969483-1cdd-4132-9344-6748326b8a81-dshm\") pod \"ipp-route-8cdaea-kserve-849b5f488c-wwzff\" (UID: \"f3969483-1cdd-4132-9344-6748326b8a81\") " pod="ai-tenant-e2e-ipp-a-8cdaea/ipp-route-8cdaea-kserve-849b5f488c-wwzff"
Sep 18 20:33:25.659474 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:33:25.659251    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bdkxx\" (UniqueName: \"kubernetes.io/projected/f3969483-1cdd-4132-9344-6748326b8a81-kube-api-access-bdkxx\") pod \"ipp-route-8cdaea-kserve-849b5f488c-wwzff\" (UID: \"f3969483-1cdd-4132-9344-6748326b8a81\") " pod="ai-tenant-e2e-ipp-a-8cdaea/ipp-route-8cdaea-kserve-849b5f488c-wwzff"
Sep 18 20:33:25.659474 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:33:25.659284    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f3969483-1cdd-4132-9344-6748326b8a81-tmp-dir\") pod \"ipp-route-8cdaea-kserve-849b5f488c-wwzff\" (UID: \"f3969483-1cdd-4132-9344-6748326b8a81\") " pod="ai-tenant-e2e-ipp-a-8cdaea/ipp-route-8cdaea-kserve-849b5f488c-wwzff"
Sep 18 20:33:25.659474 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:33:25.659312    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f3969483-1cdd-4132-9344-6748326b8a81-model-cache\") pod \"ipp-route-8cdaea-kserve-849b5f488c-wwzff\" (UID: \"f3969483-1cdd-4132-9344-6748326b8a81\") " pod="ai-tenant-e2e-ipp-a-8cdaea/ipp-route-8cdaea-kserve-849b5f488c-wwzff"
Sep 18 20:33:25.659474 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:33:25.659399    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f3969483-1cdd-4132-9344-6748326b8a81-tls-certs\") pod \"ipp-route-8cdaea-kserve-849b5f488c-wwzff\" (UID: \"f3969483-1cdd-4132-9344-6748326b8a81\") " pod="ai-tenant-e2e-ipp-a-8cdaea/ipp-route-8cdaea-kserve-849b5f488c-wwzff"
Sep 18 20:33:25.659760 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:33:25.659738    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f3969483-1cdd-4132-9344-6748326b8a81-home\") pod \"ipp-route-8cdaea-kserve-849b5f488c-wwzff\" (UID: \"f3969483-1cdd-4132-9344-6748326b8a81\") " pod="ai-tenant-e2e-ipp-a-8cdaea/ipp-route-8cdaea-kserve-849b5f488c-wwzff"
Sep 18 20:33:25.659825 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:33:25.659774    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f3969483-1cdd-4132-9344-6748326b8a81-tmp-dir\") pod \"ipp-route-8cdaea-kserve-849b5f488c-wwzff\" (UID: \"f3969483-1cdd-4132-9344-6748326b8a81\") " pod="ai-tenant-e2e-ipp-a-8cdaea/ipp-route-8cdaea-kserve-849b5f488c-wwzff"
Sep 18 20:33:25.659825 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:33:25.659802    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f3969483-1cdd-4132-9344-6748326b8a81-model-cache\") pod \"ipp-route-8cdaea-kserve-849b5f488c-wwzff\" (UID: \"f3969483-1cdd-4132-9344-6748326b8a81\") " pod="ai-tenant-e2e-ipp-a-8cdaea/ipp-route-8cdaea-kserve-849b5f488c-wwzff"
Sep 18 20:33:25.661544 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:33:25.661521    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f3969483-1cdd-4132-9344-6748326b8a81-dshm\") pod \"ipp-route-8cdaea-kserve-849b5f488c-wwzff\" (UID: \"f3969483-1cdd-4132-9344-6748326b8a81\") " pod="ai-tenant-e2e-ipp-a-8cdaea/ipp-route-8cdaea-kserve-849b5f488c-wwzff"
Sep 18 20:33:25.661896 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:33:25.661877    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f3969483-1cdd-4132-9344-6748326b8a81-tls-certs\") pod \"ipp-route-8cdaea-kserve-849b5f488c-wwzff\" (UID: \"f3969483-1cdd-4132-9344-6748326b8a81\") " pod="ai-tenant-e2e-ipp-a-8cdaea/ipp-route-8cdaea-kserve-849b5f488c-wwzff"
Sep 18 20:33:25.666631 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:33:25.666608    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bdkxx\" (UniqueName: \"kubernetes.io/projected/f3969483-1cdd-4132-9344-6748326b8a81-kube-api-access-bdkxx\") pod \"ipp-route-8cdaea-kserve-849b5f488c-wwzff\" (UID: \"f3969483-1cdd-4132-9344-6748326b8a81\") " pod="ai-tenant-e2e-ipp-a-8cdaea/ipp-route-8cdaea-kserve-849b5f488c-wwzff"
Sep 18 20:33:25.806309 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:33:25.806207    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-8cdaea/ipp-route-8cdaea-kserve-849b5f488c-wwzff"
Sep 18 20:33:26.083120 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:33:26.083095    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-ipp-a-8cdaea/ipp-route-8cdaea-kserve-849b5f488c-wwzff"]
Sep 18 20:33:26.084882 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:33:26.084855    2532 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf3969483_1cdd_4132_9344_6748326b8a81.slice/crio-a12cc8509326f6616a1ade8017e926d9a38984daca88f89e16454dd6fe3529df WatchSource:0}: Error finding container a12cc8509326f6616a1ade8017e926d9a38984daca88f89e16454dd6fe3529df: Status 404 returned error can't find the container with id a12cc8509326f6616a1ade8017e926d9a38984daca88f89e16454dd6fe3529df
Sep 18 20:33:26.305517 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:33:26.305469    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-8cdaea/ipp-route-8cdaea-kserve-849b5f488c-wwzff" event={"ID":"f3969483-1cdd-4132-9344-6748326b8a81","Type":"ContainerStarted","Data":"c311c038725d70a2ab3786a509d475f5759ed7c2a5fd13bc0123bc91ced90eb4"}
Sep 18 20:33:26.305517 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:33:26.305520    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-8cdaea/ipp-route-8cdaea-kserve-849b5f488c-wwzff" event={"ID":"f3969483-1cdd-4132-9344-6748326b8a81","Type":"ContainerStarted","Data":"a12cc8509326f6616a1ade8017e926d9a38984daca88f89e16454dd6fe3529df"}
Sep 18 20:33:26.319137 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:33:26.319082    2532 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-ipp-a-8cdaea/ipp-route-8cdaea-kserve-849b5f488c-wwzff" podStartSLOduration=1.319068309 podStartE2EDuration="1.319068309s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 20:33:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 20:33:26.317063947 +0000 UTC m=+1938.082453350" watchObservedRunningTime="2026-09-18 20:33:26.319068309 +0000 UTC m=+1938.084457707"
Sep 18 20:33:29.503013 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:33:29.502970    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-vptdl"]
Sep 18 20:33:35.806608 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:33:35.806570    2532 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-ipp-a-8cdaea/ipp-route-8cdaea-kserve-849b5f488c-wwzff"
Sep 18 20:33:35.807043 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:33:35.806911    2532 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-ipp-a-8cdaea/ipp-route-8cdaea-kserve-849b5f488c-wwzff"
Sep 18 20:33:35.819197 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:33:35.819168    2532 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-ipp-a-8cdaea/ipp-route-8cdaea-kserve-849b5f488c-wwzff"
Sep 18 20:33:36.361581 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:33:36.361552    2532 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-ipp-a-8cdaea/ipp-route-8cdaea-kserve-849b5f488c-wwzff"
Sep 18 20:33:38.792603 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:33:38.792547    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:33:47.197676 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:33:47.197642    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-vptdl"]
Sep 18 20:33:49.935083 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:33:49.935010    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:33:54.123919 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:33:54.123839    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:33:54.457760 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:33:54.457731    2532 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-186465-openshift-default-7486d5f584-xnthp"]
Sep 18 20:33:54.461297 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:33:54.461277    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-186465-openshift-default-7486d5f584-xnthp"
Sep 18 20:33:54.464116 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:33:54.464097    2532 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-cleanup-186465-openshift-default-dockercfg-tdvzh\""
Sep 18 20:33:54.478502 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:33:54.476196    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-186465-openshift-default-7486d5f584-xnthp"]
Sep 18 20:33:54.484998 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:33:54.484971    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:33:54.608324 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:33:54.608297    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:33:54.647581 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:33:54.647548    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dca54f78-d480-4710-a3be-1f730b123812-istio-envoy\") pod \"e2e-ipp-cleanup-186465-openshift-default-7486d5f584-xnthp\" (UID: \"dca54f78-d480-4710-a3be-1f730b123812\") " pod="openshift-ingress/e2e-ipp-cleanup-186465-openshift-default-7486d5f584-xnthp"
Sep 18 20:33:54.647718 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:33:54.647585    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dca54f78-d480-4710-a3be-1f730b123812-istiod-ca-cert\") pod \"e2e-ipp-cleanup-186465-openshift-default-7486d5f584-xnthp\" (UID: \"dca54f78-d480-4710-a3be-1f730b123812\") " pod="openshift-ingress/e2e-ipp-cleanup-186465-openshift-default-7486d5f584-xnthp"
Sep 18 20:33:54.647718 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:33:54.647621    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dca54f78-d480-4710-a3be-1f730b123812-istio-token\") pod \"e2e-ipp-cleanup-186465-openshift-default-7486d5f584-xnthp\" (UID: \"dca54f78-d480-4710-a3be-1f730b123812\") " pod="openshift-ingress/e2e-ipp-cleanup-186465-openshift-default-7486d5f584-xnthp"
Sep 18 20:33:54.647718 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:33:54.647695    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dca54f78-d480-4710-a3be-1f730b123812-istio-data\") pod \"e2e-ipp-cleanup-186465-openshift-default-7486d5f584-xnthp\" (UID: \"dca54f78-d480-4710-a3be-1f730b123812\") " pod="openshift-ingress/e2e-ipp-cleanup-186465-openshift-default-7486d5f584-xnthp"
Sep 18 20:33:54.647826 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:33:54.647719    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dca54f78-d480-4710-a3be-1f730b123812-workload-socket\") pod \"e2e-ipp-cleanup-186465-openshift-default-7486d5f584-xnthp\" (UID: \"dca54f78-d480-4710-a3be-1f730b123812\") " pod="openshift-ingress/e2e-ipp-cleanup-186465-openshift-default-7486d5f584-xnthp"
Sep 18 20:33:54.647826 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:33:54.647760    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dca54f78-d480-4710-a3be-1f730b123812-workload-certs\") pod \"e2e-ipp-cleanup-186465-openshift-default-7486d5f584-xnthp\" (UID: \"dca54f78-d480-4710-a3be-1f730b123812\") " pod="openshift-ingress/e2e-ipp-cleanup-186465-openshift-default-7486d5f584-xnthp"
Sep 18 20:33:54.647826 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:33:54.647816    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mpfwl\" (UniqueName: \"kubernetes.io/projected/dca54f78-d480-4710-a3be-1f730b123812-kube-api-access-mpfwl\") pod \"e2e-ipp-cleanup-186465-openshift-default-7486d5f584-xnthp\" (UID: \"dca54f78-d480-4710-a3be-1f730b123812\") " pod="openshift-ingress/e2e-ipp-cleanup-186465-openshift-default-7486d5f584-xnthp"
Sep 18 20:33:54.647926 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:33:54.647838    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dca54f78-d480-4710-a3be-1f730b123812-istio-podinfo\") pod \"e2e-ipp-cleanup-186465-openshift-default-7486d5f584-xnthp\" (UID: \"dca54f78-d480-4710-a3be-1f730b123812\") " pod="openshift-ingress/e2e-ipp-cleanup-186465-openshift-default-7486d5f584-xnthp"
Sep 18 20:33:54.647926 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:33:54.647853    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dca54f78-d480-4710-a3be-1f730b123812-credential-socket\") pod \"e2e-ipp-cleanup-186465-openshift-default-7486d5f584-xnthp\" (UID: \"dca54f78-d480-4710-a3be-1f730b123812\") " pod="openshift-ingress/e2e-ipp-cleanup-186465-openshift-default-7486d5f584-xnthp"
Sep 18 20:33:54.749199 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:33:54.749110    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dca54f78-d480-4710-a3be-1f730b123812-istio-data\") pod \"e2e-ipp-cleanup-186465-openshift-default-7486d5f584-xnthp\" (UID: \"dca54f78-d480-4710-a3be-1f730b123812\") " pod="openshift-ingress/e2e-ipp-cleanup-186465-openshift-default-7486d5f584-xnthp"
Sep 18 20:33:54.749199 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:33:54.749151    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dca54f78-d480-4710-a3be-1f730b123812-workload-socket\") pod \"e2e-ipp-cleanup-186465-openshift-default-7486d5f584-xnthp\" (UID: \"dca54f78-d480-4710-a3be-1f730b123812\") " pod="openshift-ingress/e2e-ipp-cleanup-186465-openshift-default-7486d5f584-xnthp"
Sep 18 20:33:54.749199 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:33:54.749194    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dca54f78-d480-4710-a3be-1f730b123812-workload-certs\") pod \"e2e-ipp-cleanup-186465-openshift-default-7486d5f584-xnthp\" (UID: \"dca54f78-d480-4710-a3be-1f730b123812\") " pod="openshift-ingress/e2e-ipp-cleanup-186465-openshift-default-7486d5f584-xnthp"
Sep 18 20:33:54.749503 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:33:54.749230    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mpfwl\" (UniqueName: \"kubernetes.io/projected/dca54f78-d480-4710-a3be-1f730b123812-kube-api-access-mpfwl\") pod \"e2e-ipp-cleanup-186465-openshift-default-7486d5f584-xnthp\" (UID: \"dca54f78-d480-4710-a3be-1f730b123812\") " pod="openshift-ingress/e2e-ipp-cleanup-186465-openshift-default-7486d5f584-xnthp"
Sep 18 20:33:54.749503 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:33:54.749259    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dca54f78-d480-4710-a3be-1f730b123812-istio-podinfo\") pod \"e2e-ipp-cleanup-186465-openshift-default-7486d5f584-xnthp\" (UID: \"dca54f78-d480-4710-a3be-1f730b123812\") " pod="openshift-ingress/e2e-ipp-cleanup-186465-openshift-default-7486d5f584-xnthp"
Sep 18 20:33:54.749503 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:33:54.749311    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dca54f78-d480-4710-a3be-1f730b123812-credential-socket\") pod \"e2e-ipp-cleanup-186465-openshift-default-7486d5f584-xnthp\" (UID: \"dca54f78-d480-4710-a3be-1f730b123812\") " pod="openshift-ingress/e2e-ipp-cleanup-186465-openshift-default-7486d5f584-xnthp"
Sep 18 20:33:54.749503 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:33:54.749360    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dca54f78-d480-4710-a3be-1f730b123812-istio-envoy\") pod \"e2e-ipp-cleanup-186465-openshift-default-7486d5f584-xnthp\" (UID: \"dca54f78-d480-4710-a3be-1f730b123812\") " pod="openshift-ingress/e2e-ipp-cleanup-186465-openshift-default-7486d5f584-xnthp"
Sep 18 20:33:54.749503 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:33:54.749384    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dca54f78-d480-4710-a3be-1f730b123812-istiod-ca-cert\") pod \"e2e-ipp-cleanup-186465-openshift-default-7486d5f584-xnthp\" (UID: \"dca54f78-d480-4710-a3be-1f730b123812\") " pod="openshift-ingress/e2e-ipp-cleanup-186465-openshift-default-7486d5f584-xnthp"
Sep 18 20:33:54.749503 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:33:54.749423    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dca54f78-d480-4710-a3be-1f730b123812-istio-token\") pod \"e2e-ipp-cleanup-186465-openshift-default-7486d5f584-xnthp\" (UID: \"dca54f78-d480-4710-a3be-1f730b123812\") " pod="openshift-ingress/e2e-ipp-cleanup-186465-openshift-default-7486d5f584-xnthp"
Sep 18 20:33:54.749823 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:33:54.749588    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dca54f78-d480-4710-a3be-1f730b123812-istio-data\") pod \"e2e-ipp-cleanup-186465-openshift-default-7486d5f584-xnthp\" (UID: \"dca54f78-d480-4710-a3be-1f730b123812\") " pod="openshift-ingress/e2e-ipp-cleanup-186465-openshift-default-7486d5f584-xnthp"
Sep 18 20:33:54.749823 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:33:54.749628    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dca54f78-d480-4710-a3be-1f730b123812-workload-socket\") pod \"e2e-ipp-cleanup-186465-openshift-default-7486d5f584-xnthp\" (UID: \"dca54f78-d480-4710-a3be-1f730b123812\") " pod="openshift-ingress/e2e-ipp-cleanup-186465-openshift-default-7486d5f584-xnthp"
Sep 18 20:33:54.749823 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:33:54.749723    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dca54f78-d480-4710-a3be-1f730b123812-workload-certs\") pod \"e2e-ipp-cleanup-186465-openshift-default-7486d5f584-xnthp\" (UID: \"dca54f78-d480-4710-a3be-1f730b123812\") " pod="openshift-ingress/e2e-ipp-cleanup-186465-openshift-default-7486d5f584-xnthp"
Sep 18 20:33:54.749823 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:33:54.749781    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dca54f78-d480-4710-a3be-1f730b123812-credential-socket\") pod \"e2e-ipp-cleanup-186465-openshift-default-7486d5f584-xnthp\" (UID: \"dca54f78-d480-4710-a3be-1f730b123812\") " pod="openshift-ingress/e2e-ipp-cleanup-186465-openshift-default-7486d5f584-xnthp"
Sep 18 20:33:54.750222 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:33:54.750198    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dca54f78-d480-4710-a3be-1f730b123812-istiod-ca-cert\") pod \"e2e-ipp-cleanup-186465-openshift-default-7486d5f584-xnthp\" (UID: \"dca54f78-d480-4710-a3be-1f730b123812\") " pod="openshift-ingress/e2e-ipp-cleanup-186465-openshift-default-7486d5f584-xnthp"
Sep 18 20:33:54.751717 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:33:54.751696    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dca54f78-d480-4710-a3be-1f730b123812-istio-envoy\") pod \"e2e-ipp-cleanup-186465-openshift-default-7486d5f584-xnthp\" (UID: \"dca54f78-d480-4710-a3be-1f730b123812\") " pod="openshift-ingress/e2e-ipp-cleanup-186465-openshift-default-7486d5f584-xnthp"
Sep 18 20:33:54.751919 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:33:54.751903    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dca54f78-d480-4710-a3be-1f730b123812-istio-podinfo\") pod \"e2e-ipp-cleanup-186465-openshift-default-7486d5f584-xnthp\" (UID: \"dca54f78-d480-4710-a3be-1f730b123812\") " pod="openshift-ingress/e2e-ipp-cleanup-186465-openshift-default-7486d5f584-xnthp"
Sep 18 20:33:54.757152 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:33:54.757125    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dca54f78-d480-4710-a3be-1f730b123812-istio-token\") pod \"e2e-ipp-cleanup-186465-openshift-default-7486d5f584-xnthp\" (UID: \"dca54f78-d480-4710-a3be-1f730b123812\") " pod="openshift-ingress/e2e-ipp-cleanup-186465-openshift-default-7486d5f584-xnthp"
Sep 18 20:33:54.757257 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:33:54.757192    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mpfwl\" (UniqueName: \"kubernetes.io/projected/dca54f78-d480-4710-a3be-1f730b123812-kube-api-access-mpfwl\") pod \"e2e-ipp-cleanup-186465-openshift-default-7486d5f584-xnthp\" (UID: \"dca54f78-d480-4710-a3be-1f730b123812\") " pod="openshift-ingress/e2e-ipp-cleanup-186465-openshift-default-7486d5f584-xnthp"
Sep 18 20:33:54.772144 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:33:54.772122    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-186465-openshift-default-7486d5f584-xnthp"
Sep 18 20:33:55.050467 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:33:55.050442    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-186465-openshift-default-7486d5f584-xnthp"]
Sep 18 20:33:55.052148 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:33:55.052126    2532 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddca54f78_d480_4710_a3be_1f730b123812.slice/crio-b7816775069a577f3a94c311777d0b09d302c349bd6e4435c212eac9ea5728a1 WatchSource:0}: Error finding container b7816775069a577f3a94c311777d0b09d302c349bd6e4435c212eac9ea5728a1: Status 404 returned error can't find the container with id b7816775069a577f3a94c311777d0b09d302c349bd6e4435c212eac9ea5728a1
Sep 18 20:33:55.054796 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:33:55.054763    2532 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 18 20:33:55.054867 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:33:55.054825    2532 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 18 20:33:55.054867 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:33:55.054856    2532 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 18 20:33:55.452994 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:33:55.452897    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-186465-openshift-default-7486d5f584-xnthp" event={"ID":"dca54f78-d480-4710-a3be-1f730b123812","Type":"ContainerStarted","Data":"9859c662acaa45739314f83ddcec81b2f5aea57894abf0dfc7e17b54fa38609c"}
Sep 18 20:33:55.452994 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:33:55.452964    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-186465-openshift-default-7486d5f584-xnthp" event={"ID":"dca54f78-d480-4710-a3be-1f730b123812","Type":"ContainerStarted","Data":"b7816775069a577f3a94c311777d0b09d302c349bd6e4435c212eac9ea5728a1"}
Sep 18 20:33:55.479160 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:33:55.479100    2532 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-cleanup-186465-openshift-default-7486d5f584-xnthp" podStartSLOduration=1.4790801199999999 podStartE2EDuration="1.47908012s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 20:33:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 20:33:55.474630123 +0000 UTC m=+1967.240019534" watchObservedRunningTime="2026-09-18 20:33:55.47908012 +0000 UTC m=+1967.244469520"
Sep 18 20:33:55.772418 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:33:55.772385    2532 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-cleanup-186465-openshift-default-7486d5f584-xnthp"
Sep 18 20:33:56.695725 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:33:56.695694    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:33:56.777169 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:33:56.777138    2532 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-cleanup-186465-openshift-default-7486d5f584-xnthp"
Sep 18 20:33:57.347840 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:33:57.347813    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:33:57.445960 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:33:57.445929    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:33:57.463151 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:33:57.463115    2532 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-cleanup-186465-openshift-default-7486d5f584-xnthp"
Sep 18 20:33:57.464111 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:33:57.464089    2532 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-cleanup-186465-openshift-default-7486d5f584-xnthp"
Sep 18 20:33:58.223140 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:33:58.223102    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:33:58.760434 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:33:58.760402    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:33:59.601973 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:33:59.601944    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:34:00.232876 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:34:00.232828    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:34:00.291189 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:34:00.291160    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:34:00.481074 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:34:00.481041    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:34:00.674268 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:34:00.674188    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:34:00.761711 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:34:00.761672    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:34:07.000957 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:34:07.000916    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-vptdl"]
Sep 18 20:34:17.577622 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:34:17.577579    2532 generic.go:356] "Generic (PLEG): container finished" podID="dca54f78-d480-4710-a3be-1f730b123812" containerID="9859c662acaa45739314f83ddcec81b2f5aea57894abf0dfc7e17b54fa38609c" exitCode=0
Sep 18 20:34:17.578155 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:34:17.577657    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-186465-openshift-default-7486d5f584-xnthp" event={"ID":"dca54f78-d480-4710-a3be-1f730b123812","Type":"ContainerDied","Data":"9859c662acaa45739314f83ddcec81b2f5aea57894abf0dfc7e17b54fa38609c"}
Sep 18 20:34:17.578231 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:34:17.578153    2532 scope.go:117] "RemoveContainer" containerID="9859c662acaa45739314f83ddcec81b2f5aea57894abf0dfc7e17b54fa38609c"
Sep 18 20:34:17.579544 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:34:17.579514    2532 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 18 20:34:17.579650 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:34:17.579588    2532 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 18 20:34:17.579650 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:34:17.579624    2532 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 18 20:34:18.584506 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:34:18.584447    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-186465-openshift-default-7486d5f584-xnthp" event={"ID":"dca54f78-d480-4710-a3be-1f730b123812","Type":"ContainerStarted","Data":"142858d54e8929ee4a444de967dfda100cac49a6cbb0b2b86466ebb100cbe071"}
Sep 18 20:34:18.773193 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:34:18.773155    2532 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-cleanup-186465-openshift-default-7486d5f584-xnthp"
Sep 18 20:34:18.773404 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:34:18.773372    2532 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-186465-openshift-default-7486d5f584-xnthp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.71:15021/healthz/ready\": dial tcp 10.133.0.71:15021: connect: connection refused" start-of-body=
Sep 18 20:34:18.773518 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:34:18.773472    2532 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-186465-openshift-default-7486d5f584-xnthp" podUID="dca54f78-d480-4710-a3be-1f730b123812" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.71:15021/healthz/ready\": dial tcp 10.133.0.71:15021: connect: connection refused"
Sep 18 20:34:19.773087 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:34:19.773049    2532 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-186465-openshift-default-7486d5f584-xnthp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.71:15021/healthz/ready\": dial tcp 10.133.0.71:15021: connect: connection refused" start-of-body=
Sep 18 20:34:19.773509 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:34:19.773111    2532 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-186465-openshift-default-7486d5f584-xnthp" podUID="dca54f78-d480-4710-a3be-1f730b123812" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.71:15021/healthz/ready\": dial tcp 10.133.0.71:15021: connect: connection refused"
Sep 18 20:34:20.773545 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:34:20.773476    2532 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-186465-openshift-default-7486d5f584-xnthp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.71:15021/healthz/ready\": dial tcp 10.133.0.71:15021: connect: connection refused" start-of-body=
Sep 18 20:34:20.773910 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:34:20.773564    2532 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-186465-openshift-default-7486d5f584-xnthp" podUID="dca54f78-d480-4710-a3be-1f730b123812" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.71:15021/healthz/ready\": dial tcp 10.133.0.71:15021: connect: connection refused"
Sep 18 20:34:21.772704 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:34:21.772661    2532 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-186465-openshift-default-7486d5f584-xnthp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.71:15021/healthz/ready\": dial tcp 10.133.0.71:15021: connect: connection refused" start-of-body=
Sep 18 20:34:21.772889 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:34:21.772726    2532 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-186465-openshift-default-7486d5f584-xnthp" podUID="dca54f78-d480-4710-a3be-1f730b123812" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.71:15021/healthz/ready\": dial tcp 10.133.0.71:15021: connect: connection refused"
Sep 18 20:34:22.126849 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:34:22.126759    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:34:22.290121 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:34:22.290091    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:34:22.631706 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:34:22.631665    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:34:22.773527 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:34:22.773472    2532 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-186465-openshift-default-7486d5f584-xnthp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.71:15021/healthz/ready\": dial tcp 10.133.0.71:15021: connect: connection refused" start-of-body=
Sep 18 20:34:22.773704 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:34:22.773571    2532 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-186465-openshift-default-7486d5f584-xnthp" podUID="dca54f78-d480-4710-a3be-1f730b123812" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.71:15021/healthz/ready\": dial tcp 10.133.0.71:15021: connect: connection refused"
Sep 18 20:34:23.408508 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:34:23.408459    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:34:23.772833 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:34:23.772794    2532 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-186465-openshift-default-7486d5f584-xnthp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.71:15021/healthz/ready\": dial tcp 10.133.0.71:15021: connect: connection refused" start-of-body=
Sep 18 20:34:23.773030 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:34:23.772850    2532 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-186465-openshift-default-7486d5f584-xnthp" podUID="dca54f78-d480-4710-a3be-1f730b123812" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.71:15021/healthz/ready\": dial tcp 10.133.0.71:15021: connect: connection refused"
Sep 18 20:34:24.101472 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:34:24.101391    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:34:24.724806 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:34:24.724779    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:34:24.777260 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:34:24.777230    2532 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-cleanup-186465-openshift-default-7486d5f584-xnthp"
Sep 18 20:34:24.777689 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:34:24.777665    2532 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-cleanup-186465-openshift-default-7486d5f584-xnthp"
Sep 18 20:34:24.778258 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:34:24.778240    2532 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-cleanup-186465-openshift-default-7486d5f584-xnthp"
Sep 18 20:34:25.357580 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:34:25.357546    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:34:26.111765 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:34:26.111729    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:34:26.609594 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:34:26.609555    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:34:28.637501 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:34:28.637455    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:34:29.211570 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:34:29.211537    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:34:29.274919 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:34:29.274890    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:34:29.375938 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:34:29.375900    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:34:29.438472 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:34:29.438438    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:34:29.821836 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:34:29.821805    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:34:29.921750 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:34:29.921712    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:34:30.358725 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:34:30.358695    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:34:30.999457 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:34:30.999426    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:34:31.723389 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:34:31.723360    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:34:32.431102 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:34:32.431070    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:34:33.797924 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:34:33.793835    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-vptdl"]
Sep 18 20:34:35.500049 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:34:35.500003    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:34:35.562808 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:34:35.562779    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:34:35.733329 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:34:35.733294    2532 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-186465-openshift-default-7486d5f584-xnthp"]
Sep 18 20:34:35.733566 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:34:35.733539    2532 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-cleanup-186465-openshift-default-7486d5f584-xnthp" podUID="dca54f78-d480-4710-a3be-1f730b123812" containerName="istio-proxy" containerID="cri-o://142858d54e8929ee4a444de967dfda100cac49a6cbb0b2b86466ebb100cbe071" gracePeriod=30
Sep 18 20:34:35.751614 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:34:35.751543    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:34:39.503368 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:34:39.503342    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:34:39.780378 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:34:39.780297    2532 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-186465-openshift-default-7486d5f584-xnthp container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 18 20:34:39.780536 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:34:39.780366    2532 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ipp-cleanup-186465-openshift-default-7486d5f584-xnthp" podUID="dca54f78-d480-4710-a3be-1f730b123812" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 18 20:34:40.315099 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:34:40.315069    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:34:40.912718 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:34:40.912689    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:34:40.990326 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:34:40.990304    2532 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-186465-openshift-default-7486d5f584-xnthp"
Sep 18 20:34:41.001251 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:34:41.001193    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dca54f78-d480-4710-a3be-1f730b123812-istiod-ca-cert\") pod \"dca54f78-d480-4710-a3be-1f730b123812\" (UID: \"dca54f78-d480-4710-a3be-1f730b123812\") "
Sep 18 20:34:41.001251 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:34:41.001236    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mpfwl\" (UniqueName: \"kubernetes.io/projected/dca54f78-d480-4710-a3be-1f730b123812-kube-api-access-mpfwl\") pod \"dca54f78-d480-4710-a3be-1f730b123812\" (UID: \"dca54f78-d480-4710-a3be-1f730b123812\") "
Sep 18 20:34:41.001424 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:34:41.001259    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dca54f78-d480-4710-a3be-1f730b123812-istio-envoy\") pod \"dca54f78-d480-4710-a3be-1f730b123812\" (UID: \"dca54f78-d480-4710-a3be-1f730b123812\") "
Sep 18 20:34:41.001424 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:34:41.001276    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dca54f78-d480-4710-a3be-1f730b123812-workload-certs\") pod \"dca54f78-d480-4710-a3be-1f730b123812\" (UID: \"dca54f78-d480-4710-a3be-1f730b123812\") "
Sep 18 20:34:41.001424 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:34:41.001303    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dca54f78-d480-4710-a3be-1f730b123812-istio-data\") pod \"dca54f78-d480-4710-a3be-1f730b123812\" (UID: \"dca54f78-d480-4710-a3be-1f730b123812\") "
Sep 18 20:34:41.001424 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:34:41.001331    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dca54f78-d480-4710-a3be-1f730b123812-istio-token\") pod \"dca54f78-d480-4710-a3be-1f730b123812\" (UID: \"dca54f78-d480-4710-a3be-1f730b123812\") "
Sep 18 20:34:41.001424 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:34:41.001353    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dca54f78-d480-4710-a3be-1f730b123812-credential-socket\") pod \"dca54f78-d480-4710-a3be-1f730b123812\" (UID: \"dca54f78-d480-4710-a3be-1f730b123812\") "
Sep 18 20:34:41.001424 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:34:41.001398    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dca54f78-d480-4710-a3be-1f730b123812-workload-socket\") pod \"dca54f78-d480-4710-a3be-1f730b123812\" (UID: \"dca54f78-d480-4710-a3be-1f730b123812\") "
Sep 18 20:34:41.001751 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:34:41.001445    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dca54f78-d480-4710-a3be-1f730b123812-istio-podinfo\") pod \"dca54f78-d480-4710-a3be-1f730b123812\" (UID: \"dca54f78-d480-4710-a3be-1f730b123812\") "
Sep 18 20:34:41.001751 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:34:41.001629    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dca54f78-d480-4710-a3be-1f730b123812-istio-data" (OuterVolumeSpecName: "istio-data") pod "dca54f78-d480-4710-a3be-1f730b123812" (UID: "dca54f78-d480-4710-a3be-1f730b123812"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 20:34:41.001751 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:34:41.001686    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dca54f78-d480-4710-a3be-1f730b123812-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "dca54f78-d480-4710-a3be-1f730b123812" (UID: "dca54f78-d480-4710-a3be-1f730b123812"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 20:34:41.001751 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:34:41.001690    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dca54f78-d480-4710-a3be-1f730b123812-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "dca54f78-d480-4710-a3be-1f730b123812" (UID: "dca54f78-d480-4710-a3be-1f730b123812"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 20:34:41.001966 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:34:41.001885    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dca54f78-d480-4710-a3be-1f730b123812-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "dca54f78-d480-4710-a3be-1f730b123812" (UID: "dca54f78-d480-4710-a3be-1f730b123812"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 20:34:41.001966 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:34:41.001891    2532 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dca54f78-d480-4710-a3be-1f730b123812-istiod-ca-cert\") on node \"ip-10-0-130-81.ec2.internal\" DevicePath \"\""
Sep 18 20:34:41.001966 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:34:41.001926    2532 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dca54f78-d480-4710-a3be-1f730b123812-workload-certs\") on node \"ip-10-0-130-81.ec2.internal\" DevicePath \"\""
Sep 18 20:34:41.001966 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:34:41.001941    2532 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dca54f78-d480-4710-a3be-1f730b123812-istio-data\") on node \"ip-10-0-130-81.ec2.internal\" DevicePath \"\""
Sep 18 20:34:41.002136 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:34:41.001811    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dca54f78-d480-4710-a3be-1f730b123812-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "dca54f78-d480-4710-a3be-1f730b123812" (UID: "dca54f78-d480-4710-a3be-1f730b123812"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 20:34:41.003698 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:34:41.003663    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dca54f78-d480-4710-a3be-1f730b123812-istio-token" (OuterVolumeSpecName: "istio-token") pod "dca54f78-d480-4710-a3be-1f730b123812" (UID: "dca54f78-d480-4710-a3be-1f730b123812"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 20:34:41.003946 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:34:41.003920    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dca54f78-d480-4710-a3be-1f730b123812-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "dca54f78-d480-4710-a3be-1f730b123812" (UID: "dca54f78-d480-4710-a3be-1f730b123812"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 20:34:41.004043 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:34:41.003940    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dca54f78-d480-4710-a3be-1f730b123812-kube-api-access-mpfwl" (OuterVolumeSpecName: "kube-api-access-mpfwl") pod "dca54f78-d480-4710-a3be-1f730b123812" (UID: "dca54f78-d480-4710-a3be-1f730b123812"). InnerVolumeSpecName "kube-api-access-mpfwl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 20:34:41.004043 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:34:41.004002    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/dca54f78-d480-4710-a3be-1f730b123812-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "dca54f78-d480-4710-a3be-1f730b123812" (UID: "dca54f78-d480-4710-a3be-1f730b123812"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 18 20:34:41.103414 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:34:41.103376    2532 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dca54f78-d480-4710-a3be-1f730b123812-workload-socket\") on node \"ip-10-0-130-81.ec2.internal\" DevicePath \"\""
Sep 18 20:34:41.103414 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:34:41.103406    2532 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dca54f78-d480-4710-a3be-1f730b123812-istio-podinfo\") on node \"ip-10-0-130-81.ec2.internal\" DevicePath \"\""
Sep 18 20:34:41.103414 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:34:41.103415    2532 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mpfwl\" (UniqueName: \"kubernetes.io/projected/dca54f78-d480-4710-a3be-1f730b123812-kube-api-access-mpfwl\") on node \"ip-10-0-130-81.ec2.internal\" DevicePath \"\""
Sep 18 20:34:41.103688 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:34:41.103424    2532 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dca54f78-d480-4710-a3be-1f730b123812-istio-envoy\") on node \"ip-10-0-130-81.ec2.internal\" DevicePath \"\""
Sep 18 20:34:41.103688 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:34:41.103432    2532 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dca54f78-d480-4710-a3be-1f730b123812-istio-token\") on node \"ip-10-0-130-81.ec2.internal\" DevicePath \"\""
Sep 18 20:34:41.103688 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:34:41.103440    2532 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dca54f78-d480-4710-a3be-1f730b123812-credential-socket\") on node \"ip-10-0-130-81.ec2.internal\" DevicePath \"\""
Sep 18 20:34:41.518735 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:34:41.518688    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:34:41.717704 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:34:41.717668    2532 generic.go:356] "Generic (PLEG): container finished" podID="dca54f78-d480-4710-a3be-1f730b123812" containerID="142858d54e8929ee4a444de967dfda100cac49a6cbb0b2b86466ebb100cbe071" exitCode=0
Sep 18 20:34:41.717847 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:34:41.717738    2532 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-186465-openshift-default-7486d5f584-xnthp"
Sep 18 20:34:41.717847 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:34:41.717755    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-186465-openshift-default-7486d5f584-xnthp" event={"ID":"dca54f78-d480-4710-a3be-1f730b123812","Type":"ContainerDied","Data":"142858d54e8929ee4a444de967dfda100cac49a6cbb0b2b86466ebb100cbe071"}
Sep 18 20:34:41.717847 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:34:41.717838    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-186465-openshift-default-7486d5f584-xnthp" event={"ID":"dca54f78-d480-4710-a3be-1f730b123812","Type":"ContainerDied","Data":"b7816775069a577f3a94c311777d0b09d302c349bd6e4435c212eac9ea5728a1"}
Sep 18 20:34:41.717974 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:34:41.717860    2532 scope.go:117] "RemoveContainer" containerID="142858d54e8929ee4a444de967dfda100cac49a6cbb0b2b86466ebb100cbe071"
Sep 18 20:34:41.727856 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:34:41.727839    2532 scope.go:117] "RemoveContainer" containerID="9859c662acaa45739314f83ddcec81b2f5aea57894abf0dfc7e17b54fa38609c"
Sep 18 20:34:41.735695 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:34:41.735677    2532 scope.go:117] "RemoveContainer" containerID="142858d54e8929ee4a444de967dfda100cac49a6cbb0b2b86466ebb100cbe071"
Sep 18 20:34:41.735955 ip-10-0-130-81 kubenswrapper[2532]: E0918 20:34:41.735935    2532 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"142858d54e8929ee4a444de967dfda100cac49a6cbb0b2b86466ebb100cbe071\": container with ID starting with 142858d54e8929ee4a444de967dfda100cac49a6cbb0b2b86466ebb100cbe071 not found: ID does not exist" containerID="142858d54e8929ee4a444de967dfda100cac49a6cbb0b2b86466ebb100cbe071"
Sep 18 20:34:41.735998 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:34:41.735964    2532 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"142858d54e8929ee4a444de967dfda100cac49a6cbb0b2b86466ebb100cbe071"} err="failed to get container status \"142858d54e8929ee4a444de967dfda100cac49a6cbb0b2b86466ebb100cbe071\": rpc error: code = NotFound desc = could not find container \"142858d54e8929ee4a444de967dfda100cac49a6cbb0b2b86466ebb100cbe071\": container with ID starting with 142858d54e8929ee4a444de967dfda100cac49a6cbb0b2b86466ebb100cbe071 not found: ID does not exist"
Sep 18 20:34:41.735998 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:34:41.735980    2532 scope.go:117] "RemoveContainer" containerID="9859c662acaa45739314f83ddcec81b2f5aea57894abf0dfc7e17b54fa38609c"
Sep 18 20:34:41.736229 ip-10-0-130-81 kubenswrapper[2532]: E0918 20:34:41.736210    2532 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9859c662acaa45739314f83ddcec81b2f5aea57894abf0dfc7e17b54fa38609c\": container with ID starting with 9859c662acaa45739314f83ddcec81b2f5aea57894abf0dfc7e17b54fa38609c not found: ID does not exist" containerID="9859c662acaa45739314f83ddcec81b2f5aea57894abf0dfc7e17b54fa38609c"
Sep 18 20:34:41.736271 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:34:41.736236    2532 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9859c662acaa45739314f83ddcec81b2f5aea57894abf0dfc7e17b54fa38609c"} err="failed to get container status \"9859c662acaa45739314f83ddcec81b2f5aea57894abf0dfc7e17b54fa38609c\": rpc error: code = NotFound desc = could not find container \"9859c662acaa45739314f83ddcec81b2f5aea57894abf0dfc7e17b54fa38609c\": container with ID starting with 9859c662acaa45739314f83ddcec81b2f5aea57894abf0dfc7e17b54fa38609c not found: ID does not exist"
Sep 18 20:34:41.740183 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:34:41.740162    2532 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-186465-openshift-default-7486d5f584-xnthp"]
Sep 18 20:34:41.743047 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:34:41.743028    2532 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-186465-openshift-default-7486d5f584-xnthp"]
Sep 18 20:34:42.307100 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:34:42.307058    2532 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dca54f78-d480-4710-a3be-1f730b123812" path="/var/lib/kubelet/pods/dca54f78-d480-4710-a3be-1f730b123812/volumes"
Sep 18 20:34:42.328426 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:34:42.328397    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:34:42.980570 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:34:42.980543    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:34:52.196258 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:34:52.196217    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-vptdl"]
Sep 18 20:34:54.716651 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:34:54.716620    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:34:54.816917 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:34:54.816878    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:34:54.916011 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:34:54.915974    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:34:54.967466 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:34:54.967383    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:34:55.280722 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:34:55.280640    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:34:55.361168 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:34:55.361126    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:34:55.965236 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:34:55.965183    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:34:56.702832 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:34:56.702798    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:34:57.313060 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:34:57.313026    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:35:00.257751 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:35:00.257714    2532 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-ipp-a-8cdaea/ipp-route-8cdaea-kserve-849b5f488c-wwzff"]
Sep 18 20:35:00.303754 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:35:00.303722    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:35:05.089310 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:35:05.089263    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-vptdl"]
Sep 18 20:35:15.276863 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:35:15.276774    2532 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-ipp-a-8cdaea/ipp-route-8cdaea-kserve-849b5f488c-wwzff" podUID="f3969483-1cdd-4132-9344-6748326b8a81" containerName="main" containerID="cri-o://c311c038725d70a2ab3786a509d475f5759ed7c2a5fd13bc0123bc91ced90eb4" gracePeriod=45
Sep 18 20:35:15.523987 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:35:15.523965    2532 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-8cdaea/ipp-route-8cdaea-kserve-849b5f488c-wwzff"
Sep 18 20:35:15.632188 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:35:15.632093    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bdkxx\" (UniqueName: \"kubernetes.io/projected/f3969483-1cdd-4132-9344-6748326b8a81-kube-api-access-bdkxx\") pod \"f3969483-1cdd-4132-9344-6748326b8a81\" (UID: \"f3969483-1cdd-4132-9344-6748326b8a81\") "
Sep 18 20:35:15.632188 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:35:15.632158    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f3969483-1cdd-4132-9344-6748326b8a81-model-cache\") pod \"f3969483-1cdd-4132-9344-6748326b8a81\" (UID: \"f3969483-1cdd-4132-9344-6748326b8a81\") "
Sep 18 20:35:15.632188 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:35:15.632184    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f3969483-1cdd-4132-9344-6748326b8a81-tmp-dir\") pod \"f3969483-1cdd-4132-9344-6748326b8a81\" (UID: \"f3969483-1cdd-4132-9344-6748326b8a81\") "
Sep 18 20:35:15.632475 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:35:15.632202    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f3969483-1cdd-4132-9344-6748326b8a81-tls-certs\") pod \"f3969483-1cdd-4132-9344-6748326b8a81\" (UID: \"f3969483-1cdd-4132-9344-6748326b8a81\") "
Sep 18 20:35:15.632475 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:35:15.632220    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f3969483-1cdd-4132-9344-6748326b8a81-dshm\") pod \"f3969483-1cdd-4132-9344-6748326b8a81\" (UID: \"f3969483-1cdd-4132-9344-6748326b8a81\") "
Sep 18 20:35:15.632475 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:35:15.632236    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f3969483-1cdd-4132-9344-6748326b8a81-home\") pod \"f3969483-1cdd-4132-9344-6748326b8a81\" (UID: \"f3969483-1cdd-4132-9344-6748326b8a81\") "
Sep 18 20:35:15.632657 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:35:15.632500    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f3969483-1cdd-4132-9344-6748326b8a81-model-cache" (OuterVolumeSpecName: "model-cache") pod "f3969483-1cdd-4132-9344-6748326b8a81" (UID: "f3969483-1cdd-4132-9344-6748326b8a81"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 20:35:15.632657 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:35:15.632525    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f3969483-1cdd-4132-9344-6748326b8a81-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "f3969483-1cdd-4132-9344-6748326b8a81" (UID: "f3969483-1cdd-4132-9344-6748326b8a81"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 20:35:15.632761 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:35:15.632671    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f3969483-1cdd-4132-9344-6748326b8a81-home" (OuterVolumeSpecName: "home") pod "f3969483-1cdd-4132-9344-6748326b8a81" (UID: "f3969483-1cdd-4132-9344-6748326b8a81"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 20:35:15.634499 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:35:15.634458    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f3969483-1cdd-4132-9344-6748326b8a81-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "f3969483-1cdd-4132-9344-6748326b8a81" (UID: "f3969483-1cdd-4132-9344-6748326b8a81"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 20:35:15.634608 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:35:15.634517    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f3969483-1cdd-4132-9344-6748326b8a81-dshm" (OuterVolumeSpecName: "dshm") pod "f3969483-1cdd-4132-9344-6748326b8a81" (UID: "f3969483-1cdd-4132-9344-6748326b8a81"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 20:35:15.634608 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:35:15.634546    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f3969483-1cdd-4132-9344-6748326b8a81-kube-api-access-bdkxx" (OuterVolumeSpecName: "kube-api-access-bdkxx") pod "f3969483-1cdd-4132-9344-6748326b8a81" (UID: "f3969483-1cdd-4132-9344-6748326b8a81"). InnerVolumeSpecName "kube-api-access-bdkxx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 20:35:15.733679 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:35:15.733651    2532 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bdkxx\" (UniqueName: \"kubernetes.io/projected/f3969483-1cdd-4132-9344-6748326b8a81-kube-api-access-bdkxx\") on node \"ip-10-0-130-81.ec2.internal\" DevicePath \"\""
Sep 18 20:35:15.733679 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:35:15.733677    2532 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f3969483-1cdd-4132-9344-6748326b8a81-model-cache\") on node \"ip-10-0-130-81.ec2.internal\" DevicePath \"\""
Sep 18 20:35:15.733880 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:35:15.733688    2532 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f3969483-1cdd-4132-9344-6748326b8a81-tmp-dir\") on node \"ip-10-0-130-81.ec2.internal\" DevicePath \"\""
Sep 18 20:35:15.733880 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:35:15.733696    2532 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f3969483-1cdd-4132-9344-6748326b8a81-tls-certs\") on node \"ip-10-0-130-81.ec2.internal\" DevicePath \"\""
Sep 18 20:35:15.733880 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:35:15.733703    2532 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f3969483-1cdd-4132-9344-6748326b8a81-dshm\") on node \"ip-10-0-130-81.ec2.internal\" DevicePath \"\""
Sep 18 20:35:15.733880 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:35:15.733710    2532 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f3969483-1cdd-4132-9344-6748326b8a81-home\") on node \"ip-10-0-130-81.ec2.internal\" DevicePath \"\""
Sep 18 20:35:15.897747 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:35:15.897662    2532 generic.go:356] "Generic (PLEG): container finished" podID="f3969483-1cdd-4132-9344-6748326b8a81" containerID="c311c038725d70a2ab3786a509d475f5759ed7c2a5fd13bc0123bc91ced90eb4" exitCode=0
Sep 18 20:35:15.897747 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:35:15.897705    2532 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-8cdaea/ipp-route-8cdaea-kserve-849b5f488c-wwzff"
Sep 18 20:35:15.897984 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:35:15.897754    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-8cdaea/ipp-route-8cdaea-kserve-849b5f488c-wwzff" event={"ID":"f3969483-1cdd-4132-9344-6748326b8a81","Type":"ContainerDied","Data":"c311c038725d70a2ab3786a509d475f5759ed7c2a5fd13bc0123bc91ced90eb4"}
Sep 18 20:35:15.897984 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:35:15.897802    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-8cdaea/ipp-route-8cdaea-kserve-849b5f488c-wwzff" event={"ID":"f3969483-1cdd-4132-9344-6748326b8a81","Type":"ContainerDied","Data":"a12cc8509326f6616a1ade8017e926d9a38984daca88f89e16454dd6fe3529df"}
Sep 18 20:35:15.897984 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:35:15.897823    2532 scope.go:117] "RemoveContainer" containerID="c311c038725d70a2ab3786a509d475f5759ed7c2a5fd13bc0123bc91ced90eb4"
Sep 18 20:35:15.906963 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:35:15.906946    2532 scope.go:117] "RemoveContainer" containerID="c311c038725d70a2ab3786a509d475f5759ed7c2a5fd13bc0123bc91ced90eb4"
Sep 18 20:35:15.907222 ip-10-0-130-81 kubenswrapper[2532]: E0918 20:35:15.907204    2532 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c311c038725d70a2ab3786a509d475f5759ed7c2a5fd13bc0123bc91ced90eb4\": container with ID starting with c311c038725d70a2ab3786a509d475f5759ed7c2a5fd13bc0123bc91ced90eb4 not found: ID does not exist" containerID="c311c038725d70a2ab3786a509d475f5759ed7c2a5fd13bc0123bc91ced90eb4"
Sep 18 20:35:15.907263 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:35:15.907232    2532 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c311c038725d70a2ab3786a509d475f5759ed7c2a5fd13bc0123bc91ced90eb4"} err="failed to get container status \"c311c038725d70a2ab3786a509d475f5759ed7c2a5fd13bc0123bc91ced90eb4\": rpc error: code = NotFound desc = could not find container \"c311c038725d70a2ab3786a509d475f5759ed7c2a5fd13bc0123bc91ced90eb4\": container with ID starting with c311c038725d70a2ab3786a509d475f5759ed7c2a5fd13bc0123bc91ced90eb4 not found: ID does not exist"
Sep 18 20:35:15.919024 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:35:15.919003    2532 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-ipp-a-8cdaea/ipp-route-8cdaea-kserve-849b5f488c-wwzff"]
Sep 18 20:35:15.924000 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:35:15.923980    2532 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-ipp-a-8cdaea/ipp-route-8cdaea-kserve-849b5f488c-wwzff"]
Sep 18 20:35:16.306609 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:35:16.306572    2532 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f3969483-1cdd-4132-9344-6748326b8a81" path="/var/lib/kubelet/pods/f3969483-1cdd-4132-9344-6748326b8a81/volumes"
Sep 18 20:35:34.634801 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:35:34.634717    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:35:35.492413 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:35:35.492381    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:35:40.871689 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:35:40.871647    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:35:41.077441 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:35:41.077395    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:35:41.141279 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:35:41.141196    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:35:41.348968 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:35:41.348937    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:35:42.423748 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:35:42.423708    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:35:43.119751 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:35:43.119721    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:35:43.750287 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:35:43.750258    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:35:44.339671 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:35:44.339637    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:35:51.148840 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:35:51.148808    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:35:51.202638 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:35:51.202461    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:35:52.082565 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:35:52.082535    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:35:52.150380 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:35:52.150352    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:35:52.215551 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:35:52.215523    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:35:52.288771 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:35:52.288738    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:35:52.655240 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:35:52.655207    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:35:52.758953 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:35:52.758924    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:35:53.216585 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:35:53.216554    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:35:54.014206 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:35:54.014175    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:35:55.299364 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:35:55.299329    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-vptdl"]
Sep 18 20:35:58.601900 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:35:58.601863    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:35:58.701664 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:35:58.701635    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:35:58.894937 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:35:58.894832    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:35:59.328829 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:35:59.328755    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:36:00.126954 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:36:00.126923    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:36:00.735778 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:36:00.735745    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:36:01.350060 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:36:01.350015    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:36:09.696616 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:36:09.696577    2532 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-bab50d-58db4bc787-q2s49"]
Sep 18 20:36:09.697059 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:36:09.696824    2532 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-bab50d-58db4bc787-q2s49" podUID="80f6c7c5-c42b-4c88-b5c5-bf1de8a683d1" containerName="maas-api" containerID="cri-o://ec342752243625a2a80024be5d723a0cfcd9426a60f0c3a7be6c264d1327e166" gracePeriod=30
Sep 18 20:36:09.729346 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:36:09.729318    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:36:09.827507 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:36:09.827457    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:36:09.928710 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:36:09.928682    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:36:09.945221 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:36:09.945191    2532 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-bab50d-58db4bc787-q2s49"
Sep 18 20:36:10.015973 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:36:10.015941    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/80f6c7c5-c42b-4c88-b5c5-bf1de8a683d1-openshift-service-ca\") pod \"80f6c7c5-c42b-4c88-b5c5-bf1de8a683d1\" (UID: \"80f6c7c5-c42b-4c88-b5c5-bf1de8a683d1\") "
Sep 18 20:36:10.015973 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:36:10.015980    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5vnbk\" (UniqueName: \"kubernetes.io/projected/80f6c7c5-c42b-4c88-b5c5-bf1de8a683d1-kube-api-access-5vnbk\") pod \"80f6c7c5-c42b-4c88-b5c5-bf1de8a683d1\" (UID: \"80f6c7c5-c42b-4c88-b5c5-bf1de8a683d1\") "
Sep 18 20:36:10.016227 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:36:10.016018    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/80f6c7c5-c42b-4c88-b5c5-bf1de8a683d1-maas-api-tls\") pod \"80f6c7c5-c42b-4c88-b5c5-bf1de8a683d1\" (UID: \"80f6c7c5-c42b-4c88-b5c5-bf1de8a683d1\") "
Sep 18 20:36:10.016227 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:36:10.016125    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/80f6c7c5-c42b-4c88-b5c5-bf1de8a683d1-metrics-certs\") pod \"80f6c7c5-c42b-4c88-b5c5-bf1de8a683d1\" (UID: \"80f6c7c5-c42b-4c88-b5c5-bf1de8a683d1\") "
Sep 18 20:36:10.016396 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:36:10.016368    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/80f6c7c5-c42b-4c88-b5c5-bf1de8a683d1-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "80f6c7c5-c42b-4c88-b5c5-bf1de8a683d1" (UID: "80f6c7c5-c42b-4c88-b5c5-bf1de8a683d1"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 20:36:10.016499 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:36:10.016465    2532 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/80f6c7c5-c42b-4c88-b5c5-bf1de8a683d1-openshift-service-ca\") on node \"ip-10-0-130-81.ec2.internal\" DevicePath \"\""
Sep 18 20:36:10.018270 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:36:10.018245    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/80f6c7c5-c42b-4c88-b5c5-bf1de8a683d1-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "80f6c7c5-c42b-4c88-b5c5-bf1de8a683d1" (UID: "80f6c7c5-c42b-4c88-b5c5-bf1de8a683d1"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 20:36:10.018361 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:36:10.018285    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/80f6c7c5-c42b-4c88-b5c5-bf1de8a683d1-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "80f6c7c5-c42b-4c88-b5c5-bf1de8a683d1" (UID: "80f6c7c5-c42b-4c88-b5c5-bf1de8a683d1"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 20:36:10.018361 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:36:10.018295    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/80f6c7c5-c42b-4c88-b5c5-bf1de8a683d1-kube-api-access-5vnbk" (OuterVolumeSpecName: "kube-api-access-5vnbk") pod "80f6c7c5-c42b-4c88-b5c5-bf1de8a683d1" (UID: "80f6c7c5-c42b-4c88-b5c5-bf1de8a683d1"). InnerVolumeSpecName "kube-api-access-5vnbk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 20:36:10.027934 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:36:10.027905    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:36:10.116994 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:36:10.116963    2532 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/80f6c7c5-c42b-4c88-b5c5-bf1de8a683d1-metrics-certs\") on node \"ip-10-0-130-81.ec2.internal\" DevicePath \"\""
Sep 18 20:36:10.116994 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:36:10.116992    2532 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5vnbk\" (UniqueName: \"kubernetes.io/projected/80f6c7c5-c42b-4c88-b5c5-bf1de8a683d1-kube-api-access-5vnbk\") on node \"ip-10-0-130-81.ec2.internal\" DevicePath \"\""
Sep 18 20:36:10.116994 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:36:10.117001    2532 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/80f6c7c5-c42b-4c88-b5c5-bf1de8a683d1-maas-api-tls\") on node \"ip-10-0-130-81.ec2.internal\" DevicePath \"\""
Sep 18 20:36:10.182099 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:36:10.182061    2532 generic.go:356] "Generic (PLEG): container finished" podID="80f6c7c5-c42b-4c88-b5c5-bf1de8a683d1" containerID="ec342752243625a2a80024be5d723a0cfcd9426a60f0c3a7be6c264d1327e166" exitCode=0
Sep 18 20:36:10.182276 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:36:10.182113    2532 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-bab50d-58db4bc787-q2s49"
Sep 18 20:36:10.182276 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:36:10.182148    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-bab50d-58db4bc787-q2s49" event={"ID":"80f6c7c5-c42b-4c88-b5c5-bf1de8a683d1","Type":"ContainerDied","Data":"ec342752243625a2a80024be5d723a0cfcd9426a60f0c3a7be6c264d1327e166"}
Sep 18 20:36:10.182276 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:36:10.182209    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-bab50d-58db4bc787-q2s49" event={"ID":"80f6c7c5-c42b-4c88-b5c5-bf1de8a683d1","Type":"ContainerDied","Data":"3dd0967ddbc3617b023ce953619f0c59dbde9ae346665bb61e6bc3600a91d2ea"}
Sep 18 20:36:10.182276 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:36:10.182234    2532 scope.go:117] "RemoveContainer" containerID="ec342752243625a2a80024be5d723a0cfcd9426a60f0c3a7be6c264d1327e166"
Sep 18 20:36:10.192313 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:36:10.192296    2532 scope.go:117] "RemoveContainer" containerID="ec342752243625a2a80024be5d723a0cfcd9426a60f0c3a7be6c264d1327e166"
Sep 18 20:36:10.192600 ip-10-0-130-81 kubenswrapper[2532]: E0918 20:36:10.192577    2532 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ec342752243625a2a80024be5d723a0cfcd9426a60f0c3a7be6c264d1327e166\": container with ID starting with ec342752243625a2a80024be5d723a0cfcd9426a60f0c3a7be6c264d1327e166 not found: ID does not exist" containerID="ec342752243625a2a80024be5d723a0cfcd9426a60f0c3a7be6c264d1327e166"
Sep 18 20:36:10.192700 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:36:10.192608    2532 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ec342752243625a2a80024be5d723a0cfcd9426a60f0c3a7be6c264d1327e166"} err="failed to get container status \"ec342752243625a2a80024be5d723a0cfcd9426a60f0c3a7be6c264d1327e166\": rpc error: code = NotFound desc = could not find container \"ec342752243625a2a80024be5d723a0cfcd9426a60f0c3a7be6c264d1327e166\": container with ID starting with ec342752243625a2a80024be5d723a0cfcd9426a60f0c3a7be6c264d1327e166 not found: ID does not exist"
Sep 18 20:36:10.203824 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:36:10.203793    2532 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-bab50d-58db4bc787-q2s49"]
Sep 18 20:36:10.207841 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:36:10.207817    2532 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-bab50d-58db4bc787-q2s49"]
Sep 18 20:36:10.299451 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:36:10.299362    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:36:10.307069 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:36:10.307039    2532 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="80f6c7c5-c42b-4c88-b5c5-bf1de8a683d1" path="/var/lib/kubelet/pods/80f6c7c5-c42b-4c88-b5c5-bf1de8a683d1/volumes"
Sep 18 20:36:10.402500 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:36:10.402453    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:36:10.480466 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:36:10.480438    2532 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-m7wdd_32e33d39-7826-499d-bc01-0487b7580e7a/console-operator/1.log"
Sep 18 20:36:10.482871 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:36:10.482846    2532 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-m7wdd_32e33d39-7826-499d-bc01-0487b7580e7a/console-operator/1.log"
Sep 18 20:36:10.489098 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:36:10.489073    2532 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rtw9n_73fbf16b-3549-4a6c-a959-b633237ebd0c/ovn-acl-logging/0.log"
Sep 18 20:36:10.491033 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:36:10.491014    2532 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rtw9n_73fbf16b-3549-4a6c-a959-b633237ebd0c/ovn-acl-logging/0.log"
Sep 18 20:36:10.941917 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:36:10.941830    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:36:12.907532 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:36:12.907498    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-vptdl"]
Sep 18 20:36:16.046646 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:36:16.046608    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:36:16.334265 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:36:16.334187    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:36:16.438816 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:36:16.438779    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:36:16.606598 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:36:16.606519    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:36:17.217152 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:36:17.217114    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:36:17.384252 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:36:17.384220    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:36:17.915539 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:36:17.915478    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:36:18.429330 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:36:18.429293    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:36:27.450015 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:36:27.449973    2532 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-35558b-646544f8fb-cnxv2"]
Sep 18 20:36:27.450602 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:36:27.450230    2532 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-35558b-646544f8fb-cnxv2" podUID="e919b8e7-fb73-484a-9992-72991abbeb4d" containerName="maas-api" containerID="cri-o://a7a138cca229cd21aecea07f43a84a3b0c5876cfda3ff1d2c3321feb385a7b52" gracePeriod=30
Sep 18 20:36:27.478397 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:36:27.478370    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:36:27.578718 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:36:27.578672    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:36:27.679507 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:36:27.679468    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:36:27.692165 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:36:27.692145    2532 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-35558b-646544f8fb-cnxv2"
Sep 18 20:36:27.758833 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:36:27.758793    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e919b8e7-fb73-484a-9992-72991abbeb4d-maas-api-tls\") pod \"e919b8e7-fb73-484a-9992-72991abbeb4d\" (UID: \"e919b8e7-fb73-484a-9992-72991abbeb4d\") "
Sep 18 20:36:27.759015 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:36:27.758860    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e919b8e7-fb73-484a-9992-72991abbeb4d-openshift-service-ca\") pod \"e919b8e7-fb73-484a-9992-72991abbeb4d\" (UID: \"e919b8e7-fb73-484a-9992-72991abbeb4d\") "
Sep 18 20:36:27.759015 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:36:27.758960    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-c7txq\" (UniqueName: \"kubernetes.io/projected/e919b8e7-fb73-484a-9992-72991abbeb4d-kube-api-access-c7txq\") pod \"e919b8e7-fb73-484a-9992-72991abbeb4d\" (UID: \"e919b8e7-fb73-484a-9992-72991abbeb4d\") "
Sep 18 20:36:27.759015 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:36:27.759000    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e919b8e7-fb73-484a-9992-72991abbeb4d-metrics-certs\") pod \"e919b8e7-fb73-484a-9992-72991abbeb4d\" (UID: \"e919b8e7-fb73-484a-9992-72991abbeb4d\") "
Sep 18 20:36:27.759302 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:36:27.759280    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e919b8e7-fb73-484a-9992-72991abbeb4d-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "e919b8e7-fb73-484a-9992-72991abbeb4d" (UID: "e919b8e7-fb73-484a-9992-72991abbeb4d"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 20:36:27.761196 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:36:27.761172    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e919b8e7-fb73-484a-9992-72991abbeb4d-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "e919b8e7-fb73-484a-9992-72991abbeb4d" (UID: "e919b8e7-fb73-484a-9992-72991abbeb4d"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 20:36:27.761314 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:36:27.761194    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e919b8e7-fb73-484a-9992-72991abbeb4d-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "e919b8e7-fb73-484a-9992-72991abbeb4d" (UID: "e919b8e7-fb73-484a-9992-72991abbeb4d"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 20:36:27.761314 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:36:27.761272    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e919b8e7-fb73-484a-9992-72991abbeb4d-kube-api-access-c7txq" (OuterVolumeSpecName: "kube-api-access-c7txq") pod "e919b8e7-fb73-484a-9992-72991abbeb4d" (UID: "e919b8e7-fb73-484a-9992-72991abbeb4d"). InnerVolumeSpecName "kube-api-access-c7txq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 20:36:27.781732 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:36:27.781698    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:36:27.860198 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:36:27.860167    2532 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c7txq\" (UniqueName: \"kubernetes.io/projected/e919b8e7-fb73-484a-9992-72991abbeb4d-kube-api-access-c7txq\") on node \"ip-10-0-130-81.ec2.internal\" DevicePath \"\""
Sep 18 20:36:27.860198 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:36:27.860195    2532 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e919b8e7-fb73-484a-9992-72991abbeb4d-metrics-certs\") on node \"ip-10-0-130-81.ec2.internal\" DevicePath \"\""
Sep 18 20:36:27.860198 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:36:27.860206    2532 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e919b8e7-fb73-484a-9992-72991abbeb4d-maas-api-tls\") on node \"ip-10-0-130-81.ec2.internal\" DevicePath \"\""
Sep 18 20:36:27.860437 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:36:27.860214    2532 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e919b8e7-fb73-484a-9992-72991abbeb4d-openshift-service-ca\") on node \"ip-10-0-130-81.ec2.internal\" DevicePath \"\""
Sep 18 20:36:28.041922 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:36:28.041834    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:36:28.223139 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:36:28.223105    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:36:28.278932 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:36:28.278896    2532 generic.go:356] "Generic (PLEG): container finished" podID="e919b8e7-fb73-484a-9992-72991abbeb4d" containerID="a7a138cca229cd21aecea07f43a84a3b0c5876cfda3ff1d2c3321feb385a7b52" exitCode=0
Sep 18 20:36:28.278932 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:36:28.278938    2532 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-35558b-646544f8fb-cnxv2"
Sep 18 20:36:28.279133 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:36:28.278986    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-35558b-646544f8fb-cnxv2" event={"ID":"e919b8e7-fb73-484a-9992-72991abbeb4d","Type":"ContainerDied","Data":"a7a138cca229cd21aecea07f43a84a3b0c5876cfda3ff1d2c3321feb385a7b52"}
Sep 18 20:36:28.279133 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:36:28.279033    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-35558b-646544f8fb-cnxv2" event={"ID":"e919b8e7-fb73-484a-9992-72991abbeb4d","Type":"ContainerDied","Data":"5dfc41ca8a9c28ef1588ce2f7f842bfbaf94b336d4b4638cc6b4a868be10415b"}
Sep 18 20:36:28.279133 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:36:28.279055    2532 scope.go:117] "RemoveContainer" containerID="a7a138cca229cd21aecea07f43a84a3b0c5876cfda3ff1d2c3321feb385a7b52"
Sep 18 20:36:28.288357 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:36:28.288341    2532 scope.go:117] "RemoveContainer" containerID="a7a138cca229cd21aecea07f43a84a3b0c5876cfda3ff1d2c3321feb385a7b52"
Sep 18 20:36:28.288696 ip-10-0-130-81 kubenswrapper[2532]: E0918 20:36:28.288679    2532 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a7a138cca229cd21aecea07f43a84a3b0c5876cfda3ff1d2c3321feb385a7b52\": container with ID starting with a7a138cca229cd21aecea07f43a84a3b0c5876cfda3ff1d2c3321feb385a7b52 not found: ID does not exist" containerID="a7a138cca229cd21aecea07f43a84a3b0c5876cfda3ff1d2c3321feb385a7b52"
Sep 18 20:36:28.288757 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:36:28.288711    2532 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a7a138cca229cd21aecea07f43a84a3b0c5876cfda3ff1d2c3321feb385a7b52"} err="failed to get container status \"a7a138cca229cd21aecea07f43a84a3b0c5876cfda3ff1d2c3321feb385a7b52\": rpc error: code = NotFound desc = could not find container \"a7a138cca229cd21aecea07f43a84a3b0c5876cfda3ff1d2c3321feb385a7b52\": container with ID starting with a7a138cca229cd21aecea07f43a84a3b0c5876cfda3ff1d2c3321feb385a7b52 not found: ID does not exist"
Sep 18 20:36:28.307108 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:36:28.307051    2532 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-35558b-646544f8fb-cnxv2"]
Sep 18 20:36:28.308730 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:36:28.308712    2532 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-35558b-646544f8fb-cnxv2"]
Sep 18 20:36:30.305577 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:36:30.305541    2532 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e919b8e7-fb73-484a-9992-72991abbeb4d" path="/var/lib/kubelet/pods/e919b8e7-fb73-484a-9992-72991abbeb4d/volumes"
Sep 18 20:36:30.499154 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:36:30.499117    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-vptdl"]
Sep 18 20:36:34.063058 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:36:34.063028    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:36:34.163104 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:36:34.163062    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:36:34.314834 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:36:34.314753    2532 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-35558b-openshift-default-6bbb5bcc84-6gxxh"]
Sep 18 20:36:34.314984 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:36:34.314950    2532 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-w2-35558b-openshift-default-6bbb5bcc84-6gxxh" podUID="66b1dac9-2d9a-4f89-9e49-17293b282e3a" containerName="istio-proxy" containerID="cri-o://810359afa0469f6a6ab2be449203c0cdd3c9b61f4469ebf4551c26cdecd5e846" gracePeriod=30
Sep 18 20:36:34.340325 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:36:34.340295    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:36:35.263184 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:36:35.263141    2532 patch_prober.go:28] interesting pod/e2e-shared-b-w2-35558b-openshift-default-6bbb5bcc84-6gxxh container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 18 20:36:35.263672 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:36:35.263212    2532 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-w2-35558b-openshift-default-6bbb5bcc84-6gxxh" podUID="66b1dac9-2d9a-4f89-9e49-17293b282e3a" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 18 20:36:39.390812 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:36:39.390771    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-vptdl"]
Sep 18 20:36:39.561029 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:36:39.561006    2532 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w2-35558b-openshift-default-6bbb5bcc84-6gxxh"
Sep 18 20:36:39.667510 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:36:39.667407    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/66b1dac9-2d9a-4f89-9e49-17293b282e3a-workload-certs\") pod \"66b1dac9-2d9a-4f89-9e49-17293b282e3a\" (UID: \"66b1dac9-2d9a-4f89-9e49-17293b282e3a\") "
Sep 18 20:36:39.667510 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:36:39.667448    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/66b1dac9-2d9a-4f89-9e49-17293b282e3a-istio-token\") pod \"66b1dac9-2d9a-4f89-9e49-17293b282e3a\" (UID: \"66b1dac9-2d9a-4f89-9e49-17293b282e3a\") "
Sep 18 20:36:39.667510 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:36:39.667477    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/66b1dac9-2d9a-4f89-9e49-17293b282e3a-workload-socket\") pod \"66b1dac9-2d9a-4f89-9e49-17293b282e3a\" (UID: \"66b1dac9-2d9a-4f89-9e49-17293b282e3a\") "
Sep 18 20:36:39.667791 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:36:39.667527    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/66b1dac9-2d9a-4f89-9e49-17293b282e3a-credential-socket\") pod \"66b1dac9-2d9a-4f89-9e49-17293b282e3a\" (UID: \"66b1dac9-2d9a-4f89-9e49-17293b282e3a\") "
Sep 18 20:36:39.667791 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:36:39.667550    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/66b1dac9-2d9a-4f89-9e49-17293b282e3a-istio-podinfo\") pod \"66b1dac9-2d9a-4f89-9e49-17293b282e3a\" (UID: \"66b1dac9-2d9a-4f89-9e49-17293b282e3a\") "
Sep 18 20:36:39.667791 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:36:39.667581    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/66b1dac9-2d9a-4f89-9e49-17293b282e3a-istio-data\") pod \"66b1dac9-2d9a-4f89-9e49-17293b282e3a\" (UID: \"66b1dac9-2d9a-4f89-9e49-17293b282e3a\") "
Sep 18 20:36:39.667791 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:36:39.667603    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/66b1dac9-2d9a-4f89-9e49-17293b282e3a-istio-envoy\") pod \"66b1dac9-2d9a-4f89-9e49-17293b282e3a\" (UID: \"66b1dac9-2d9a-4f89-9e49-17293b282e3a\") "
Sep 18 20:36:39.667791 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:36:39.667650    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kvnfm\" (UniqueName: \"kubernetes.io/projected/66b1dac9-2d9a-4f89-9e49-17293b282e3a-kube-api-access-kvnfm\") pod \"66b1dac9-2d9a-4f89-9e49-17293b282e3a\" (UID: \"66b1dac9-2d9a-4f89-9e49-17293b282e3a\") "
Sep 18 20:36:39.667791 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:36:39.667674    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/66b1dac9-2d9a-4f89-9e49-17293b282e3a-istiod-ca-cert\") pod \"66b1dac9-2d9a-4f89-9e49-17293b282e3a\" (UID: \"66b1dac9-2d9a-4f89-9e49-17293b282e3a\") "
Sep 18 20:36:39.668075 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:36:39.667777    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/66b1dac9-2d9a-4f89-9e49-17293b282e3a-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "66b1dac9-2d9a-4f89-9e49-17293b282e3a" (UID: "66b1dac9-2d9a-4f89-9e49-17293b282e3a"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 20:36:39.668075 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:36:39.667833    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/66b1dac9-2d9a-4f89-9e49-17293b282e3a-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "66b1dac9-2d9a-4f89-9e49-17293b282e3a" (UID: "66b1dac9-2d9a-4f89-9e49-17293b282e3a"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 20:36:39.668075 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:36:39.667870    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/66b1dac9-2d9a-4f89-9e49-17293b282e3a-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "66b1dac9-2d9a-4f89-9e49-17293b282e3a" (UID: "66b1dac9-2d9a-4f89-9e49-17293b282e3a"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 20:36:39.668075 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:36:39.667947    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/66b1dac9-2d9a-4f89-9e49-17293b282e3a-istio-data" (OuterVolumeSpecName: "istio-data") pod "66b1dac9-2d9a-4f89-9e49-17293b282e3a" (UID: "66b1dac9-2d9a-4f89-9e49-17293b282e3a"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 20:36:39.668075 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:36:39.667983    2532 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/66b1dac9-2d9a-4f89-9e49-17293b282e3a-workload-certs\") on node \"ip-10-0-130-81.ec2.internal\" DevicePath \"\""
Sep 18 20:36:39.668075 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:36:39.668002    2532 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/66b1dac9-2d9a-4f89-9e49-17293b282e3a-workload-socket\") on node \"ip-10-0-130-81.ec2.internal\" DevicePath \"\""
Sep 18 20:36:39.668075 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:36:39.668016    2532 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/66b1dac9-2d9a-4f89-9e49-17293b282e3a-credential-socket\") on node \"ip-10-0-130-81.ec2.internal\" DevicePath \"\""
Sep 18 20:36:39.668370 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:36:39.668100    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/66b1dac9-2d9a-4f89-9e49-17293b282e3a-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "66b1dac9-2d9a-4f89-9e49-17293b282e3a" (UID: "66b1dac9-2d9a-4f89-9e49-17293b282e3a"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 20:36:39.669926 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:36:39.669901    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/66b1dac9-2d9a-4f89-9e49-17293b282e3a-kube-api-access-kvnfm" (OuterVolumeSpecName: "kube-api-access-kvnfm") pod "66b1dac9-2d9a-4f89-9e49-17293b282e3a" (UID: "66b1dac9-2d9a-4f89-9e49-17293b282e3a"). InnerVolumeSpecName "kube-api-access-kvnfm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 20:36:39.670062 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:36:39.670044    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/66b1dac9-2d9a-4f89-9e49-17293b282e3a-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "66b1dac9-2d9a-4f89-9e49-17293b282e3a" (UID: "66b1dac9-2d9a-4f89-9e49-17293b282e3a"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 20:36:39.670241 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:36:39.670215    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/66b1dac9-2d9a-4f89-9e49-17293b282e3a-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "66b1dac9-2d9a-4f89-9e49-17293b282e3a" (UID: "66b1dac9-2d9a-4f89-9e49-17293b282e3a"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 18 20:36:39.670241 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:36:39.670236    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/66b1dac9-2d9a-4f89-9e49-17293b282e3a-istio-token" (OuterVolumeSpecName: "istio-token") pod "66b1dac9-2d9a-4f89-9e49-17293b282e3a" (UID: "66b1dac9-2d9a-4f89-9e49-17293b282e3a"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 20:36:39.768972 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:36:39.768937    2532 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/66b1dac9-2d9a-4f89-9e49-17293b282e3a-istio-token\") on node \"ip-10-0-130-81.ec2.internal\" DevicePath \"\""
Sep 18 20:36:39.768972 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:36:39.768965    2532 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/66b1dac9-2d9a-4f89-9e49-17293b282e3a-istio-podinfo\") on node \"ip-10-0-130-81.ec2.internal\" DevicePath \"\""
Sep 18 20:36:39.768972 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:36:39.768975    2532 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/66b1dac9-2d9a-4f89-9e49-17293b282e3a-istio-data\") on node \"ip-10-0-130-81.ec2.internal\" DevicePath \"\""
Sep 18 20:36:39.768972 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:36:39.768982    2532 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/66b1dac9-2d9a-4f89-9e49-17293b282e3a-istio-envoy\") on node \"ip-10-0-130-81.ec2.internal\" DevicePath \"\""
Sep 18 20:36:39.769271 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:36:39.768991    2532 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kvnfm\" (UniqueName: \"kubernetes.io/projected/66b1dac9-2d9a-4f89-9e49-17293b282e3a-kube-api-access-kvnfm\") on node \"ip-10-0-130-81.ec2.internal\" DevicePath \"\""
Sep 18 20:36:39.769271 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:36:39.769000    2532 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/66b1dac9-2d9a-4f89-9e49-17293b282e3a-istiod-ca-cert\") on node \"ip-10-0-130-81.ec2.internal\" DevicePath \"\""
Sep 18 20:36:40.341975 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:36:40.341941    2532 generic.go:356] "Generic (PLEG): container finished" podID="66b1dac9-2d9a-4f89-9e49-17293b282e3a" containerID="810359afa0469f6a6ab2be449203c0cdd3c9b61f4469ebf4551c26cdecd5e846" exitCode=0
Sep 18 20:36:40.342154 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:36:40.341986    2532 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w2-35558b-openshift-default-6bbb5bcc84-6gxxh"
Sep 18 20:36:40.342154 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:36:40.342025    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-35558b-openshift-default-6bbb5bcc84-6gxxh" event={"ID":"66b1dac9-2d9a-4f89-9e49-17293b282e3a","Type":"ContainerDied","Data":"810359afa0469f6a6ab2be449203c0cdd3c9b61f4469ebf4551c26cdecd5e846"}
Sep 18 20:36:40.342154 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:36:40.342077    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-35558b-openshift-default-6bbb5bcc84-6gxxh" event={"ID":"66b1dac9-2d9a-4f89-9e49-17293b282e3a","Type":"ContainerDied","Data":"33bab6861cd3a965f30ff2424562e563d943ddfe6884586cbd4de9c2b84b3eb3"}
Sep 18 20:36:40.342154 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:36:40.342099    2532 scope.go:117] "RemoveContainer" containerID="810359afa0469f6a6ab2be449203c0cdd3c9b61f4469ebf4551c26cdecd5e846"
Sep 18 20:36:40.350952 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:36:40.350935    2532 scope.go:117] "RemoveContainer" containerID="b341ce93ad3047427fbc641fdab61eacca106a3d9f4619ad84dc904f8ae38e82"
Sep 18 20:36:40.359014 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:36:40.358998    2532 scope.go:117] "RemoveContainer" containerID="810359afa0469f6a6ab2be449203c0cdd3c9b61f4469ebf4551c26cdecd5e846"
Sep 18 20:36:40.359262 ip-10-0-130-81 kubenswrapper[2532]: E0918 20:36:40.359243    2532 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"810359afa0469f6a6ab2be449203c0cdd3c9b61f4469ebf4551c26cdecd5e846\": container with ID starting with 810359afa0469f6a6ab2be449203c0cdd3c9b61f4469ebf4551c26cdecd5e846 not found: ID does not exist" containerID="810359afa0469f6a6ab2be449203c0cdd3c9b61f4469ebf4551c26cdecd5e846"
Sep 18 20:36:40.359314 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:36:40.359271    2532 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"810359afa0469f6a6ab2be449203c0cdd3c9b61f4469ebf4551c26cdecd5e846"} err="failed to get container status \"810359afa0469f6a6ab2be449203c0cdd3c9b61f4469ebf4551c26cdecd5e846\": rpc error: code = NotFound desc = could not find container \"810359afa0469f6a6ab2be449203c0cdd3c9b61f4469ebf4551c26cdecd5e846\": container with ID starting with 810359afa0469f6a6ab2be449203c0cdd3c9b61f4469ebf4551c26cdecd5e846 not found: ID does not exist"
Sep 18 20:36:40.359314 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:36:40.359288    2532 scope.go:117] "RemoveContainer" containerID="b341ce93ad3047427fbc641fdab61eacca106a3d9f4619ad84dc904f8ae38e82"
Sep 18 20:36:40.359586 ip-10-0-130-81 kubenswrapper[2532]: E0918 20:36:40.359564    2532 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b341ce93ad3047427fbc641fdab61eacca106a3d9f4619ad84dc904f8ae38e82\": container with ID starting with b341ce93ad3047427fbc641fdab61eacca106a3d9f4619ad84dc904f8ae38e82 not found: ID does not exist" containerID="b341ce93ad3047427fbc641fdab61eacca106a3d9f4619ad84dc904f8ae38e82"
Sep 18 20:36:40.359675 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:36:40.359591    2532 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b341ce93ad3047427fbc641fdab61eacca106a3d9f4619ad84dc904f8ae38e82"} err="failed to get container status \"b341ce93ad3047427fbc641fdab61eacca106a3d9f4619ad84dc904f8ae38e82\": rpc error: code = NotFound desc = could not find container \"b341ce93ad3047427fbc641fdab61eacca106a3d9f4619ad84dc904f8ae38e82\": container with ID starting with b341ce93ad3047427fbc641fdab61eacca106a3d9f4619ad84dc904f8ae38e82 not found: ID does not exist"
Sep 18 20:36:40.363040 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:36:40.363021    2532 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-35558b-openshift-default-6bbb5bcc84-6gxxh"]
Sep 18 20:36:40.366073 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:36:40.366053    2532 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-35558b-openshift-default-6bbb5bcc84-6gxxh"]
Sep 18 20:36:42.306437 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:36:42.306404    2532 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="66b1dac9-2d9a-4f89-9e49-17293b282e3a" path="/var/lib/kubelet/pods/66b1dac9-2d9a-4f89-9e49-17293b282e3a/volumes"
Sep 18 20:37:58.195324 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:37:58.195286    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-vptdl"]
Sep 18 20:37:59.377034 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:37:59.377004    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:38:00.889406 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:38:00.889371    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:38:06.843263 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:38:06.843227    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:38:06.943377 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:38:06.943343    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:38:24.606508 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:38:24.606416    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:38:24.665931 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:38:24.665894    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:38:24.999147 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:38:24.999111    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-vptdl"]
Sep 18 20:38:32.993088 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:38:32.993051    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-vptdl"]
Sep 18 20:38:39.900636 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:38:39.900595    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-vptdl"]
Sep 18 20:38:47.312340 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:38:47.312305    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-vptdl"]
Sep 18 20:39:01.896122 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:39:01.896074    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-vptdl"]
Sep 18 20:39:03.222978 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:39:03.222938    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:39:12.200079 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:39:12.200036    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-vptdl"]
Sep 18 20:39:15.965545 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:39:15.965514    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:39:27.497052 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:39:27.497009    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:40:40.191415 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:40:40.191379    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-vptdl"]
Sep 18 20:40:47.996625 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:40:47.996581    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-vptdl"]
Sep 18 20:40:53.888894 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:40:53.888856    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-vptdl"]
Sep 18 20:41:10.514198 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:41:10.514170    2532 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-m7wdd_32e33d39-7826-499d-bc01-0487b7580e7a/console-operator/1.log"
Sep 18 20:41:10.518226 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:41:10.518201    2532 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-m7wdd_32e33d39-7826-499d-bc01-0487b7580e7a/console-operator/1.log"
Sep 18 20:41:10.522932 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:41:10.522910    2532 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rtw9n_73fbf16b-3549-4a6c-a959-b633237ebd0c/ovn-acl-logging/0.log"
Sep 18 20:41:10.530026 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:41:10.530004    2532 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rtw9n_73fbf16b-3549-4a6c-a959-b633237ebd0c/ovn-acl-logging/0.log"
Sep 18 20:41:13.544717 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:41:13.544686    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:41:16.093083 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:41:16.093040    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-vptdl"]
Sep 18 20:41:29.998395 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:41:29.998314    2532 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-b-7e57cbca-kserve-7bb84d5879-6q4rn"]
Sep 18 20:41:29.998822 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:41:29.998650    2532 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dca54f78-d480-4710-a3be-1f730b123812" containerName="istio-proxy"
Sep 18 20:41:29.998822 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:41:29.998662    2532 state_mem.go:107] "Deleted CPUSet assignment" podUID="dca54f78-d480-4710-a3be-1f730b123812" containerName="istio-proxy"
Sep 18 20:41:29.998822 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:41:29.998677    2532 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="80f6c7c5-c42b-4c88-b5c5-bf1de8a683d1" containerName="maas-api"
Sep 18 20:41:29.998822 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:41:29.998684    2532 state_mem.go:107] "Deleted CPUSet assignment" podUID="80f6c7c5-c42b-4c88-b5c5-bf1de8a683d1" containerName="maas-api"
Sep 18 20:41:29.998822 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:41:29.998692    2532 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e919b8e7-fb73-484a-9992-72991abbeb4d" containerName="maas-api"
Sep 18 20:41:29.998822 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:41:29.998697    2532 state_mem.go:107] "Deleted CPUSet assignment" podUID="e919b8e7-fb73-484a-9992-72991abbeb4d" containerName="maas-api"
Sep 18 20:41:29.998822 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:41:29.998706    2532 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f3969483-1cdd-4132-9344-6748326b8a81" containerName="main"
Sep 18 20:41:29.998822 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:41:29.998712    2532 state_mem.go:107] "Deleted CPUSet assignment" podUID="f3969483-1cdd-4132-9344-6748326b8a81" containerName="main"
Sep 18 20:41:29.998822 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:41:29.998722    2532 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="66b1dac9-2d9a-4f89-9e49-17293b282e3a" containerName="istio-proxy"
Sep 18 20:41:29.998822 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:41:29.998729    2532 state_mem.go:107] "Deleted CPUSet assignment" podUID="66b1dac9-2d9a-4f89-9e49-17293b282e3a" containerName="istio-proxy"
Sep 18 20:41:29.998822 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:41:29.998737    2532 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="66b1dac9-2d9a-4f89-9e49-17293b282e3a" containerName="istio-proxy"
Sep 18 20:41:29.998822 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:41:29.998742    2532 state_mem.go:107] "Deleted CPUSet assignment" podUID="66b1dac9-2d9a-4f89-9e49-17293b282e3a" containerName="istio-proxy"
Sep 18 20:41:29.998822 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:41:29.998793    2532 memory_manager.go:368] "RemoveStaleState removing state" podUID="dca54f78-d480-4710-a3be-1f730b123812" containerName="istio-proxy"
Sep 18 20:41:29.998822 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:41:29.998800    2532 memory_manager.go:368] "RemoveStaleState removing state" podUID="66b1dac9-2d9a-4f89-9e49-17293b282e3a" containerName="istio-proxy"
Sep 18 20:41:29.998822 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:41:29.998807    2532 memory_manager.go:368] "RemoveStaleState removing state" podUID="e919b8e7-fb73-484a-9992-72991abbeb4d" containerName="maas-api"
Sep 18 20:41:29.998822 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:41:29.998814    2532 memory_manager.go:368] "RemoveStaleState removing state" podUID="66b1dac9-2d9a-4f89-9e49-17293b282e3a" containerName="istio-proxy"
Sep 18 20:41:29.998822 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:41:29.998819    2532 memory_manager.go:368] "RemoveStaleState removing state" podUID="dca54f78-d480-4710-a3be-1f730b123812" containerName="istio-proxy"
Sep 18 20:41:29.998822 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:41:29.998826    2532 memory_manager.go:368] "RemoveStaleState removing state" podUID="f3969483-1cdd-4132-9344-6748326b8a81" containerName="main"
Sep 18 20:41:29.998822 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:41:29.998832    2532 memory_manager.go:368] "RemoveStaleState removing state" podUID="80f6c7c5-c42b-4c88-b5c5-bf1de8a683d1" containerName="maas-api"
Sep 18 20:41:30.001783 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:41:30.001763    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-7e57cbca-kserve-7bb84d5879-6q4rn"
Sep 18 20:41:30.004541 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:41:30.004516    2532 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-b-7e57cbca-kserve-self-signed-certs\""
Sep 18 20:41:30.010879 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:41:30.010853    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-7e57cbca-kserve-7bb84d5879-6q4rn"]
Sep 18 20:41:30.109838 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:41:30.109798    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/20cad10d-e5d5-4805-bab2-20ea100955ee-tls-certs\") pod \"e2e-same-id-b-7e57cbca-kserve-7bb84d5879-6q4rn\" (UID: \"20cad10d-e5d5-4805-bab2-20ea100955ee\") " pod="llm/e2e-same-id-b-7e57cbca-kserve-7bb84d5879-6q4rn"
Sep 18 20:41:30.110029 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:41:30.109851    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/20cad10d-e5d5-4805-bab2-20ea100955ee-tmp-dir\") pod \"e2e-same-id-b-7e57cbca-kserve-7bb84d5879-6q4rn\" (UID: \"20cad10d-e5d5-4805-bab2-20ea100955ee\") " pod="llm/e2e-same-id-b-7e57cbca-kserve-7bb84d5879-6q4rn"
Sep 18 20:41:30.110029 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:41:30.109886    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/20cad10d-e5d5-4805-bab2-20ea100955ee-model-cache\") pod \"e2e-same-id-b-7e57cbca-kserve-7bb84d5879-6q4rn\" (UID: \"20cad10d-e5d5-4805-bab2-20ea100955ee\") " pod="llm/e2e-same-id-b-7e57cbca-kserve-7bb84d5879-6q4rn"
Sep 18 20:41:30.110029 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:41:30.109911    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/20cad10d-e5d5-4805-bab2-20ea100955ee-home\") pod \"e2e-same-id-b-7e57cbca-kserve-7bb84d5879-6q4rn\" (UID: \"20cad10d-e5d5-4805-bab2-20ea100955ee\") " pod="llm/e2e-same-id-b-7e57cbca-kserve-7bb84d5879-6q4rn"
Sep 18 20:41:30.110029 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:41:30.109987    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z5zvb\" (UniqueName: \"kubernetes.io/projected/20cad10d-e5d5-4805-bab2-20ea100955ee-kube-api-access-z5zvb\") pod \"e2e-same-id-b-7e57cbca-kserve-7bb84d5879-6q4rn\" (UID: \"20cad10d-e5d5-4805-bab2-20ea100955ee\") " pod="llm/e2e-same-id-b-7e57cbca-kserve-7bb84d5879-6q4rn"
Sep 18 20:41:30.110221 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:41:30.110069    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/20cad10d-e5d5-4805-bab2-20ea100955ee-dshm\") pod \"e2e-same-id-b-7e57cbca-kserve-7bb84d5879-6q4rn\" (UID: \"20cad10d-e5d5-4805-bab2-20ea100955ee\") " pod="llm/e2e-same-id-b-7e57cbca-kserve-7bb84d5879-6q4rn"
Sep 18 20:41:30.110726 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:41:30.110707    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:41:30.210860 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:41:30.210821    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/20cad10d-e5d5-4805-bab2-20ea100955ee-dshm\") pod \"e2e-same-id-b-7e57cbca-kserve-7bb84d5879-6q4rn\" (UID: \"20cad10d-e5d5-4805-bab2-20ea100955ee\") " pod="llm/e2e-same-id-b-7e57cbca-kserve-7bb84d5879-6q4rn"
Sep 18 20:41:30.211054 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:41:30.210896    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/20cad10d-e5d5-4805-bab2-20ea100955ee-tls-certs\") pod \"e2e-same-id-b-7e57cbca-kserve-7bb84d5879-6q4rn\" (UID: \"20cad10d-e5d5-4805-bab2-20ea100955ee\") " pod="llm/e2e-same-id-b-7e57cbca-kserve-7bb84d5879-6q4rn"
Sep 18 20:41:30.211054 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:41:30.211022    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/20cad10d-e5d5-4805-bab2-20ea100955ee-tmp-dir\") pod \"e2e-same-id-b-7e57cbca-kserve-7bb84d5879-6q4rn\" (UID: \"20cad10d-e5d5-4805-bab2-20ea100955ee\") " pod="llm/e2e-same-id-b-7e57cbca-kserve-7bb84d5879-6q4rn"
Sep 18 20:41:30.211174 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:41:30.211058    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/20cad10d-e5d5-4805-bab2-20ea100955ee-model-cache\") pod \"e2e-same-id-b-7e57cbca-kserve-7bb84d5879-6q4rn\" (UID: \"20cad10d-e5d5-4805-bab2-20ea100955ee\") " pod="llm/e2e-same-id-b-7e57cbca-kserve-7bb84d5879-6q4rn"
Sep 18 20:41:30.211174 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:41:30.211082    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/20cad10d-e5d5-4805-bab2-20ea100955ee-home\") pod \"e2e-same-id-b-7e57cbca-kserve-7bb84d5879-6q4rn\" (UID: \"20cad10d-e5d5-4805-bab2-20ea100955ee\") " pod="llm/e2e-same-id-b-7e57cbca-kserve-7bb84d5879-6q4rn"
Sep 18 20:41:30.211174 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:41:30.211112    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z5zvb\" (UniqueName: \"kubernetes.io/projected/20cad10d-e5d5-4805-bab2-20ea100955ee-kube-api-access-z5zvb\") pod \"e2e-same-id-b-7e57cbca-kserve-7bb84d5879-6q4rn\" (UID: \"20cad10d-e5d5-4805-bab2-20ea100955ee\") " pod="llm/e2e-same-id-b-7e57cbca-kserve-7bb84d5879-6q4rn"
Sep 18 20:41:30.211432 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:41:30.211401    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/20cad10d-e5d5-4805-bab2-20ea100955ee-tmp-dir\") pod \"e2e-same-id-b-7e57cbca-kserve-7bb84d5879-6q4rn\" (UID: \"20cad10d-e5d5-4805-bab2-20ea100955ee\") " pod="llm/e2e-same-id-b-7e57cbca-kserve-7bb84d5879-6q4rn"
Sep 18 20:41:30.211528 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:41:30.211452    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/20cad10d-e5d5-4805-bab2-20ea100955ee-model-cache\") pod \"e2e-same-id-b-7e57cbca-kserve-7bb84d5879-6q4rn\" (UID: \"20cad10d-e5d5-4805-bab2-20ea100955ee\") " pod="llm/e2e-same-id-b-7e57cbca-kserve-7bb84d5879-6q4rn"
Sep 18 20:41:30.211528 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:41:30.211476    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/20cad10d-e5d5-4805-bab2-20ea100955ee-home\") pod \"e2e-same-id-b-7e57cbca-kserve-7bb84d5879-6q4rn\" (UID: \"20cad10d-e5d5-4805-bab2-20ea100955ee\") " pod="llm/e2e-same-id-b-7e57cbca-kserve-7bb84d5879-6q4rn"
Sep 18 20:41:30.213630 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:41:30.213603    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/20cad10d-e5d5-4805-bab2-20ea100955ee-dshm\") pod \"e2e-same-id-b-7e57cbca-kserve-7bb84d5879-6q4rn\" (UID: \"20cad10d-e5d5-4805-bab2-20ea100955ee\") " pod="llm/e2e-same-id-b-7e57cbca-kserve-7bb84d5879-6q4rn"
Sep 18 20:41:30.213877 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:41:30.213858    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/20cad10d-e5d5-4805-bab2-20ea100955ee-tls-certs\") pod \"e2e-same-id-b-7e57cbca-kserve-7bb84d5879-6q4rn\" (UID: \"20cad10d-e5d5-4805-bab2-20ea100955ee\") " pod="llm/e2e-same-id-b-7e57cbca-kserve-7bb84d5879-6q4rn"
Sep 18 20:41:30.219134 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:41:30.219107    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z5zvb\" (UniqueName: \"kubernetes.io/projected/20cad10d-e5d5-4805-bab2-20ea100955ee-kube-api-access-z5zvb\") pod \"e2e-same-id-b-7e57cbca-kserve-7bb84d5879-6q4rn\" (UID: \"20cad10d-e5d5-4805-bab2-20ea100955ee\") " pod="llm/e2e-same-id-b-7e57cbca-kserve-7bb84d5879-6q4rn"
Sep 18 20:41:30.311890 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:41:30.311812    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-7e57cbca-kserve-7bb84d5879-6q4rn"
Sep 18 20:41:30.582919 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:41:30.582891    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-7e57cbca-kserve-7bb84d5879-6q4rn"]
Sep 18 20:41:30.585577 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:41:30.585546    2532 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod20cad10d_e5d5_4805_bab2_20ea100955ee.slice/crio-85117ce4b16377ccb55cc838c1a8f93d425743db552d61a220776a83e8c4c3c5 WatchSource:0}: Error finding container 85117ce4b16377ccb55cc838c1a8f93d425743db552d61a220776a83e8c4c3c5: Status 404 returned error can't find the container with id 85117ce4b16377ccb55cc838c1a8f93d425743db552d61a220776a83e8c4c3c5
Sep 18 20:41:30.822642 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:41:30.822600    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-7e57cbca-kserve-7bb84d5879-6q4rn" event={"ID":"20cad10d-e5d5-4805-bab2-20ea100955ee","Type":"ContainerStarted","Data":"e6644345dc0c7658911e8d96dc463f13c2fd36a25aa62c712ae8ba34e6e4e68b"}
Sep 18 20:41:30.822642 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:41:30.822647    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-7e57cbca-kserve-7bb84d5879-6q4rn" event={"ID":"20cad10d-e5d5-4805-bab2-20ea100955ee","Type":"ContainerStarted","Data":"85117ce4b16377ccb55cc838c1a8f93d425743db552d61a220776a83e8c4c3c5"}
Sep 18 20:41:30.841369 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:41:30.841264    2532 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-b-7e57cbca-kserve-7bb84d5879-6q4rn" podStartSLOduration=1.841248387 podStartE2EDuration="1.841248387s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 20:41:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 20:41:30.838747849 +0000 UTC m=+2422.604137248" watchObservedRunningTime="2026-09-18 20:41:30.841248387 +0000 UTC m=+2422.606637786"
Sep 18 20:41:32.894087 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:41:32.894047    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-vptdl"]
Sep 18 20:41:40.312873 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:41:40.312842    2532 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-b-7e57cbca-kserve-7bb84d5879-6q4rn"
Sep 18 20:41:40.313370 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:41:40.312885    2532 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-b-7e57cbca-kserve-7bb84d5879-6q4rn"
Sep 18 20:41:40.325271 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:41:40.325240    2532 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-b-7e57cbca-kserve-7bb84d5879-6q4rn"
Sep 18 20:41:40.891736 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:41:40.891707    2532 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-b-7e57cbca-kserve-7bb84d5879-6q4rn"
Sep 18 20:42:14.293641 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:42:14.293605    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:42:15.484643 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:42:15.484611    2532 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-7e57cbca-kserve-7bb84d5879-6q4rn"]
Sep 18 20:42:15.506001 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:42:15.505970    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:42:17.314395 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:42:17.314359    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-vptdl"]
Sep 18 20:42:17.798209 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:42:17.798165    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:42:17.898712 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:42:17.898675    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:42:24.791851 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:42:24.791811    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-vptdl"]
Sep 18 20:42:30.504916 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:42:30.504872    2532 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-b-7e57cbca-kserve-7bb84d5879-6q4rn" podUID="20cad10d-e5d5-4805-bab2-20ea100955ee" containerName="main" containerID="cri-o://e6644345dc0c7658911e8d96dc463f13c2fd36a25aa62c712ae8ba34e6e4e68b" gracePeriod=45
Sep 18 20:42:30.751856 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:42:30.751828    2532 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-7e57cbca-kserve-7bb84d5879-6q4rn"
Sep 18 20:42:30.854736 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:42:30.854649    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/20cad10d-e5d5-4805-bab2-20ea100955ee-model-cache\") pod \"20cad10d-e5d5-4805-bab2-20ea100955ee\" (UID: \"20cad10d-e5d5-4805-bab2-20ea100955ee\") "
Sep 18 20:42:30.854919 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:42:30.854737    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z5zvb\" (UniqueName: \"kubernetes.io/projected/20cad10d-e5d5-4805-bab2-20ea100955ee-kube-api-access-z5zvb\") pod \"20cad10d-e5d5-4805-bab2-20ea100955ee\" (UID: \"20cad10d-e5d5-4805-bab2-20ea100955ee\") "
Sep 18 20:42:30.854919 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:42:30.854784    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/20cad10d-e5d5-4805-bab2-20ea100955ee-tls-certs\") pod \"20cad10d-e5d5-4805-bab2-20ea100955ee\" (UID: \"20cad10d-e5d5-4805-bab2-20ea100955ee\") "
Sep 18 20:42:30.854919 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:42:30.854802    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/20cad10d-e5d5-4805-bab2-20ea100955ee-home\") pod \"20cad10d-e5d5-4805-bab2-20ea100955ee\" (UID: \"20cad10d-e5d5-4805-bab2-20ea100955ee\") "
Sep 18 20:42:30.854919 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:42:30.854842    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/20cad10d-e5d5-4805-bab2-20ea100955ee-dshm\") pod \"20cad10d-e5d5-4805-bab2-20ea100955ee\" (UID: \"20cad10d-e5d5-4805-bab2-20ea100955ee\") "
Sep 18 20:42:30.854919 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:42:30.854876    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/20cad10d-e5d5-4805-bab2-20ea100955ee-tmp-dir\") pod \"20cad10d-e5d5-4805-bab2-20ea100955ee\" (UID: \"20cad10d-e5d5-4805-bab2-20ea100955ee\") "
Sep 18 20:42:30.855198 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:42:30.854984    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/20cad10d-e5d5-4805-bab2-20ea100955ee-model-cache" (OuterVolumeSpecName: "model-cache") pod "20cad10d-e5d5-4805-bab2-20ea100955ee" (UID: "20cad10d-e5d5-4805-bab2-20ea100955ee"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 20:42:30.855198 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:42:30.855071    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/20cad10d-e5d5-4805-bab2-20ea100955ee-home" (OuterVolumeSpecName: "home") pod "20cad10d-e5d5-4805-bab2-20ea100955ee" (UID: "20cad10d-e5d5-4805-bab2-20ea100955ee"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 20:42:30.855198 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:42:30.855178    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/20cad10d-e5d5-4805-bab2-20ea100955ee-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "20cad10d-e5d5-4805-bab2-20ea100955ee" (UID: "20cad10d-e5d5-4805-bab2-20ea100955ee"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 20:42:30.855355 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:42:30.855225    2532 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/20cad10d-e5d5-4805-bab2-20ea100955ee-model-cache\") on node \"ip-10-0-130-81.ec2.internal\" DevicePath \"\""
Sep 18 20:42:30.855355 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:42:30.855245    2532 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/20cad10d-e5d5-4805-bab2-20ea100955ee-home\") on node \"ip-10-0-130-81.ec2.internal\" DevicePath \"\""
Sep 18 20:42:30.857143 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:42:30.857102    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/20cad10d-e5d5-4805-bab2-20ea100955ee-dshm" (OuterVolumeSpecName: "dshm") pod "20cad10d-e5d5-4805-bab2-20ea100955ee" (UID: "20cad10d-e5d5-4805-bab2-20ea100955ee"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 20:42:30.857284 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:42:30.857181    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/20cad10d-e5d5-4805-bab2-20ea100955ee-kube-api-access-z5zvb" (OuterVolumeSpecName: "kube-api-access-z5zvb") pod "20cad10d-e5d5-4805-bab2-20ea100955ee" (UID: "20cad10d-e5d5-4805-bab2-20ea100955ee"). InnerVolumeSpecName "kube-api-access-z5zvb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 20:42:30.857284 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:42:30.857186    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/20cad10d-e5d5-4805-bab2-20ea100955ee-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "20cad10d-e5d5-4805-bab2-20ea100955ee" (UID: "20cad10d-e5d5-4805-bab2-20ea100955ee"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 20:42:30.956009 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:42:30.955972    2532 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/20cad10d-e5d5-4805-bab2-20ea100955ee-tls-certs\") on node \"ip-10-0-130-81.ec2.internal\" DevicePath \"\""
Sep 18 20:42:30.956009 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:42:30.955998    2532 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/20cad10d-e5d5-4805-bab2-20ea100955ee-dshm\") on node \"ip-10-0-130-81.ec2.internal\" DevicePath \"\""
Sep 18 20:42:30.956009 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:42:30.956006    2532 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/20cad10d-e5d5-4805-bab2-20ea100955ee-tmp-dir\") on node \"ip-10-0-130-81.ec2.internal\" DevicePath \"\""
Sep 18 20:42:30.956009 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:42:30.956015    2532 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z5zvb\" (UniqueName: \"kubernetes.io/projected/20cad10d-e5d5-4805-bab2-20ea100955ee-kube-api-access-z5zvb\") on node \"ip-10-0-130-81.ec2.internal\" DevicePath \"\""
Sep 18 20:42:31.136360 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:42:31.136281    2532 generic.go:356] "Generic (PLEG): container finished" podID="20cad10d-e5d5-4805-bab2-20ea100955ee" containerID="e6644345dc0c7658911e8d96dc463f13c2fd36a25aa62c712ae8ba34e6e4e68b" exitCode=0
Sep 18 20:42:31.136360 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:42:31.136323    2532 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-7e57cbca-kserve-7bb84d5879-6q4rn"
Sep 18 20:42:31.136558 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:42:31.136356    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-7e57cbca-kserve-7bb84d5879-6q4rn" event={"ID":"20cad10d-e5d5-4805-bab2-20ea100955ee","Type":"ContainerDied","Data":"e6644345dc0c7658911e8d96dc463f13c2fd36a25aa62c712ae8ba34e6e4e68b"}
Sep 18 20:42:31.136558 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:42:31.136402    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-7e57cbca-kserve-7bb84d5879-6q4rn" event={"ID":"20cad10d-e5d5-4805-bab2-20ea100955ee","Type":"ContainerDied","Data":"85117ce4b16377ccb55cc838c1a8f93d425743db552d61a220776a83e8c4c3c5"}
Sep 18 20:42:31.136558 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:42:31.136423    2532 scope.go:117] "RemoveContainer" containerID="e6644345dc0c7658911e8d96dc463f13c2fd36a25aa62c712ae8ba34e6e4e68b"
Sep 18 20:42:31.145870 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:42:31.145851    2532 scope.go:117] "RemoveContainer" containerID="e6644345dc0c7658911e8d96dc463f13c2fd36a25aa62c712ae8ba34e6e4e68b"
Sep 18 20:42:31.146155 ip-10-0-130-81 kubenswrapper[2532]: E0918 20:42:31.146136    2532 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e6644345dc0c7658911e8d96dc463f13c2fd36a25aa62c712ae8ba34e6e4e68b\": container with ID starting with e6644345dc0c7658911e8d96dc463f13c2fd36a25aa62c712ae8ba34e6e4e68b not found: ID does not exist" containerID="e6644345dc0c7658911e8d96dc463f13c2fd36a25aa62c712ae8ba34e6e4e68b"
Sep 18 20:42:31.146233 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:42:31.146162    2532 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e6644345dc0c7658911e8d96dc463f13c2fd36a25aa62c712ae8ba34e6e4e68b"} err="failed to get container status \"e6644345dc0c7658911e8d96dc463f13c2fd36a25aa62c712ae8ba34e6e4e68b\": rpc error: code = NotFound desc = could not find container \"e6644345dc0c7658911e8d96dc463f13c2fd36a25aa62c712ae8ba34e6e4e68b\": container with ID starting with e6644345dc0c7658911e8d96dc463f13c2fd36a25aa62c712ae8ba34e6e4e68b not found: ID does not exist"
Sep 18 20:42:31.157501 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:42:31.157465    2532 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-7e57cbca-kserve-7bb84d5879-6q4rn"]
Sep 18 20:42:31.161033 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:42:31.161011    2532 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-b-7e57cbca-kserve-7bb84d5879-6q4rn"]
Sep 18 20:42:32.305406 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:42:32.305370    2532 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="20cad10d-e5d5-4805-bab2-20ea100955ee" path="/var/lib/kubelet/pods/20cad10d-e5d5-4805-bab2-20ea100955ee/volumes"
Sep 18 20:42:32.397545 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:42:32.397515    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-vptdl"]
Sep 18 20:42:32.508231 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:42:32.508193    2532 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-59f6566bbc-bd9vx"]
Sep 18 20:42:32.508538 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:42:32.508526    2532 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="20cad10d-e5d5-4805-bab2-20ea100955ee" containerName="main"
Sep 18 20:42:32.508587 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:42:32.508539    2532 state_mem.go:107] "Deleted CPUSet assignment" podUID="20cad10d-e5d5-4805-bab2-20ea100955ee" containerName="main"
Sep 18 20:42:32.508587 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:42:32.508547    2532 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dca54f78-d480-4710-a3be-1f730b123812" containerName="istio-proxy"
Sep 18 20:42:32.508587 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:42:32.508553    2532 state_mem.go:107] "Deleted CPUSet assignment" podUID="dca54f78-d480-4710-a3be-1f730b123812" containerName="istio-proxy"
Sep 18 20:42:32.508679 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:42:32.508625    2532 memory_manager.go:368] "RemoveStaleState removing state" podUID="20cad10d-e5d5-4805-bab2-20ea100955ee" containerName="main"
Sep 18 20:42:32.513045 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:42:32.513024    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-59f6566bbc-bd9vx"
Sep 18 20:42:32.515678 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:42:32.515657    2532 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 18 20:42:32.516833 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:42:32.516794    2532 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Sep 18 20:42:32.516833 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:42:32.516794    2532 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Sep 18 20:42:32.517004 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:42:32.516840    2532 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\""
Sep 18 20:42:32.517004 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:42:32.516869    2532 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-fhqv4\""
Sep 18 20:42:32.521678 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:42:32.521654    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-59f6566bbc-bd9vx"]
Sep 18 20:42:32.671820 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:42:32.671736    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c679641e-6128-419d-a841-eab743d1b274-maas-api-tls\") pod \"maas-api-59f6566bbc-bd9vx\" (UID: \"c679641e-6128-419d-a841-eab743d1b274\") " pod="odh-ai-gateway-infra/maas-api-59f6566bbc-bd9vx"
Sep 18 20:42:32.671820 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:42:32.671772    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4xps7\" (UniqueName: \"kubernetes.io/projected/c679641e-6128-419d-a841-eab743d1b274-kube-api-access-4xps7\") pod \"maas-api-59f6566bbc-bd9vx\" (UID: \"c679641e-6128-419d-a841-eab743d1b274\") " pod="odh-ai-gateway-infra/maas-api-59f6566bbc-bd9vx"
Sep 18 20:42:32.671820 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:42:32.671798    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c679641e-6128-419d-a841-eab743d1b274-openshift-service-ca\") pod \"maas-api-59f6566bbc-bd9vx\" (UID: \"c679641e-6128-419d-a841-eab743d1b274\") " pod="odh-ai-gateway-infra/maas-api-59f6566bbc-bd9vx"
Sep 18 20:42:32.672036 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:42:32.671911    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c679641e-6128-419d-a841-eab743d1b274-metrics-certs\") pod \"maas-api-59f6566bbc-bd9vx\" (UID: \"c679641e-6128-419d-a841-eab743d1b274\") " pod="odh-ai-gateway-infra/maas-api-59f6566bbc-bd9vx"
Sep 18 20:42:32.772566 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:42:32.772527    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c679641e-6128-419d-a841-eab743d1b274-openshift-service-ca\") pod \"maas-api-59f6566bbc-bd9vx\" (UID: \"c679641e-6128-419d-a841-eab743d1b274\") " pod="odh-ai-gateway-infra/maas-api-59f6566bbc-bd9vx"
Sep 18 20:42:32.772742 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:42:32.772601    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c679641e-6128-419d-a841-eab743d1b274-metrics-certs\") pod \"maas-api-59f6566bbc-bd9vx\" (UID: \"c679641e-6128-419d-a841-eab743d1b274\") " pod="odh-ai-gateway-infra/maas-api-59f6566bbc-bd9vx"
Sep 18 20:42:32.772742 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:42:32.772654    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c679641e-6128-419d-a841-eab743d1b274-maas-api-tls\") pod \"maas-api-59f6566bbc-bd9vx\" (UID: \"c679641e-6128-419d-a841-eab743d1b274\") " pod="odh-ai-gateway-infra/maas-api-59f6566bbc-bd9vx"
Sep 18 20:42:32.772742 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:42:32.772679    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4xps7\" (UniqueName: \"kubernetes.io/projected/c679641e-6128-419d-a841-eab743d1b274-kube-api-access-4xps7\") pod \"maas-api-59f6566bbc-bd9vx\" (UID: \"c679641e-6128-419d-a841-eab743d1b274\") " pod="odh-ai-gateway-infra/maas-api-59f6566bbc-bd9vx"
Sep 18 20:42:32.773197 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:42:32.773178    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c679641e-6128-419d-a841-eab743d1b274-openshift-service-ca\") pod \"maas-api-59f6566bbc-bd9vx\" (UID: \"c679641e-6128-419d-a841-eab743d1b274\") " pod="odh-ai-gateway-infra/maas-api-59f6566bbc-bd9vx"
Sep 18 20:42:32.775125 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:42:32.775096    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c679641e-6128-419d-a841-eab743d1b274-metrics-certs\") pod \"maas-api-59f6566bbc-bd9vx\" (UID: \"c679641e-6128-419d-a841-eab743d1b274\") " pod="odh-ai-gateway-infra/maas-api-59f6566bbc-bd9vx"
Sep 18 20:42:32.775231 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:42:32.775125    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c679641e-6128-419d-a841-eab743d1b274-maas-api-tls\") pod \"maas-api-59f6566bbc-bd9vx\" (UID: \"c679641e-6128-419d-a841-eab743d1b274\") " pod="odh-ai-gateway-infra/maas-api-59f6566bbc-bd9vx"
Sep 18 20:42:32.780271 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:42:32.780239    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4xps7\" (UniqueName: \"kubernetes.io/projected/c679641e-6128-419d-a841-eab743d1b274-kube-api-access-4xps7\") pod \"maas-api-59f6566bbc-bd9vx\" (UID: \"c679641e-6128-419d-a841-eab743d1b274\") " pod="odh-ai-gateway-infra/maas-api-59f6566bbc-bd9vx"
Sep 18 20:42:32.824796 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:42:32.824761    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-59f6566bbc-bd9vx"
Sep 18 20:42:33.095594 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:42:33.095568    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-59f6566bbc-bd9vx"]
Sep 18 20:42:33.097583 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:42:33.097553    2532 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc679641e_6128_419d_a841_eab743d1b274.slice/crio-26aeb6223694e120947b6fe548dcb6f3cf2527fb4ea9be33c173d6f14249a888 WatchSource:0}: Error finding container 26aeb6223694e120947b6fe548dcb6f3cf2527fb4ea9be33c173d6f14249a888: Status 404 returned error can't find the container with id 26aeb6223694e120947b6fe548dcb6f3cf2527fb4ea9be33c173d6f14249a888
Sep 18 20:42:33.098885 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:42:33.098867    2532 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 18 20:42:33.148368 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:42:33.148329    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-59f6566bbc-bd9vx" event={"ID":"c679641e-6128-419d-a841-eab743d1b274","Type":"ContainerStarted","Data":"26aeb6223694e120947b6fe548dcb6f3cf2527fb4ea9be33c173d6f14249a888"}
Sep 18 20:42:34.155504 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:42:34.155450    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-59f6566bbc-bd9vx" event={"ID":"c679641e-6128-419d-a841-eab743d1b274","Type":"ContainerStarted","Data":"5834bfbd31da879c7bcf0ae121cb2daa1d350b0911a6b047f1971f37ad811736"}
Sep 18 20:42:34.155965 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:42:34.155660    2532 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-59f6566bbc-bd9vx"
Sep 18 20:42:34.171671 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:42:34.171600    2532 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-59f6566bbc-bd9vx" podStartSLOduration=1.8534968859999998 podStartE2EDuration="2.171580537s" totalImagesPullingTime="318.083651ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 20:42:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 20:42:34.171318852 +0000 UTC m=+2485.936708253" watchObservedRunningTime="2026-09-18 20:42:34.171580537 +0000 UTC m=+2485.936969936"
Sep 18 20:42:38.493333 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:42:38.493294    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-vptdl"]
Sep 18 20:42:39.565946 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:42:39.565913    2532 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-59f6566bbc-bd9vx"]
Sep 18 20:42:39.566411 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:42:39.566177    2532 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-59f6566bbc-bd9vx" podUID="c679641e-6128-419d-a841-eab743d1b274" containerName="maas-api" containerID="cri-o://5834bfbd31da879c7bcf0ae121cb2daa1d350b0911a6b047f1971f37ad811736" gracePeriod=30
Sep 18 20:42:39.572758 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:42:39.572727    2532 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-59f6566bbc-bd9vx"
Sep 18 20:42:39.598048 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:42:39.598014    2532 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-86d9ddc875-ngzwq"]
Sep 18 20:42:39.601454 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:42:39.601439    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-86d9ddc875-ngzwq"
Sep 18 20:42:39.612653 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:42:39.612616    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-86d9ddc875-ngzwq"]
Sep 18 20:42:39.737125 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:42:39.737057    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c49afe7b-d9c5-4df4-8681-b64f8769d558-maas-api-tls\") pod \"maas-api-86d9ddc875-ngzwq\" (UID: \"c49afe7b-d9c5-4df4-8681-b64f8769d558\") " pod="odh-ai-gateway-infra/maas-api-86d9ddc875-ngzwq"
Sep 18 20:42:39.737125 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:42:39.737098    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z587s\" (UniqueName: \"kubernetes.io/projected/c49afe7b-d9c5-4df4-8681-b64f8769d558-kube-api-access-z587s\") pod \"maas-api-86d9ddc875-ngzwq\" (UID: \"c49afe7b-d9c5-4df4-8681-b64f8769d558\") " pod="odh-ai-gateway-infra/maas-api-86d9ddc875-ngzwq"
Sep 18 20:42:39.737312 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:42:39.737134    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c49afe7b-d9c5-4df4-8681-b64f8769d558-openshift-service-ca\") pod \"maas-api-86d9ddc875-ngzwq\" (UID: \"c49afe7b-d9c5-4df4-8681-b64f8769d558\") " pod="odh-ai-gateway-infra/maas-api-86d9ddc875-ngzwq"
Sep 18 20:42:39.737312 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:42:39.737202    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c49afe7b-d9c5-4df4-8681-b64f8769d558-metrics-certs\") pod \"maas-api-86d9ddc875-ngzwq\" (UID: \"c49afe7b-d9c5-4df4-8681-b64f8769d558\") " pod="odh-ai-gateway-infra/maas-api-86d9ddc875-ngzwq"
Sep 18 20:42:39.817909 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:42:39.817846    2532 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-59f6566bbc-bd9vx"
Sep 18 20:42:39.838192 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:42:39.838089    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c49afe7b-d9c5-4df4-8681-b64f8769d558-metrics-certs\") pod \"maas-api-86d9ddc875-ngzwq\" (UID: \"c49afe7b-d9c5-4df4-8681-b64f8769d558\") " pod="odh-ai-gateway-infra/maas-api-86d9ddc875-ngzwq"
Sep 18 20:42:39.838365 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:42:39.838240    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c49afe7b-d9c5-4df4-8681-b64f8769d558-maas-api-tls\") pod \"maas-api-86d9ddc875-ngzwq\" (UID: \"c49afe7b-d9c5-4df4-8681-b64f8769d558\") " pod="odh-ai-gateway-infra/maas-api-86d9ddc875-ngzwq"
Sep 18 20:42:39.838365 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:42:39.838275    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z587s\" (UniqueName: \"kubernetes.io/projected/c49afe7b-d9c5-4df4-8681-b64f8769d558-kube-api-access-z587s\") pod \"maas-api-86d9ddc875-ngzwq\" (UID: \"c49afe7b-d9c5-4df4-8681-b64f8769d558\") " pod="odh-ai-gateway-infra/maas-api-86d9ddc875-ngzwq"
Sep 18 20:42:39.838365 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:42:39.838310    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c49afe7b-d9c5-4df4-8681-b64f8769d558-openshift-service-ca\") pod \"maas-api-86d9ddc875-ngzwq\" (UID: \"c49afe7b-d9c5-4df4-8681-b64f8769d558\") " pod="odh-ai-gateway-infra/maas-api-86d9ddc875-ngzwq"
Sep 18 20:42:39.839107 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:42:39.839049    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c49afe7b-d9c5-4df4-8681-b64f8769d558-openshift-service-ca\") pod \"maas-api-86d9ddc875-ngzwq\" (UID: \"c49afe7b-d9c5-4df4-8681-b64f8769d558\") " pod="odh-ai-gateway-infra/maas-api-86d9ddc875-ngzwq"
Sep 18 20:42:39.840890 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:42:39.840869    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c49afe7b-d9c5-4df4-8681-b64f8769d558-metrics-certs\") pod \"maas-api-86d9ddc875-ngzwq\" (UID: \"c49afe7b-d9c5-4df4-8681-b64f8769d558\") " pod="odh-ai-gateway-infra/maas-api-86d9ddc875-ngzwq"
Sep 18 20:42:39.841020 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:42:39.841003    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c49afe7b-d9c5-4df4-8681-b64f8769d558-maas-api-tls\") pod \"maas-api-86d9ddc875-ngzwq\" (UID: \"c49afe7b-d9c5-4df4-8681-b64f8769d558\") " pod="odh-ai-gateway-infra/maas-api-86d9ddc875-ngzwq"
Sep 18 20:42:39.846153 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:42:39.846130    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z587s\" (UniqueName: \"kubernetes.io/projected/c49afe7b-d9c5-4df4-8681-b64f8769d558-kube-api-access-z587s\") pod \"maas-api-86d9ddc875-ngzwq\" (UID: \"c49afe7b-d9c5-4df4-8681-b64f8769d558\") " pod="odh-ai-gateway-infra/maas-api-86d9ddc875-ngzwq"
Sep 18 20:42:39.939597 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:42:39.939567    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c679641e-6128-419d-a841-eab743d1b274-maas-api-tls\") pod \"c679641e-6128-419d-a841-eab743d1b274\" (UID: \"c679641e-6128-419d-a841-eab743d1b274\") "
Sep 18 20:42:39.939777 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:42:39.939619    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c679641e-6128-419d-a841-eab743d1b274-openshift-service-ca\") pod \"c679641e-6128-419d-a841-eab743d1b274\" (UID: \"c679641e-6128-419d-a841-eab743d1b274\") "
Sep 18 20:42:39.939777 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:42:39.939654    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c679641e-6128-419d-a841-eab743d1b274-metrics-certs\") pod \"c679641e-6128-419d-a841-eab743d1b274\" (UID: \"c679641e-6128-419d-a841-eab743d1b274\") "
Sep 18 20:42:39.939777 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:42:39.939697    2532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4xps7\" (UniqueName: \"kubernetes.io/projected/c679641e-6128-419d-a841-eab743d1b274-kube-api-access-4xps7\") pod \"c679641e-6128-419d-a841-eab743d1b274\" (UID: \"c679641e-6128-419d-a841-eab743d1b274\") "
Sep 18 20:42:39.940061 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:42:39.940037    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c679641e-6128-419d-a841-eab743d1b274-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "c679641e-6128-419d-a841-eab743d1b274" (UID: "c679641e-6128-419d-a841-eab743d1b274"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 20:42:39.941761 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:42:39.941735    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c679641e-6128-419d-a841-eab743d1b274-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "c679641e-6128-419d-a841-eab743d1b274" (UID: "c679641e-6128-419d-a841-eab743d1b274"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 20:42:39.941858 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:42:39.941824    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c679641e-6128-419d-a841-eab743d1b274-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "c679641e-6128-419d-a841-eab743d1b274" (UID: "c679641e-6128-419d-a841-eab743d1b274"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 20:42:39.941858 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:42:39.941838    2532 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c679641e-6128-419d-a841-eab743d1b274-kube-api-access-4xps7" (OuterVolumeSpecName: "kube-api-access-4xps7") pod "c679641e-6128-419d-a841-eab743d1b274" (UID: "c679641e-6128-419d-a841-eab743d1b274"). InnerVolumeSpecName "kube-api-access-4xps7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 20:42:39.949006 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:42:39.948990    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-86d9ddc875-ngzwq"
Sep 18 20:42:40.041126 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:42:40.041091    2532 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c679641e-6128-419d-a841-eab743d1b274-metrics-certs\") on node \"ip-10-0-130-81.ec2.internal\" DevicePath \"\""
Sep 18 20:42:40.041126 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:42:40.041119    2532 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4xps7\" (UniqueName: \"kubernetes.io/projected/c679641e-6128-419d-a841-eab743d1b274-kube-api-access-4xps7\") on node \"ip-10-0-130-81.ec2.internal\" DevicePath \"\""
Sep 18 20:42:40.041126 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:42:40.041130    2532 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c679641e-6128-419d-a841-eab743d1b274-maas-api-tls\") on node \"ip-10-0-130-81.ec2.internal\" DevicePath \"\""
Sep 18 20:42:40.041355 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:42:40.041138    2532 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c679641e-6128-419d-a841-eab743d1b274-openshift-service-ca\") on node \"ip-10-0-130-81.ec2.internal\" DevicePath \"\""
Sep 18 20:42:40.187595 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:42:40.187512    2532 generic.go:356] "Generic (PLEG): container finished" podID="c679641e-6128-419d-a841-eab743d1b274" containerID="5834bfbd31da879c7bcf0ae121cb2daa1d350b0911a6b047f1971f37ad811736" exitCode=0
Sep 18 20:42:40.187595 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:42:40.187552    2532 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-59f6566bbc-bd9vx"
Sep 18 20:42:40.187770 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:42:40.187590    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-59f6566bbc-bd9vx" event={"ID":"c679641e-6128-419d-a841-eab743d1b274","Type":"ContainerDied","Data":"5834bfbd31da879c7bcf0ae121cb2daa1d350b0911a6b047f1971f37ad811736"}
Sep 18 20:42:40.187770 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:42:40.187644    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-59f6566bbc-bd9vx" event={"ID":"c679641e-6128-419d-a841-eab743d1b274","Type":"ContainerDied","Data":"26aeb6223694e120947b6fe548dcb6f3cf2527fb4ea9be33c173d6f14249a888"}
Sep 18 20:42:40.187770 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:42:40.187670    2532 scope.go:117] "RemoveContainer" containerID="5834bfbd31da879c7bcf0ae121cb2daa1d350b0911a6b047f1971f37ad811736"
Sep 18 20:42:40.197022 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:42:40.197004    2532 scope.go:117] "RemoveContainer" containerID="5834bfbd31da879c7bcf0ae121cb2daa1d350b0911a6b047f1971f37ad811736"
Sep 18 20:42:40.197295 ip-10-0-130-81 kubenswrapper[2532]: E0918 20:42:40.197276    2532 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5834bfbd31da879c7bcf0ae121cb2daa1d350b0911a6b047f1971f37ad811736\": container with ID starting with 5834bfbd31da879c7bcf0ae121cb2daa1d350b0911a6b047f1971f37ad811736 not found: ID does not exist" containerID="5834bfbd31da879c7bcf0ae121cb2daa1d350b0911a6b047f1971f37ad811736"
Sep 18 20:42:40.197363 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:42:40.197308    2532 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5834bfbd31da879c7bcf0ae121cb2daa1d350b0911a6b047f1971f37ad811736"} err="failed to get container status \"5834bfbd31da879c7bcf0ae121cb2daa1d350b0911a6b047f1971f37ad811736\": rpc error: code = NotFound desc = could not find container \"5834bfbd31da879c7bcf0ae121cb2daa1d350b0911a6b047f1971f37ad811736\": container with ID starting with 5834bfbd31da879c7bcf0ae121cb2daa1d350b0911a6b047f1971f37ad811736 not found: ID does not exist"
Sep 18 20:42:40.208910 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:42:40.208883    2532 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-59f6566bbc-bd9vx"]
Sep 18 20:42:40.214228 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:42:40.214200    2532 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-59f6566bbc-bd9vx"]
Sep 18 20:42:40.220155 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:42:40.220133    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-86d9ddc875-ngzwq"]
Sep 18 20:42:40.221927 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:42:40.221893    2532 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc49afe7b_d9c5_4df4_8681_b64f8769d558.slice/crio-645d0b0aab9850b215a211f6146496ff1c2e6e889c4ad2861a0fd57c396eb480 WatchSource:0}: Error finding container 645d0b0aab9850b215a211f6146496ff1c2e6e889c4ad2861a0fd57c396eb480: Status 404 returned error can't find the container with id 645d0b0aab9850b215a211f6146496ff1c2e6e889c4ad2861a0fd57c396eb480
Sep 18 20:42:40.306704 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:42:40.306669    2532 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c679641e-6128-419d-a841-eab743d1b274" path="/var/lib/kubelet/pods/c679641e-6128-419d-a841-eab743d1b274/volumes"
Sep 18 20:42:41.194548 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:42:41.194478    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-86d9ddc875-ngzwq" event={"ID":"c49afe7b-d9c5-4df4-8681-b64f8769d558","Type":"ContainerStarted","Data":"2f4085ca0d25f7b67ffa99a28d52c63eabe283196dabba38f4a7a490266ab0ff"}
Sep 18 20:42:41.195008 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:42:41.194684    2532 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-86d9ddc875-ngzwq"
Sep 18 20:42:41.195008 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:42:41.194705    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-86d9ddc875-ngzwq" event={"ID":"c49afe7b-d9c5-4df4-8681-b64f8769d558","Type":"ContainerStarted","Data":"645d0b0aab9850b215a211f6146496ff1c2e6e889c4ad2861a0fd57c396eb480"}
Sep 18 20:42:41.211843 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:42:41.211786    2532 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-86d9ddc875-ngzwq" podStartSLOduration=1.871461499 podStartE2EDuration="2.211772491s" totalImagesPullingTime="340.310992ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 20:42:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 20:42:41.210015166 +0000 UTC m=+2492.975404564" watchObservedRunningTime="2026-09-18 20:42:41.211772491 +0000 UTC m=+2492.977161890"
Sep 18 20:42:43.052913 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:42:43.052878    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:42:43.493402 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:42:43.493363    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-vptdl"]
Sep 18 20:42:47.205965 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:42:47.205931    2532 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-86d9ddc875-ngzwq"
Sep 18 20:42:49.293452 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:42:49.293407    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-vptdl"]
Sep 18 20:42:55.991293 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:42:55.991211    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-vptdl"]
Sep 18 20:43:00.079738 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:43:00.079697    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:43:00.147227 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:43:00.147163    2532 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log" limit=1024
Sep 18 20:43:02.895222 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:43:02.895188    2532 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-gd9qz_b377a9e6-dbbb-4023-8997-70178a63034f/global-pull-secret-syncer/0.log"
Sep 18 20:43:02.988951 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:43:02.988911    2532 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-g749g_d94756ff-5c71-43bc-ab39-b0fde15a36a2/konnectivity-agent/0.log"
Sep 18 20:43:03.143168 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:43:03.143140    2532 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-130-81.ec2.internal_7719088cb6c9e3ddb918a0f82975ee3c/haproxy/0.log"
Sep 18 20:43:07.488891 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:43:07.488856    2532 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-68d6f84597-k9t4q_3d16378d-1758-4be8-b381-f4f337ba2efc/authorino/0.log"
Sep 18 20:43:07.599398 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:43:07.599370    2532 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-fbdd4f445-hnxk4_8d1b6ff3-8023-4e34-9e10-2bcc51ef5468/kuadrant-console-plugin/0.log"
Sep 18 20:43:07.665309 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:43:07.665276    2532 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-84994f7ffc-vptdl_a9d9b684-8e0c-404c-b00f-a18940fbfe2f/limitador/0.log"
Sep 18 20:43:09.426439 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:43:09.426409    2532 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-847b8bd8d7-6k4dr_dba6d982-36df-45cd-bb26-4deb21429b12/cluster-monitoring-operator/0.log"
Sep 18 20:43:09.525175 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:43:09.525133    2532 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-647c4f6b67-8bh9b_83993929-c843-41ee-825b-7362813e6a1d/metrics-server/0.log"
Sep 18 20:43:09.772641 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:43:09.772611    2532 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-trx7g_73d422ef-a46d-46ec-baba-15a648a415fe/node-exporter/0.log"
Sep 18 20:43:09.793674 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:43:09.793649    2532 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-trx7g_73d422ef-a46d-46ec-baba-15a648a415fe/kube-rbac-proxy/0.log"
Sep 18 20:43:09.819884 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:43:09.819862    2532 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-trx7g_73d422ef-a46d-46ec-baba-15a648a415fe/init-textfile/0.log"
Sep 18 20:43:10.283607 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:43:10.283578    2532 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-fff57f85f-8qw8c_fd54629b-72af-4c18-b7e1-1fcda1e4e6d4/telemeter-client/0.log"
Sep 18 20:43:10.309529 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:43:10.309429    2532 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-fff57f85f-8qw8c_fd54629b-72af-4c18-b7e1-1fcda1e4e6d4/reload/0.log"
Sep 18 20:43:10.334600 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:43:10.334572    2532 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-fff57f85f-8qw8c_fd54629b-72af-4c18-b7e1-1fcda1e4e6d4/kube-rbac-proxy/0.log"
Sep 18 20:43:12.449475 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:43:12.449447    2532 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-m7wdd_32e33d39-7826-499d-bc01-0487b7580e7a/console-operator/1.log"
Sep 18 20:43:12.454009 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:43:12.453992    2532 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-m7wdd_32e33d39-7826-499d-bc01-0487b7580e7a/console-operator/2.log"
Sep 18 20:43:12.648587 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:43:12.648552    2532 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-jwttx/perf-node-gather-daemonset-h8vxp"]
Sep 18 20:43:12.648903 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:43:12.648891    2532 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c679641e-6128-419d-a841-eab743d1b274" containerName="maas-api"
Sep 18 20:43:12.648948 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:43:12.648905    2532 state_mem.go:107] "Deleted CPUSet assignment" podUID="c679641e-6128-419d-a841-eab743d1b274" containerName="maas-api"
Sep 18 20:43:12.648984 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:43:12.648959    2532 memory_manager.go:368] "RemoveStaleState removing state" podUID="c679641e-6128-419d-a841-eab743d1b274" containerName="maas-api"
Sep 18 20:43:12.653469 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:43:12.653447    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-jwttx/perf-node-gather-daemonset-h8vxp"
Sep 18 20:43:12.656473 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:43:12.656447    2532 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-jwttx\"/\"openshift-service-ca.crt\""
Sep 18 20:43:12.657642 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:43:12.657624    2532 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-jwttx\"/\"default-dockercfg-hp79t\""
Sep 18 20:43:12.657710 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:43:12.657650    2532 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-jwttx\"/\"kube-root-ca.crt\""
Sep 18 20:43:12.661947 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:43:12.661923    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-jwttx/perf-node-gather-daemonset-h8vxp"]
Sep 18 20:43:12.719455 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:43:12.719410    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/bfe376b6-04a5-4d7a-9d9d-151e9fa86750-podres\") pod \"perf-node-gather-daemonset-h8vxp\" (UID: \"bfe376b6-04a5-4d7a-9d9d-151e9fa86750\") " pod="openshift-must-gather-jwttx/perf-node-gather-daemonset-h8vxp"
Sep 18 20:43:12.719455 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:43:12.719462    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/bfe376b6-04a5-4d7a-9d9d-151e9fa86750-lib-modules\") pod \"perf-node-gather-daemonset-h8vxp\" (UID: \"bfe376b6-04a5-4d7a-9d9d-151e9fa86750\") " pod="openshift-must-gather-jwttx/perf-node-gather-daemonset-h8vxp"
Sep 18 20:43:12.719735 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:43:12.719522    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xxm4j\" (UniqueName: \"kubernetes.io/projected/bfe376b6-04a5-4d7a-9d9d-151e9fa86750-kube-api-access-xxm4j\") pod \"perf-node-gather-daemonset-h8vxp\" (UID: \"bfe376b6-04a5-4d7a-9d9d-151e9fa86750\") " pod="openshift-must-gather-jwttx/perf-node-gather-daemonset-h8vxp"
Sep 18 20:43:12.719735 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:43:12.719545    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/bfe376b6-04a5-4d7a-9d9d-151e9fa86750-proc\") pod \"perf-node-gather-daemonset-h8vxp\" (UID: \"bfe376b6-04a5-4d7a-9d9d-151e9fa86750\") " pod="openshift-must-gather-jwttx/perf-node-gather-daemonset-h8vxp"
Sep 18 20:43:12.719735 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:43:12.719573    2532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/bfe376b6-04a5-4d7a-9d9d-151e9fa86750-sys\") pod \"perf-node-gather-daemonset-h8vxp\" (UID: \"bfe376b6-04a5-4d7a-9d9d-151e9fa86750\") " pod="openshift-must-gather-jwttx/perf-node-gather-daemonset-h8vxp"
Sep 18 20:43:12.820444 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:43:12.820389    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/bfe376b6-04a5-4d7a-9d9d-151e9fa86750-podres\") pod \"perf-node-gather-daemonset-h8vxp\" (UID: \"bfe376b6-04a5-4d7a-9d9d-151e9fa86750\") " pod="openshift-must-gather-jwttx/perf-node-gather-daemonset-h8vxp"
Sep 18 20:43:12.820673 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:43:12.820449    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/bfe376b6-04a5-4d7a-9d9d-151e9fa86750-lib-modules\") pod \"perf-node-gather-daemonset-h8vxp\" (UID: \"bfe376b6-04a5-4d7a-9d9d-151e9fa86750\") " pod="openshift-must-gather-jwttx/perf-node-gather-daemonset-h8vxp"
Sep 18 20:43:12.820673 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:43:12.820560    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xxm4j\" (UniqueName: \"kubernetes.io/projected/bfe376b6-04a5-4d7a-9d9d-151e9fa86750-kube-api-access-xxm4j\") pod \"perf-node-gather-daemonset-h8vxp\" (UID: \"bfe376b6-04a5-4d7a-9d9d-151e9fa86750\") " pod="openshift-must-gather-jwttx/perf-node-gather-daemonset-h8vxp"
Sep 18 20:43:12.820673 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:43:12.820580    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/bfe376b6-04a5-4d7a-9d9d-151e9fa86750-proc\") pod \"perf-node-gather-daemonset-h8vxp\" (UID: \"bfe376b6-04a5-4d7a-9d9d-151e9fa86750\") " pod="openshift-must-gather-jwttx/perf-node-gather-daemonset-h8vxp"
Sep 18 20:43:12.820673 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:43:12.820602    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/bfe376b6-04a5-4d7a-9d9d-151e9fa86750-podres\") pod \"perf-node-gather-daemonset-h8vxp\" (UID: \"bfe376b6-04a5-4d7a-9d9d-151e9fa86750\") " pod="openshift-must-gather-jwttx/perf-node-gather-daemonset-h8vxp"
Sep 18 20:43:12.820673 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:43:12.820611    2532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/bfe376b6-04a5-4d7a-9d9d-151e9fa86750-sys\") pod \"perf-node-gather-daemonset-h8vxp\" (UID: \"bfe376b6-04a5-4d7a-9d9d-151e9fa86750\") " pod="openshift-must-gather-jwttx/perf-node-gather-daemonset-h8vxp"
Sep 18 20:43:12.820673 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:43:12.820602    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/bfe376b6-04a5-4d7a-9d9d-151e9fa86750-lib-modules\") pod \"perf-node-gather-daemonset-h8vxp\" (UID: \"bfe376b6-04a5-4d7a-9d9d-151e9fa86750\") " pod="openshift-must-gather-jwttx/perf-node-gather-daemonset-h8vxp"
Sep 18 20:43:12.820888 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:43:12.820675    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/bfe376b6-04a5-4d7a-9d9d-151e9fa86750-sys\") pod \"perf-node-gather-daemonset-h8vxp\" (UID: \"bfe376b6-04a5-4d7a-9d9d-151e9fa86750\") " pod="openshift-must-gather-jwttx/perf-node-gather-daemonset-h8vxp"
Sep 18 20:43:12.820888 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:43:12.820673    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/bfe376b6-04a5-4d7a-9d9d-151e9fa86750-proc\") pod \"perf-node-gather-daemonset-h8vxp\" (UID: \"bfe376b6-04a5-4d7a-9d9d-151e9fa86750\") " pod="openshift-must-gather-jwttx/perf-node-gather-daemonset-h8vxp"
Sep 18 20:43:12.828329 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:43:12.828297    2532 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xxm4j\" (UniqueName: \"kubernetes.io/projected/bfe376b6-04a5-4d7a-9d9d-151e9fa86750-kube-api-access-xxm4j\") pod \"perf-node-gather-daemonset-h8vxp\" (UID: \"bfe376b6-04a5-4d7a-9d9d-151e9fa86750\") " pod="openshift-must-gather-jwttx/perf-node-gather-daemonset-h8vxp"
Sep 18 20:43:12.964481 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:43:12.964446    2532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-jwttx/perf-node-gather-daemonset-h8vxp"
Sep 18 20:43:13.038356 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:43:13.038279    2532 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-6594f4c75-5vxpb_3992f04b-9fe3-4dcd-bcbc-8baf2889081c/download-server/0.log"
Sep 18 20:43:13.237188 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:43:13.234480    2532 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-jwttx/perf-node-gather-daemonset-h8vxp"]
Sep 18 20:43:13.237923 ip-10-0-130-81 kubenswrapper[2532]: W0918 20:43:13.237776    2532 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podbfe376b6_04a5_4d7a_9d9d_151e9fa86750.slice/crio-7ebb7e051441ea4957ddd7eb7673266d782bc8f2b497106448eaff346a27aa18 WatchSource:0}: Error finding container 7ebb7e051441ea4957ddd7eb7673266d782bc8f2b497106448eaff346a27aa18: Status 404 returned error can't find the container with id 7ebb7e051441ea4957ddd7eb7673266d782bc8f2b497106448eaff346a27aa18
Sep 18 20:43:13.361863 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:43:13.361826    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-jwttx/perf-node-gather-daemonset-h8vxp" event={"ID":"bfe376b6-04a5-4d7a-9d9d-151e9fa86750","Type":"ContainerStarted","Data":"c8cff0998c828751889207c91f65f7f597ffccca354ed04d40ab2e61a5c5fc75"}
Sep 18 20:43:13.362053 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:43:13.361882    2532 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-jwttx/perf-node-gather-daemonset-h8vxp" event={"ID":"bfe376b6-04a5-4d7a-9d9d-151e9fa86750","Type":"ContainerStarted","Data":"7ebb7e051441ea4957ddd7eb7673266d782bc8f2b497106448eaff346a27aa18"}
Sep 18 20:43:13.362053 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:43:13.361928    2532 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-jwttx/perf-node-gather-daemonset-h8vxp"
Sep 18 20:43:13.379872 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:43:13.379822    2532 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-jwttx/perf-node-gather-daemonset-h8vxp" podStartSLOduration=1.379806946 podStartE2EDuration="1.379806946s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-18 20:43:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 20:43:13.377036795 +0000 UTC m=+2525.142426195" watchObservedRunningTime="2026-09-18 20:43:13.379806946 +0000 UTC m=+2525.145196344"
Sep 18 20:43:14.808720 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:43:14.808665    2532 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-l8bvt_13532b96-fcd8-4f12-a4df-b4041abe4cf3/dns/0.log"
Sep 18 20:43:14.829038 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:43:14.829012    2532 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-l8bvt_13532b96-fcd8-4f12-a4df-b4041abe4cf3/kube-rbac-proxy/0.log"
Sep 18 20:43:14.922722 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:43:14.922695    2532 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-b8jbz_87c4bfcc-4ad2-4166-b5c9-1051bdd2717d/dns-node-resolver/0.log"
Sep 18 20:43:15.508289 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:43:15.508256    2532 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-5546d96886-g8mqv_20b90eb2-bbca-49d7-b2f3-c8be70bc619e/registry/0.log"
Sep 18 20:43:15.574862 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:43:15.574821    2532 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-wkwnc_b771ff0e-edf5-4d81-adf7-6350fe1ea781/node-ca/0.log"
Sep 18 20:43:16.885104 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:43:16.885074    2532 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-66b8956864-v2z9s_a26ec002-d403-4adf-b6c9-e8e74c7c812f/kube-auth-proxy/0.log"
Sep 18 20:43:17.867878 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:43:17.867846    2532 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-qd64x_60bb026c-35d3-4f8b-bdb8-fb0b698c024c/serve-healthcheck-canary/0.log"
Sep 18 20:43:18.493631 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:43:18.493588    2532 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-vhbcf_c0ef2e11-9230-478a-982a-bae678c749b4/kube-rbac-proxy/0.log"
Sep 18 20:43:18.513827 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:43:18.513800    2532 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-vhbcf_c0ef2e11-9230-478a-982a-bae678c749b4/exporter/0.log"
Sep 18 20:43:18.534321 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:43:18.534264    2532 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-vhbcf_c0ef2e11-9230-478a-982a-bae678c749b4/extractor/0.log"
Sep 18 20:43:19.375946 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:43:19.375922    2532 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-jwttx/perf-node-gather-daemonset-h8vxp"
Sep 18 20:43:21.115205 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:43:21.115178    2532 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-668c7cd96d-6blck_e064dbb8-6962-4c10-a13f-5893654576f5/manager/0.log"
Sep 18 20:43:21.135143 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:43:21.135114    2532 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-668c7cd96d-6blck_e064dbb8-6962-4c10-a13f-5893654576f5/copy-manifests/0.log"
Sep 18 20:43:22.508709 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:43:22.508667    2532 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-76d6fbff56-qvklf_b0c74281-fc9b-4303-b135-41059fa4e970/manager/0.log"
Sep 18 20:43:22.938683 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:43:22.938610    2532 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-66b8956864-v2z9s_a26ec002-d403-4adf-b6c9-e8e74c7c812f/kube-auth-proxy/0.log"
Sep 18 20:43:24.924840 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:43:24.924818    2532 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-66b8956864-v2z9s_a26ec002-d403-4adf-b6c9-e8e74c7c812f/kube-auth-proxy/0.log"
Sep 18 20:43:29.241194 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:43:29.241165    2532 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-pn4rk_5c8c76fd-1219-40d0-b010-cd89d714a3fe/kube-multus-additional-cni-plugins/0.log"
Sep 18 20:43:29.266129 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:43:29.266102    2532 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-pn4rk_5c8c76fd-1219-40d0-b010-cd89d714a3fe/egress-router-binary-copy/0.log"
Sep 18 20:43:29.286699 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:43:29.286673    2532 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-pn4rk_5c8c76fd-1219-40d0-b010-cd89d714a3fe/cni-plugins/0.log"
Sep 18 20:43:29.309242 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:43:29.309212    2532 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-pn4rk_5c8c76fd-1219-40d0-b010-cd89d714a3fe/bond-cni-plugin/0.log"
Sep 18 20:43:29.329852 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:43:29.329826    2532 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-pn4rk_5c8c76fd-1219-40d0-b010-cd89d714a3fe/routeoverride-cni/0.log"
Sep 18 20:43:29.350385 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:43:29.350366    2532 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-pn4rk_5c8c76fd-1219-40d0-b010-cd89d714a3fe/whereabouts-cni-bincopy/0.log"
Sep 18 20:43:29.371263 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:43:29.371241    2532 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-pn4rk_5c8c76fd-1219-40d0-b010-cd89d714a3fe/whereabouts-cni/0.log"
Sep 18 20:43:29.412573 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:43:29.412546    2532 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-jbmm2_bf9010d7-90b5-4f95-8dea-8af5839c0f87/kube-multus/0.log"
Sep 18 20:43:29.521714 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:43:29.521638    2532 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-xx7f7_96167e3f-fb44-48e0-b86e-419a4a3b117c/network-metrics-daemon/0.log"
Sep 18 20:43:29.539708 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:43:29.539680    2532 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-xx7f7_96167e3f-fb44-48e0-b86e-419a4a3b117c/kube-rbac-proxy/0.log"
Sep 18 20:43:30.769244 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:43:30.769192    2532 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rtw9n_73fbf16b-3549-4a6c-a959-b633237ebd0c/ovn-controller/0.log"
Sep 18 20:43:30.785633 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:43:30.785605    2532 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rtw9n_73fbf16b-3549-4a6c-a959-b633237ebd0c/ovn-acl-logging/0.log"
Sep 18 20:43:30.785784 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:43:30.785642    2532 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rtw9n_73fbf16b-3549-4a6c-a959-b633237ebd0c/ovn-acl-logging/1.log"
Sep 18 20:43:30.807420 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:43:30.807390    2532 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rtw9n_73fbf16b-3549-4a6c-a959-b633237ebd0c/kube-rbac-proxy-node/0.log"
Sep 18 20:43:30.828080 ip-10-0-130-81 kubenswrapper[2532]: I0918 20:43:30.828054    2532 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rtw9n_73fbf16b-3549-4a6c-a959-b633237ebd0c/kube-rbac-proxy-ovn-metrics/0.log"