Sep 10 15:25:09.540561 ip-10-0-128-241 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Sep 10 15:25:09.540570 ip-10-0-128-241 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Sep 10 15:25:09.540577 ip-10-0-128-241 systemd[1]: kubelet.service: Failed with result 'resources'.
Sep 10 15:25:09.540721 ip-10-0-128-241 systemd[1]: Failed to start Kubernetes Kubelet.
Sep 10 15:25:19.686081 ip-10-0-128-241 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Sep 10 15:25:19.686099 ip-10-0-128-241 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot b2d872f40a5e433baacb927e5bc93c31 --
Sep 10 15:26:54.315542 ip-10-0-128-241 systemd[1]: Starting Kubernetes Kubelet...
Sep 10 15:26:54.726146 ip-10-0-128-241 kubenswrapper[2567]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 10 15:26:54.726146 ip-10-0-128-241 kubenswrapper[2567]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Sep 10 15:26:54.726146 ip-10-0-128-241 kubenswrapper[2567]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 10 15:26:54.726146 ip-10-0-128-241 kubenswrapper[2567]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Sep 10 15:26:54.726146 ip-10-0-128-241 kubenswrapper[2567]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 10 15:26:54.726964 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:54.726225    2567 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Sep 10 15:26:54.730618 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.730596    2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 10 15:26:54.730618 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.730614    2567 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 10 15:26:54.730618 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.730621    2567 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 10 15:26:54.730836 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.730626    2567 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 10 15:26:54.730836 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.730630    2567 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 10 15:26:54.730836 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.730634    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 10 15:26:54.730836 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.730638    2567 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 10 15:26:54.730836 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.730642    2567 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 10 15:26:54.730836 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.730646    2567 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 10 15:26:54.730836 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.730650    2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 10 15:26:54.730836 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.730653    2567 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 10 15:26:54.730836 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.730657    2567 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 10 15:26:54.730836 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.730661    2567 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 10 15:26:54.730836 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.730664    2567 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 10 15:26:54.730836 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.730668    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 10 15:26:54.730836 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.730672    2567 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 10 15:26:54.730836 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.730675    2567 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 10 15:26:54.730836 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.730679    2567 feature_gate.go:328] unrecognized feature gate: Example2
Sep 10 15:26:54.730836 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.730683    2567 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 10 15:26:54.730836 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.730686    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 10 15:26:54.730836 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.730690    2567 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 10 15:26:54.730836 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.730702    2567 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 10 15:26:54.730836 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.730706    2567 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 10 15:26:54.731682 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.730709    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 10 15:26:54.731682 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.730713    2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 10 15:26:54.731682 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.730717    2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 10 15:26:54.731682 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.730721    2567 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 10 15:26:54.731682 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.730725    2567 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 10 15:26:54.731682 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.730729    2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 10 15:26:54.731682 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.730732    2567 feature_gate.go:328] unrecognized feature gate: Example
Sep 10 15:26:54.731682 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.730736    2567 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 10 15:26:54.731682 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.730740    2567 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 10 15:26:54.731682 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.730744    2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 10 15:26:54.731682 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.730748    2567 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 10 15:26:54.731682 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.730752    2567 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 10 15:26:54.731682 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.730756    2567 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 10 15:26:54.731682 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.730761    2567 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 10 15:26:54.731682 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.730768    2567 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 10 15:26:54.731682 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.730776    2567 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 10 15:26:54.731682 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.730780    2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 10 15:26:54.731682 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.730785    2567 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 10 15:26:54.731682 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.730792    2567 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 10 15:26:54.732551 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.730797    2567 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 10 15:26:54.732551 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.730802    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 10 15:26:54.732551 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.730806    2567 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 10 15:26:54.732551 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.730810    2567 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 10 15:26:54.732551 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.730814    2567 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 10 15:26:54.732551 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.730819    2567 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 10 15:26:54.732551 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.730823    2567 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 10 15:26:54.732551 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.730827    2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 10 15:26:54.732551 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.730831    2567 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 10 15:26:54.732551 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.730835    2567 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 10 15:26:54.732551 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.730839    2567 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 10 15:26:54.732551 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.730843    2567 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 10 15:26:54.732551 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.730847    2567 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 10 15:26:54.732551 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.730852    2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 10 15:26:54.732551 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.730856    2567 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 10 15:26:54.732551 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.730860    2567 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 10 15:26:54.732551 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.730864    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 10 15:26:54.732551 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.730869    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 10 15:26:54.732551 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.730873    2567 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 10 15:26:54.732551 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.730878    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 10 15:26:54.733088 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.730882    2567 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 10 15:26:54.733088 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.730886    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 10 15:26:54.733088 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.730890    2567 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 10 15:26:54.733088 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.730895    2567 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 10 15:26:54.733088 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.730899    2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 10 15:26:54.733088 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.730903    2567 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 10 15:26:54.733088 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.730908    2567 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 10 15:26:54.733088 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.730912    2567 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 10 15:26:54.733088 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.730917    2567 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 10 15:26:54.733088 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.730923    2567 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 10 15:26:54.733088 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.730927    2567 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 10 15:26:54.733088 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.730931    2567 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 10 15:26:54.733088 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.730935    2567 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 10 15:26:54.733088 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.730940    2567 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 10 15:26:54.733088 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.730946    2567 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 10 15:26:54.733088 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.730952    2567 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 10 15:26:54.733088 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.730957    2567 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 10 15:26:54.733088 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.730961    2567 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 10 15:26:54.733088 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.730966    2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 10 15:26:54.733088 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.730970    2567 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 10 15:26:54.733606 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.730978    2567 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 10 15:26:54.733606 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.730984    2567 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 10 15:26:54.733606 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.730988    2567 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 10 15:26:54.733606 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.730993    2567 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 10 15:26:54.733606 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.730998    2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 10 15:26:54.733606 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.731003    2567 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 10 15:26:54.733606 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.731008    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 10 15:26:54.733606 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.731012    2567 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 10 15:26:54.733606 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.731017    2567 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 10 15:26:54.733606 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.731021    2567 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 10 15:26:54.733606 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.731026    2567 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 10 15:26:54.733606 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.731030    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 10 15:26:54.733606 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.731035    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 10 15:26:54.733606 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.731039    2567 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 10 15:26:54.733606 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.731043    2567 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 10 15:26:54.733606 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.731047    2567 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 10 15:26:54.733606 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.731052    2567 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 10 15:26:54.733606 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.732270    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 10 15:26:54.733606 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.732280    2567 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 10 15:26:54.734073 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.732285    2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 10 15:26:54.734073 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.732289    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 10 15:26:54.734073 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.732294    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 10 15:26:54.734073 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.732299    2567 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 10 15:26:54.734073 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.732307    2567 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 10 15:26:54.734073 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.732313    2567 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 10 15:26:54.734073 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.732318    2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 10 15:26:54.734073 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.732323    2567 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 10 15:26:54.734073 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.732327    2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 10 15:26:54.734073 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.732331    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 10 15:26:54.734073 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.732336    2567 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 10 15:26:54.734073 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.732340    2567 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 10 15:26:54.734073 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.732344    2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 10 15:26:54.734073 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.732348    2567 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 10 15:26:54.734073 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.732352    2567 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 10 15:26:54.734073 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.732356    2567 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 10 15:26:54.734073 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.732361    2567 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 10 15:26:54.734073 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.732365    2567 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 10 15:26:54.734073 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.732369    2567 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 10 15:26:54.734073 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.732373    2567 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 10 15:26:54.734598 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.732377    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 10 15:26:54.734598 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.732383    2567 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 10 15:26:54.734598 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.732389    2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 10 15:26:54.734598 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.732396    2567 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 10 15:26:54.734598 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.732401    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 10 15:26:54.734598 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.732406    2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 10 15:26:54.734598 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.732411    2567 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 10 15:26:54.734598 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.732415    2567 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 10 15:26:54.734598 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.732419    2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 10 15:26:54.734598 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.732424    2567 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 10 15:26:54.734598 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.732429    2567 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 10 15:26:54.734598 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.732435    2567 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 10 15:26:54.734598 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.732442    2567 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 10 15:26:54.734598 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.732446    2567 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 10 15:26:54.734598 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.732450    2567 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 10 15:26:54.734598 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.732455    2567 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 10 15:26:54.734598 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.732459    2567 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 10 15:26:54.734598 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.732463    2567 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 10 15:26:54.734598 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.732468    2567 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 10 15:26:54.735087 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.732472    2567 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 10 15:26:54.735087 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.732476    2567 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 10 15:26:54.735087 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.732481    2567 feature_gate.go:328] unrecognized feature gate: Example
Sep 10 15:26:54.735087 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.732485    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 10 15:26:54.735087 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.732489    2567 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 10 15:26:54.735087 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.732493    2567 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 10 15:26:54.735087 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.732497    2567 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 10 15:26:54.735087 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.732503    2567 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 10 15:26:54.735087 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.732507    2567 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 10 15:26:54.735087 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.732512    2567 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 10 15:26:54.735087 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.732516    2567 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 10 15:26:54.735087 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.732520    2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 10 15:26:54.735087 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.732524    2567 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 10 15:26:54.735087 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.732528    2567 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 10 15:26:54.735087 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.732533    2567 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 10 15:26:54.735087 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.732537    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 10 15:26:54.735087 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.732541    2567 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 10 15:26:54.735087 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.732545    2567 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 10 15:26:54.735087 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.732550    2567 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 10 15:26:54.735087 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.732553    2567 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 10 15:26:54.735639 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.732558    2567 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 10 15:26:54.735639 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.732562    2567 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 10 15:26:54.735639 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.732567    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 10 15:26:54.735639 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.732572    2567 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 10 15:26:54.735639 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.732577    2567 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 10 15:26:54.735639 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.732581    2567 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 10 15:26:54.735639 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.732586    2567 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 10 15:26:54.735639 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.732590    2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 10 15:26:54.735639 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.732594    2567 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 10 15:26:54.735639 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.732598    2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 10 15:26:54.735639 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.732603    2567 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 10 15:26:54.735639 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.732608    2567 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 10 15:26:54.735639 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.732612    2567 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 10 15:26:54.735639 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.732616    2567 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 10 15:26:54.735639 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.732620    2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 10 15:26:54.735639 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.732624    2567 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 10 15:26:54.735639 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.732628    2567 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 10 15:26:54.735639 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.732632    2567 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 10 15:26:54.735639 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.732636    2567 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 10 15:26:54.735639 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.732684    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 10 15:26:54.736146 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.732690    2567 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 10 15:26:54.736146 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.732702    2567 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 10 15:26:54.736146 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.732705    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 10 15:26:54.736146 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.732708    2567 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 10 15:26:54.736146 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.732711    2567 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 10 15:26:54.736146 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.732715    2567 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 10 15:26:54.736146 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.732720    2567 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 10 15:26:54.736146 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.732724    2567 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 10 15:26:54.736146 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.732728    2567 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 10 15:26:54.736146 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.732733    2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 10 15:26:54.736146 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.732736    2567 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 10 15:26:54.736146 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.732739    2567 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 10 15:26:54.736146 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.732741    2567 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 10 15:26:54.736146 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.732744    2567 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 10 15:26:54.736146 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.732746    2567 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 10 15:26:54.736146 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.732750    2567 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 10 15:26:54.736146 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.732752    2567 feature_gate.go:328] unrecognized feature gate: Example2
Sep 10 15:26:54.736146 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.732755    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 10 15:26:54.736146 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.733248    2567 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 10 15:26:54.736146 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.733255    2567 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 10 15:26:54.736670 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.733257    2567 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 10 15:26:54.736670 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.733260    2567 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 10 15:26:54.736670 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.733263    2567 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 10 15:26:54.736670 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.733266    2567 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 10 15:26:54.736670 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.733269    2567 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 10 15:26:54.736670 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.733274    2567 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 10 15:26:54.736670 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.733278    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 10 15:26:54.736670 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.733283    2567 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 10 15:26:54.736670 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.733287    2567 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 10 15:26:54.736670 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.733290    2567 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 10 15:26:54.736670 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.733293    2567 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 10 15:26:54.736670 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.733298    2567 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 10 15:26:54.736670 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.733300    2567 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 10 15:26:54.736670 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.733304    2567 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 10 15:26:54.736670 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.733306    2567 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 10 15:26:54.736670 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.733309    2567 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 10 15:26:54.736670 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.733312    2567 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 10 15:26:54.736670 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.733315    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 10 15:26:54.736670 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.733317    2567 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 10 15:26:54.736670 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.733320    2567 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 10 15:26:54.737148 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.733322    2567 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 10 15:26:54.737148 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.733325    2567 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 10 15:26:54.737148 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.733327    2567 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 10 15:26:54.737148 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.733330    2567 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 10 15:26:54.737148 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.733332    2567 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 10 15:26:54.737148 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.733337    2567 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 10 15:26:54.737148 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.733341    2567 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 10 15:26:54.737148 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.733345    2567 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 10 15:26:54.737148 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.733347    2567 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 10 15:26:54.737148 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.733351    2567 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 10 15:26:54.737148 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.733358    2567 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 10 15:26:54.737148 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.733363    2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 10 15:26:54.737148 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.733367    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 10 15:26:54.737148 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.733370    2567 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 10 15:26:54.737148 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.733373    2567 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 10 15:26:54.737148 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.733376    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 10 15:26:54.737148 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.733379    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 10 15:26:54.737148 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.733382    2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 10 15:26:54.737148 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.733385    2567 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 10 15:26:54.737636 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.733388    2567 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 10 15:26:54.737636 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.733390    2567 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 10 15:26:54.737636 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.733393    2567 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 10 15:26:54.737636 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.733397    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 10 15:26:54.737636 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.733400    2567 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 10 15:26:54.737636 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.733403    2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 10 15:26:54.737636 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.733406    2567 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 10 15:26:54.737636 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.733409    2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 10 15:26:54.737636 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.733412    2567 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 10 15:26:54.737636 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.733414    2567 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 10 15:26:54.737636 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.733417    2567 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 10 15:26:54.737636 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.733420    2567 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 10 15:26:54.737636 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.733422    2567 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 10 15:26:54.737636 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.733425    2567 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 10 15:26:54.737636 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.733428    2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 10 15:26:54.737636 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.733432    2567 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 10 15:26:54.737636 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.733436    2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 10 15:26:54.737636 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.733440    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 10 15:26:54.737636 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.733444    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 10 15:26:54.737636 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.733446    2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 10 15:26:54.738212 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.733449    2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 10 15:26:54.738212 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.733451    2567 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 10 15:26:54.738212 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.733454    2567 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 10 15:26:54.738212 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.733456    2567 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 10 15:26:54.738212 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.733459    2567 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 10 15:26:54.738212 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.733461    2567 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 10 15:26:54.738212 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.733464    2567 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 10 15:26:54.738212 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.733466    2567 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 10 15:26:54.738212 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.733469    2567 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 10 15:26:54.738212 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.733471    2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 10 15:26:54.738212 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.733474    2567 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 10 15:26:54.738212 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.733476    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 10 15:26:54.738212 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.733480    2567 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 10 15:26:54.738212 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.733484    2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 10 15:26:54.738212 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.733487    2567 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 10 15:26:54.738212 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.733490    2567 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 10 15:26:54.738212 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.733492    2567 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 10 15:26:54.738212 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.733495    2567 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 10 15:26:54.738212 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.733499    2567 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 10 15:26:54.738212 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.733501    2567 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 10 15:26:54.738760 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.733503    2567 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 10 15:26:54.738760 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.733506    2567 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 10 15:26:54.738760 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.733511    2567 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 10 15:26:54.738760 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.733514    2567 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 10 15:26:54.738760 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.733518    2567 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 10 15:26:54.738760 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.733522    2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 10 15:26:54.738760 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.733525    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 10 15:26:54.738760 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.733527    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 10 15:26:54.738760 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.733530    2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 10 15:26:54.738760 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.733532    2567 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 10 15:26:54.738760 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.733542    2567 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 10 15:26:54.738760 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.733545    2567 feature_gate.go:328] unrecognized feature gate: Example
Sep 10 15:26:54.738760 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.733547    2567 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 10 15:26:54.738760 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.733550    2567 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 10 15:26:54.738760 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.733552    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 10 15:26:54.738760 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.733555    2567 feature_gate.go:328] unrecognized feature gate: Example2
Sep 10 15:26:54.738760 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.733558    2567 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 10 15:26:54.738760 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.733560    2567 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 10 15:26:54.738760 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:54.733642    2567 flags.go:64] FLAG: --address="0.0.0.0"
Sep 10 15:26:54.738760 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:54.733685    2567 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Sep 10 15:26:54.738760 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:54.733693    2567 flags.go:64] FLAG: --anonymous-auth="true"
Sep 10 15:26:54.739287 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:54.733698    2567 flags.go:64] FLAG: --application-metrics-count-limit="100"
Sep 10 15:26:54.739287 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:54.733703    2567 flags.go:64] FLAG: --authentication-token-webhook="false"
Sep 10 15:26:54.739287 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:54.733706    2567 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Sep 10 15:26:54.739287 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:54.733710    2567 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Sep 10 15:26:54.739287 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:54.733715    2567 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Sep 10 15:26:54.739287 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:54.733718    2567 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Sep 10 15:26:54.739287 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:54.733721    2567 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Sep 10 15:26:54.739287 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:54.733725    2567 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Sep 10 15:26:54.739287 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:54.733729    2567 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Sep 10 15:26:54.739287 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:54.733732    2567 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Sep 10 15:26:54.739287 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:54.733736    2567 flags.go:64] FLAG: --cgroup-root=""
Sep 10 15:26:54.739287 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:54.733739    2567 flags.go:64] FLAG: --cgroups-per-qos="true"
Sep 10 15:26:54.739287 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:54.733742    2567 flags.go:64] FLAG: --client-ca-file=""
Sep 10 15:26:54.739287 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:54.733746    2567 flags.go:64] FLAG: --cloud-provider="external"
Sep 10 15:26:54.739287 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:54.733751    2567 flags.go:64] FLAG: --cluster-dns="[]"
Sep 10 15:26:54.739287 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:54.733757    2567 flags.go:64] FLAG: --cluster-domain=""
Sep 10 15:26:54.739287 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:54.733762    2567 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Sep 10 15:26:54.739287 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:54.733766    2567 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Sep 10 15:26:54.739287 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:54.733769    2567 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Sep 10 15:26:54.739287 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:54.733772    2567 flags.go:64] FLAG: --container-log-max-files="5"
Sep 10 15:26:54.739287 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:54.733776    2567 flags.go:64] FLAG: --container-log-max-size="10Mi"
Sep 10 15:26:54.739287 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:54.733779    2567 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Sep 10 15:26:54.739287 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:54.733782    2567 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Sep 10 15:26:54.739827 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:54.733786    2567 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Sep 10 15:26:54.739827 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:54.733788    2567 flags.go:64] FLAG: --contention-profiling="false"
Sep 10 15:26:54.739827 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:54.733791    2567 flags.go:64] FLAG: --cpu-cfs-quota="true"
Sep 10 15:26:54.739827 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:54.733794    2567 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Sep 10 15:26:54.739827 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:54.733798    2567 flags.go:64] FLAG: --cpu-manager-policy="none"
Sep 10 15:26:54.739827 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:54.733800    2567 flags.go:64] FLAG: --cpu-manager-policy-options=""
Sep 10 15:26:54.739827 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:54.733805    2567 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Sep 10 15:26:54.739827 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:54.733808    2567 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Sep 10 15:26:54.739827 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:54.733811    2567 flags.go:64] FLAG: --enable-debugging-handlers="true"
Sep 10 15:26:54.739827 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:54.733814    2567 flags.go:64] FLAG: --enable-load-reader="false"
Sep 10 15:26:54.739827 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:54.733817    2567 flags.go:64] FLAG: --enable-server="true"
Sep 10 15:26:54.739827 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:54.733821    2567 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Sep 10 15:26:54.739827 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:54.733827    2567 flags.go:64] FLAG: --event-burst="100"
Sep 10 15:26:54.739827 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:54.733832    2567 flags.go:64] FLAG: --event-qps="50"
Sep 10 15:26:54.739827 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:54.733837    2567 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Sep 10 15:26:54.739827 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:54.733842    2567 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Sep 10 15:26:54.739827 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:54.733847    2567 flags.go:64] FLAG: --eviction-hard=""
Sep 10 15:26:54.739827 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:54.733851    2567 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Sep 10 15:26:54.739827 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:54.733854    2567 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Sep 10 15:26:54.739827 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:54.733857    2567 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Sep 10 15:26:54.739827 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:54.733861    2567 flags.go:64] FLAG: --eviction-soft=""
Sep 10 15:26:54.739827 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:54.733864    2567 flags.go:64] FLAG: --eviction-soft-grace-period=""
Sep 10 15:26:54.739827 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:54.733867    2567 flags.go:64] FLAG: --exit-on-lock-contention="false"
Sep 10 15:26:54.739827 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:54.733870    2567 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Sep 10 15:26:54.739827 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:54.733873    2567 flags.go:64] FLAG: --experimental-mounter-path=""
Sep 10 15:26:54.740544 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:54.733876    2567 flags.go:64] FLAG: --fail-cgroupv1="false"
Sep 10 15:26:54.740544 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:54.733879    2567 flags.go:64] FLAG: --fail-swap-on="true"
Sep 10 15:26:54.740544 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:54.733882    2567 flags.go:64] FLAG: --feature-gates=""
Sep 10 15:26:54.740544 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:54.733885    2567 flags.go:64] FLAG: --file-check-frequency="20s"
Sep 10 15:26:54.740544 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:54.733888    2567 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Sep 10 15:26:54.740544 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:54.733891    2567 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Sep 10 15:26:54.740544 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:54.733894    2567 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Sep 10 15:26:54.740544 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:54.733897    2567 flags.go:64] FLAG: --healthz-port="10248"
Sep 10 15:26:54.740544 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:54.733900    2567 flags.go:64] FLAG: --help="false"
Sep 10 15:26:54.740544 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:54.733903    2567 flags.go:64] FLAG: --hostname-override="ip-10-0-128-241.ec2.internal"
Sep 10 15:26:54.740544 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:54.733908    2567 flags.go:64] FLAG: --housekeeping-interval="10s"
Sep 10 15:26:54.740544 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:54.733913    2567 flags.go:64] FLAG: --http-check-frequency="20s"
Sep 10 15:26:54.740544 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:54.733918    2567 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Sep 10 15:26:54.740544 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:54.733923    2567 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Sep 10 15:26:54.740544 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:54.733926    2567 flags.go:64] FLAG: --image-gc-high-threshold="85"
Sep 10 15:26:54.740544 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:54.733929    2567 flags.go:64] FLAG: --image-gc-low-threshold="80"
Sep 10 15:26:54.740544 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:54.733931    2567 flags.go:64] FLAG: --image-service-endpoint=""
Sep 10 15:26:54.740544 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:54.733934    2567 flags.go:64] FLAG: --kernel-memcg-notification="false"
Sep 10 15:26:54.740544 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:54.733937    2567 flags.go:64] FLAG: --kube-api-burst="100"
Sep 10 15:26:54.740544 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:54.733940    2567 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Sep 10 15:26:54.740544 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:54.733943    2567 flags.go:64] FLAG: --kube-api-qps="50"
Sep 10 15:26:54.740544 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:54.733946    2567 flags.go:64] FLAG: --kube-reserved=""
Sep 10 15:26:54.740544 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:54.733949    2567 flags.go:64] FLAG: --kube-reserved-cgroup=""
Sep 10 15:26:54.740544 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:54.733952    2567 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Sep 10 15:26:54.741151 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:54.733955    2567 flags.go:64] FLAG: --kubelet-cgroups=""
Sep 10 15:26:54.741151 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:54.733958    2567 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Sep 10 15:26:54.741151 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:54.733961    2567 flags.go:64] FLAG: --lock-file=""
Sep 10 15:26:54.741151 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:54.733964    2567 flags.go:64] FLAG: --log-cadvisor-usage="false"
Sep 10 15:26:54.741151 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:54.733968    2567 flags.go:64] FLAG: --log-flush-frequency="5s"
Sep 10 15:26:54.741151 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:54.733971    2567 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Sep 10 15:26:54.741151 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:54.733977    2567 flags.go:64] FLAG: --log-json-split-stream="false"
Sep 10 15:26:54.741151 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:54.733980    2567 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Sep 10 15:26:54.741151 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:54.733983    2567 flags.go:64] FLAG: --log-text-split-stream="false"
Sep 10 15:26:54.741151 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:54.733988    2567 flags.go:64] FLAG: --logging-format="text"
Sep 10 15:26:54.741151 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:54.733993    2567 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Sep 10 15:26:54.741151 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:54.733998    2567 flags.go:64] FLAG: --make-iptables-util-chains="true"
Sep 10 15:26:54.741151 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:54.734001    2567 flags.go:64] FLAG: --manifest-url=""
Sep 10 15:26:54.741151 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:54.734004    2567 flags.go:64] FLAG: --manifest-url-header=""
Sep 10 15:26:54.741151 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:54.734009    2567 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Sep 10 15:26:54.741151 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:54.734012    2567 flags.go:64] FLAG: --max-open-files="1000000"
Sep 10 15:26:54.741151 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:54.734020    2567 flags.go:64] FLAG: --max-pods="110"
Sep 10 15:26:54.741151 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:54.734023    2567 flags.go:64] FLAG: --maximum-dead-containers="-1"
Sep 10 15:26:54.741151 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:54.734026    2567 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Sep 10 15:26:54.741151 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:54.734029    2567 flags.go:64] FLAG: --memory-manager-policy="None"
Sep 10 15:26:54.741151 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:54.734032    2567 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Sep 10 15:26:54.741151 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:54.734035    2567 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Sep 10 15:26:54.741151 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:54.734038    2567 flags.go:64] FLAG: --node-ip="0.0.0.0"
Sep 10 15:26:54.741151 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:54.734041    2567 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Sep 10 15:26:54.741151 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:54.734048    2567 flags.go:64] FLAG: --node-status-max-images="50"
Sep 10 15:26:54.741791 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:54.734051    2567 flags.go:64] FLAG: --node-status-update-frequency="10s"
Sep 10 15:26:54.741791 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:54.734054    2567 flags.go:64] FLAG: --oom-score-adj="-999"
Sep 10 15:26:54.741791 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:54.734058    2567 flags.go:64] FLAG: --pod-cidr=""
Sep 10 15:26:54.741791 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:54.734061    2567 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8ccbacc999dbf9abad8f4009240188312b683f36ced4763eed166c1500c164b1"
Sep 10 15:26:54.741791 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:54.734068    2567 flags.go:64] FLAG: --pod-manifest-path=""
Sep 10 15:26:54.741791 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:54.734077    2567 flags.go:64] FLAG: --pod-max-pids="-1"
Sep 10 15:26:54.741791 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:54.734081    2567 flags.go:64] FLAG: --pods-per-core="0"
Sep 10 15:26:54.741791 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:54.734085    2567 flags.go:64] FLAG: --port="10250"
Sep 10 15:26:54.741791 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:54.734088    2567 flags.go:64] FLAG: --protect-kernel-defaults="false"
Sep 10 15:26:54.741791 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:54.734091    2567 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-010a0b897ade1290f"
Sep 10 15:26:54.741791 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:54.734094    2567 flags.go:64] FLAG: --qos-reserved=""
Sep 10 15:26:54.741791 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:54.734097    2567 flags.go:64] FLAG: --read-only-port="10255"
Sep 10 15:26:54.741791 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:54.734101    2567 flags.go:64] FLAG: --register-node="true"
Sep 10 15:26:54.741791 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:54.734104    2567 flags.go:64] FLAG: --register-with-taints=""
Sep 10 15:26:54.741791 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:54.734108    2567 flags.go:64] FLAG: --registry-burst="10"
Sep 10 15:26:54.741791 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:54.734111    2567 flags.go:64] FLAG: --registry-qps="5"
Sep 10 15:26:54.741791 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:54.734113    2567 flags.go:64] FLAG: --reserved-cpus=""
Sep 10 15:26:54.741791 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:54.734116    2567 flags.go:64] FLAG: --reserved-memory=""
Sep 10 15:26:54.741791 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:54.734120    2567 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Sep 10 15:26:54.741791 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:54.734123    2567 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Sep 10 15:26:54.741791 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:54.734126    2567 flags.go:64] FLAG: --rotate-certificates="false"
Sep 10 15:26:54.741791 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:54.734129    2567 flags.go:64] FLAG: --rotate-server-certificates="false"
Sep 10 15:26:54.741791 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:54.734132    2567 flags.go:64] FLAG: --runonce="false"
Sep 10 15:26:54.741791 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:54.734135    2567 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Sep 10 15:26:54.741791 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:54.734138    2567 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Sep 10 15:26:54.742415 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:54.734142    2567 flags.go:64] FLAG: --seccomp-default="false"
Sep 10 15:26:54.742415 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:54.734147    2567 flags.go:64] FLAG: --serialize-image-pulls="true"
Sep 10 15:26:54.742415 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:54.734151    2567 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Sep 10 15:26:54.742415 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:54.734156    2567 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Sep 10 15:26:54.742415 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:54.734159    2567 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Sep 10 15:26:54.742415 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:54.734162    2567 flags.go:64] FLAG: --storage-driver-password="root"
Sep 10 15:26:54.742415 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:54.734165    2567 flags.go:64] FLAG: --storage-driver-secure="false"
Sep 10 15:26:54.742415 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:54.734168    2567 flags.go:64] FLAG: --storage-driver-table="stats"
Sep 10 15:26:54.742415 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:54.734171    2567 flags.go:64] FLAG: --storage-driver-user="root"
Sep 10 15:26:54.742415 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:54.734174    2567 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Sep 10 15:26:54.742415 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:54.734177    2567 flags.go:64] FLAG: --sync-frequency="1m0s"
Sep 10 15:26:54.742415 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:54.734179    2567 flags.go:64] FLAG: --system-cgroups=""
Sep 10 15:26:54.742415 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:54.734184    2567 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Sep 10 15:26:54.742415 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:54.734189    2567 flags.go:64] FLAG: --system-reserved-cgroup=""
Sep 10 15:26:54.742415 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:54.734192    2567 flags.go:64] FLAG: --tls-cert-file=""
Sep 10 15:26:54.742415 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:54.734195    2567 flags.go:64] FLAG: --tls-cipher-suites="[]"
Sep 10 15:26:54.742415 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:54.734199    2567 flags.go:64] FLAG: --tls-min-version=""
Sep 10 15:26:54.742415 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:54.734202    2567 flags.go:64] FLAG: --tls-private-key-file=""
Sep 10 15:26:54.742415 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:54.734204    2567 flags.go:64] FLAG: --topology-manager-policy="none"
Sep 10 15:26:54.742415 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:54.734207    2567 flags.go:64] FLAG: --topology-manager-policy-options=""
Sep 10 15:26:54.742415 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:54.734211    2567 flags.go:64] FLAG: --topology-manager-scope="container"
Sep 10 15:26:54.742415 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:54.734214    2567 flags.go:64] FLAG: --v="2"
Sep 10 15:26:54.742415 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:54.734218    2567 flags.go:64] FLAG: --version="false"
Sep 10 15:26:54.742415 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:54.734224    2567 flags.go:64] FLAG: --vmodule=""
Sep 10 15:26:54.742415 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:54.734241    2567 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Sep 10 15:26:54.743034 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:54.734247    2567 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Sep 10 15:26:54.743034 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.734368    2567 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 10 15:26:54.743034 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.734372    2567 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 10 15:26:54.743034 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.734375    2567 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 10 15:26:54.743034 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.734378    2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 10 15:26:54.743034 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.734380    2567 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 10 15:26:54.743034 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.734383    2567 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 10 15:26:54.743034 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.734386    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 10 15:26:54.743034 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.734388    2567 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 10 15:26:54.743034 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.734391    2567 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 10 15:26:54.743034 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.734397    2567 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 10 15:26:54.743034 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.734403    2567 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 10 15:26:54.743034 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.734408    2567 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 10 15:26:54.743034 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.734412    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 10 15:26:54.743034 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.734414    2567 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 10 15:26:54.743034 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.734417    2567 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 10 15:26:54.743034 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.734419    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 10 15:26:54.743034 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.734422    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 10 15:26:54.743034 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.734425    2567 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 10 15:26:54.743034 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.734430    2567 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 10 15:26:54.743550 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.734433    2567 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 10 15:26:54.743550 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.734436    2567 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 10 15:26:54.743550 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.734438    2567 feature_gate.go:328] unrecognized feature gate: Example2
Sep 10 15:26:54.743550 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.734441    2567 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 10 15:26:54.743550 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.734444    2567 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 10 15:26:54.743550 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.734446    2567 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 10 15:26:54.743550 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.734449    2567 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 10 15:26:54.743550 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.734452    2567 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 10 15:26:54.743550 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.734457    2567 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 10 15:26:54.743550 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.734459    2567 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 10 15:26:54.743550 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.734462    2567 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 10 15:26:54.743550 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.734465    2567 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 10 15:26:54.743550 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.734468    2567 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 10 15:26:54.743550 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.734472    2567 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 10 15:26:54.743550 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.734476    2567 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 10 15:26:54.743550 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.734480    2567 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 10 15:26:54.743550 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.734484    2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 10 15:26:54.743550 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.734488    2567 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 10 15:26:54.743550 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.734491    2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 10 15:26:54.743550 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.734493    2567 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 10 15:26:54.744037 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.734496    2567 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 10 15:26:54.744037 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.734499    2567 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 10 15:26:54.744037 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.734501    2567 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 10 15:26:54.744037 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.734504    2567 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 10 15:26:54.744037 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.734507    2567 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 10 15:26:54.744037 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.734509    2567 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 10 15:26:54.744037 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.734512    2567 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 10 15:26:54.744037 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.734514    2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 10 15:26:54.744037 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.734517    2567 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 10 15:26:54.744037 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.734519    2567 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 10 15:26:54.744037 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.734522    2567 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 10 15:26:54.744037 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.734526    2567 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 10 15:26:54.744037 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.734529    2567 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 10 15:26:54.744037 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.734532    2567 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 10 15:26:54.744037 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.734536    2567 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 10 15:26:54.744037 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.734540    2567 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 10 15:26:54.744037 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.734543    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 10 15:26:54.744037 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.734545    2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 10 15:26:54.744037 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.734548    2567 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 10 15:26:54.744519 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.734551    2567 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 10 15:26:54.744519 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.734556    2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 10 15:26:54.744519 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.734561    2567 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 10 15:26:54.744519 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.734565    2567 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 10 15:26:54.744519 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.734568    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 10 15:26:54.744519 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.734571    2567 feature_gate.go:328] unrecognized feature gate: Example
Sep 10 15:26:54.744519 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.734573    2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 10 15:26:54.744519 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.734576    2567 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 10 15:26:54.744519 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.734578    2567 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 10 15:26:54.744519 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.734580    2567 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 10 15:26:54.744519 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.734583    2567 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 10 15:26:54.744519 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.734585    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 10 15:26:54.744519 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.734588    2567 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 10 15:26:54.744519 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.734590    2567 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 10 15:26:54.744519 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.734593    2567 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 10 15:26:54.744519 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.734596    2567 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 10 15:26:54.744519 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.734598    2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 10 15:26:54.744519 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.734601    2567 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 10 15:26:54.744519 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.734604    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 10 15:26:54.744519 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.734607    2567 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 10 15:26:54.745326 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.734610    2567 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 10 15:26:54.745326 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.734612    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 10 15:26:54.745326 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.734615    2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 10 15:26:54.745326 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.734618    2567 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 10 15:26:54.745326 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.734623    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 10 15:26:54.745326 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.734626    2567 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 10 15:26:54.745326 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.734628    2567 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 10 15:26:54.745326 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.734632    2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 10 15:26:54.745326 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.734637    2567 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 10 15:26:54.745326 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.734642    2567 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 10 15:26:54.745326 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.734647    2567 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 10 15:26:54.745326 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.734650    2567 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 10 15:26:54.745326 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.734653    2567 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 10 15:26:54.745326 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.734655    2567 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 10 15:26:54.745326 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.734658    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 10 15:26:54.745326 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.734661    2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 10 15:26:54.745326 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.734664    2567 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 10 15:26:54.745326 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.734667    2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 10 15:26:54.745326 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.734669    2567 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 10 15:26:54.745326 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.734672    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 10 15:26:54.745921 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.734674    2567 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 10 15:26:54.745921 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:54.734680    2567 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 10 15:26:54.745921 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:54.741799    2567 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Sep 10 15:26:54.745921 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:54.741815    2567 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Sep 10 15:26:54.745921 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:54.742834    2567 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Sep 10 15:26:54.745921 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:54.742843    2567 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Sep 10 15:26:54.745921 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.742901    2567 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 10 15:26:54.745921 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.742907    2567 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 10 15:26:54.745921 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.742910    2567 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 10 15:26:54.745921 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.742914    2567 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 10 15:26:54.745921 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.742917    2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 10 15:26:54.745921 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.742920    2567 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 10 15:26:54.745921 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.742923    2567 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 10 15:26:54.745921 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.742926    2567 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 10 15:26:54.745921 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.742930    2567 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 10 15:26:54.746333 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.742932    2567 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 10 15:26:54.746333 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.742935    2567 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 10 15:26:54.746333 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.742938    2567 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 10 15:26:54.746333 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.742941    2567 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 10 15:26:54.746333 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.742945    2567 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 10 15:26:54.746333 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.742948    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 10 15:26:54.746333 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.742951    2567 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 10 15:26:54.746333 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.742954    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 10 15:26:54.746333 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.742957    2567 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 10 15:26:54.746333 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.742961    2567 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 10 15:26:54.746333 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.742965    2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 10 15:26:54.746333 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.742968    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 10 15:26:54.746333 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.742971    2567 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 10 15:26:54.746333 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.742974    2567 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 10 15:26:54.746333 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.742977    2567 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 10 15:26:54.746333 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.742980    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 10 15:26:54.746333 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.742983    2567 feature_gate.go:328] unrecognized feature gate: Example
Sep 10 15:26:54.746333 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.742986    2567 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 10 15:26:54.746333 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.742988    2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 10 15:26:54.746333 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.742991    2567 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 10 15:26:54.746808 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.742996    2567 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 10 15:26:54.746808 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.742999    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 10 15:26:54.746808 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.743002    2567 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 10 15:26:54.746808 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.743004    2567 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 10 15:26:54.746808 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.743007    2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 10 15:26:54.746808 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.743010    2567 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 10 15:26:54.746808 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.743013    2567 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 10 15:26:54.746808 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.743015    2567 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 10 15:26:54.746808 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.743018    2567 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 10 15:26:54.746808 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.743021    2567 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 10 15:26:54.746808 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.743024    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 10 15:26:54.746808 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.743027    2567 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 10 15:26:54.746808 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.743030    2567 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 10 15:26:54.746808 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.743034    2567 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 10 15:26:54.746808 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.743037    2567 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 10 15:26:54.746808 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.743040    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 10 15:26:54.746808 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.743043    2567 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 10 15:26:54.746808 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.743047    2567 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 10 15:26:54.746808 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.743050    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 10 15:26:54.747383 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.743053    2567 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 10 15:26:54.747383 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.743059    2567 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 10 15:26:54.747383 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.743062    2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 10 15:26:54.747383 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.743064    2567 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 10 15:26:54.747383 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.743067    2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 10 15:26:54.747383 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.743070    2567 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 10 15:26:54.747383 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.743072    2567 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 10 15:26:54.747383 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.743075    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 10 15:26:54.747383 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.743078    2567 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 10 15:26:54.747383 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.743080    2567 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 10 15:26:54.747383 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.743083    2567 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 10 15:26:54.747383 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.743085    2567 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 10 15:26:54.747383 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.743089    2567 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 10 15:26:54.747383 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.743092    2567 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 10 15:26:54.747383 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.743096    2567 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 10 15:26:54.747383 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.743098    2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 10 15:26:54.747383 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.743101    2567 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 10 15:26:54.747383 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.743104    2567 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 10 15:26:54.747383 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.743107    2567 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 10 15:26:54.747383 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.743109    2567 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 10 15:26:54.747892 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.743112    2567 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 10 15:26:54.747892 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.743115    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 10 15:26:54.747892 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.743117    2567 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 10 15:26:54.747892 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.743120    2567 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 10 15:26:54.747892 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.743122    2567 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 10 15:26:54.747892 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.743125    2567 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 10 15:26:54.747892 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.743128    2567 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 10 15:26:54.747892 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.743130    2567 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 10 15:26:54.747892 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.743133    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 10 15:26:54.747892 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.743135    2567 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 10 15:26:54.747892 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.743138    2567 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 10 15:26:54.747892 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.743141    2567 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 10 15:26:54.747892 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.743143    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 10 15:26:54.747892 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.743145    2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 10 15:26:54.747892 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.743148    2567 feature_gate.go:328] unrecognized feature gate: Example2
Sep 10 15:26:54.747892 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.743150    2567 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 10 15:26:54.747892 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.743153    2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 10 15:26:54.747892 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.743155    2567 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 10 15:26:54.747892 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.743158    2567 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 10 15:26:54.747892 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.743160    2567 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 10 15:26:54.748387 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.743163    2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 10 15:26:54.748387 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.743167    2567 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 10 15:26:54.748387 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.743171    2567 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 10 15:26:54.748387 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.743174    2567 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 10 15:26:54.748387 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.743177    2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 10 15:26:54.748387 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.743180    2567 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 10 15:26:54.748387 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.743182    2567 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 10 15:26:54.748387 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.743186    2567 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 10 15:26:54.748387 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.743188    2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 10 15:26:54.748387 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.743191    2567 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 10 15:26:54.748387 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.743193    2567 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 10 15:26:54.748387 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:54.743198    2567 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 10 15:26:54.748387 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.743329    2567 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 10 15:26:54.748387 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.743335    2567 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 10 15:26:54.748783 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.743339    2567 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 10 15:26:54.748783 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.743342    2567 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 10 15:26:54.748783 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.743345    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 10 15:26:54.748783 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.743347    2567 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 10 15:26:54.748783 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.743350    2567 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 10 15:26:54.748783 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.743352    2567 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 10 15:26:54.748783 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.743355    2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 10 15:26:54.748783 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.743358    2567 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 10 15:26:54.748783 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.743360    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 10 15:26:54.748783 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.743363    2567 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 10 15:26:54.748783 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.743365    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 10 15:26:54.748783 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.743368    2567 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 10 15:26:54.748783 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.743371    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 10 15:26:54.748783 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.743374    2567 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 10 15:26:54.748783 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.743377    2567 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 10 15:26:54.748783 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.743379    2567 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 10 15:26:54.748783 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.743381    2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 10 15:26:54.748783 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.743384    2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 10 15:26:54.748783 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.743386    2567 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 10 15:26:54.748783 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.743389    2567 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 10 15:26:54.749289 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.743392    2567 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 10 15:26:54.749289 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.743394    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 10 15:26:54.749289 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.743397    2567 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 10 15:26:54.749289 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.743399    2567 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 10 15:26:54.749289 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.743402    2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 10 15:26:54.749289 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.743404    2567 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 10 15:26:54.749289 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.743407    2567 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 10 15:26:54.749289 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.743410    2567 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 10 15:26:54.749289 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.743413    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 10 15:26:54.749289 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.743415    2567 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 10 15:26:54.749289 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.743418    2567 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 10 15:26:54.749289 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.743420    2567 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 10 15:26:54.749289 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.743422    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 10 15:26:54.749289 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.743425    2567 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 10 15:26:54.749289 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.743428    2567 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 10 15:26:54.749289 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.743431    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 10 15:26:54.749289 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.743433    2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 10 15:26:54.749289 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.743435    2567 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 10 15:26:54.749289 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.743438    2567 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 10 15:26:54.749289 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.743440    2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 10 15:26:54.749796 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.743443    2567 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 10 15:26:54.749796 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.743446    2567 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 10 15:26:54.749796 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.743448    2567 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 10 15:26:54.749796 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.743451    2567 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 10 15:26:54.749796 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.743454    2567 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 10 15:26:54.749796 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.743456    2567 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 10 15:26:54.749796 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.743459    2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 10 15:26:54.749796 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.743461    2567 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 10 15:26:54.749796 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.743463    2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 10 15:26:54.749796 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.743466    2567 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 10 15:26:54.749796 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.743469    2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 10 15:26:54.749796 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.743471    2567 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 10 15:26:54.749796 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.743475    2567 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 10 15:26:54.749796 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.743478    2567 feature_gate.go:328] unrecognized feature gate: Example
Sep 10 15:26:54.749796 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.743481    2567 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 10 15:26:54.749796 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.743484    2567 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 10 15:26:54.749796 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.743487    2567 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 10 15:26:54.749796 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.743490    2567 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 10 15:26:54.749796 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.743492    2567 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 10 15:26:54.749796 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.743495    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 10 15:26:54.750299 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.743498    2567 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 10 15:26:54.750299 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.743500    2567 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 10 15:26:54.750299 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.743503    2567 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 10 15:26:54.750299 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.743505    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 10 15:26:54.750299 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.743508    2567 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 10 15:26:54.750299 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.743510    2567 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 10 15:26:54.750299 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.743513    2567 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 10 15:26:54.750299 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.743515    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 10 15:26:54.750299 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.743518    2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 10 15:26:54.750299 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.743520    2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 10 15:26:54.750299 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.743522    2567 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 10 15:26:54.750299 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.743525    2567 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 10 15:26:54.750299 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.743527    2567 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 10 15:26:54.750299 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.743530    2567 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 10 15:26:54.750299 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.743532    2567 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 10 15:26:54.750299 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.743535    2567 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 10 15:26:54.750299 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.743537    2567 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 10 15:26:54.750299 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.743540    2567 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 10 15:26:54.750299 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.743543    2567 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 10 15:26:54.750299 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.743545    2567 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 10 15:26:54.750795 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.743548    2567 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 10 15:26:54.750795 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.743550    2567 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 10 15:26:54.750795 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.743553    2567 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 10 15:26:54.750795 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.743556    2567 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 10 15:26:54.750795 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.743558    2567 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 10 15:26:54.750795 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.743561    2567 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 10 15:26:54.750795 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.743564    2567 feature_gate.go:328] unrecognized feature gate: Example2
Sep 10 15:26:54.750795 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.743568    2567 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 10 15:26:54.750795 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.743572    2567 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 10 15:26:54.750795 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.743575    2567 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 10 15:26:54.750795 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.743578    2567 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 10 15:26:54.750795 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.743581    2567 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 10 15:26:54.750795 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.743585    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 10 15:26:54.750795 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.743587    2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 10 15:26:54.750795 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.743590    2567 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 10 15:26:54.750795 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.743593    2567 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 10 15:26:54.750795 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:54.743596    2567 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 10 15:26:54.751214 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:54.743600    2567 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 10 15:26:54.751214 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:54.743714    2567 server.go:962] "Client rotation is on, will bootstrap in background"
Sep 10 15:26:54.751214 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:54.747949    2567 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Sep 10 15:26:54.751214 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:54.748969    2567 server.go:1019] "Starting client certificate rotation"
Sep 10 15:26:54.751214 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:54.749069    2567 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 10 15:26:54.751214 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:54.749106    2567 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 10 15:26:54.751546 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:54.751532    2567 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 10 15:26:54.754359 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:54.754346    2567 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 10 15:26:54.765130 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:54.765114    2567 log.go:25] "Validated CRI v1 runtime API"
Sep 10 15:26:54.768434 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:54.768416    2567 log.go:25] "Validated CRI v1 image API"
Sep 10 15:26:54.769392 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:54.769378    2567 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Sep 10 15:26:54.772893 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:54.772872    2567 fs.go:135] Filesystem UUIDs: map[01fb67a2-5dab-4ad4-9c60-b4d00428e65d:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 aa38b7e2-a888-4452-92f2-d5816a19ed73:/dev/nvme0n1p3]
Sep 10 15:26:54.772946 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:54.772893    2567 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Sep 10 15:26:54.778339 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:54.778204    2567 manager.go:216] Machine: {Timestamp:2026-09-10 15:26:54.77677702 +0000 UTC m=+0.345008741 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:2499996 MemoryCapacity:33164476416 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec27e2e802f3f5e57314393b91d77fa9 SystemUUID:ec27e2e8-02f3-f5e5-7314-393b91d77fa9 BootID:b2d872f4-0a5e-433b-aacb-927e5bc93c31 Filesystems:[{Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582238208 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582238208 Type:vfs Inodes:4048398 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:a4:31:d3:61:a1 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:a4:31:d3:61:a1 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:12:77:06:03:84:51 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164476416 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Sep 10 15:26:54.778339 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:54.778327    2567 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Sep 10 15:26:54.778452 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:54.778415    2567 manager.go:232] Version: {KernelVersion:5.14.0-570.138.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260901-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Sep 10 15:26:54.779978 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:54.779959    2567 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 10 15:26:54.781187 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:54.781150    2567 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Sep 10 15:26:54.781391 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:54.781192    2567 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-128-241.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Sep 10 15:26:54.782412 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:54.782399    2567 topology_manager.go:138] "Creating topology manager with none policy"
Sep 10 15:26:54.782471 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:54.782417    2567 container_manager_linux.go:309] "Creating device plugin manager"
Sep 10 15:26:54.782471 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:54.782430    2567 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 10 15:26:54.783108 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:54.783097    2567 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 10 15:26:54.784303 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:54.784291    2567 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Sep 10 15:26:54.787045 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:54.787031    2567 state_mem.go:36] "Initialized new in-memory state store"
Sep 10 15:26:54.787190 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:54.787179    2567 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Sep 10 15:26:54.787273 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:54.787244    2567 kubelet.go:486] "Attempting to sync node with API server"
Sep 10 15:26:54.787273 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:54.787267    2567 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Sep 10 15:26:54.787369 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:54.787287    2567 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Sep 10 15:26:54.787411 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:54.787369    2567 kubelet.go:398] "Adding apiserver pod source"
Sep 10 15:26:54.787411 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:54.787383    2567 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Sep 10 15:26:54.787819 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:54.787806    2567 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 10 15:26:54.787888 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:54.787831    2567 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 10 15:26:54.790270 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:54.790247    2567 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.12-5.rhaos4.21.gitd1ec63c.el9" apiVersion="v1"
Sep 10 15:26:54.793044 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:54.793027    2567 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Sep 10 15:26:54.793123 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:54.793067    2567 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Sep 10 15:26:54.793314 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:54.793301    2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Sep 10 15:26:54.793380 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:54.793320    2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Sep 10 15:26:54.793380 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:54.793329    2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Sep 10 15:26:54.793380 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:54.793336    2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Sep 10 15:26:54.793380 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:54.793347    2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Sep 10 15:26:54.793380 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:54.793355    2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Sep 10 15:26:54.793903 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:54.793893    2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Sep 10 15:26:54.793960 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:54.793904    2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Sep 10 15:26:54.793960 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:54.793915    2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Sep 10 15:26:54.793960 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:54.793923    2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Sep 10 15:26:54.793960 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:54.793935    2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Sep 10 15:26:54.793960 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:54.793948    2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Sep 10 15:26:54.794814 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:54.794802    2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Sep 10 15:26:54.794865 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:54.794816    2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Sep 10 15:26:54.798192 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:54.798173    2567 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-128-241.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Sep 10 15:26:54.798769 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:54.798754    2567 server.go:1268] "Started kubelet"
Sep 10 15:26:54.798844 ip-10-0-128-241 kubenswrapper[2567]: E0910 15:26:54.798822    2567 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Sep 10 15:26:54.798889 ip-10-0-128-241 kubenswrapper[2567]: E0910 15:26:54.798824    2567 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-128-241.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Sep 10 15:26:54.798889 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:54.798822    2567 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Sep 10 15:26:54.798889 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:54.798858    2567 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Sep 10 15:26:54.799002 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:54.798916    2567 server_v1.go:49] "podresources" method="list" useActivePods=true
Sep 10 15:26:54.799764 ip-10-0-128-241 systemd[1]: Started Kubernetes Kubelet.
Sep 10 15:26:54.799923 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:54.799907    2567 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Sep 10 15:26:54.800801 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:54.800786    2567 server.go:310] "Adding debug handlers to kubelet server"
Sep 10 15:26:54.802780 ip-10-0-128-241 kubenswrapper[2567]: E0910 15:26:54.802755    2567 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Sep 10 15:26:54.804602 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:54.804584    2567 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Sep 10 15:26:54.804688 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:54.804584    2567 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Sep 10 15:26:54.804951 ip-10-0-128-241 kubenswrapper[2567]: E0910 15:26:54.804913    2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-241.ec2.internal\" not found"
Sep 10 15:26:54.805131 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:54.805112    2567 volume_manager.go:311] "The desired_state_of_world populator starts"
Sep 10 15:26:54.805210 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:54.805149    2567 volume_manager.go:313] "Starting Kubelet Volume Manager"
Sep 10 15:26:54.805210 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:54.805122    2567 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Sep 10 15:26:54.805335 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:54.805268    2567 reconstruct.go:97] "Volume reconstruction finished"
Sep 10 15:26:54.805335 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:54.805277    2567 reconciler.go:29] "Reconciler: start to sync state"
Sep 10 15:26:54.806200 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:54.806034    2567 factory.go:55] Registering systemd factory
Sep 10 15:26:54.806291 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:54.806206    2567 factory.go:223] Registration of the systemd container factory successfully
Sep 10 15:26:54.807024 ip-10-0-128-241 kubenswrapper[2567]: E0910 15:26:54.805973    2567 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-128-241.ec2.internal.18d3ff25cc108528  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-128-241.ec2.internal,UID:ip-10-0-128-241.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-128-241.ec2.internal,},FirstTimestamp:2026-09-10 15:26:54.79872644 +0000 UTC m=+0.366958161,LastTimestamp:2026-09-10 15:26:54.79872644 +0000 UTC m=+0.366958161,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-128-241.ec2.internal,}"
Sep 10 15:26:54.808195 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:54.807337    2567 factory.go:153] Registering CRI-O factory
Sep 10 15:26:54.808195 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:54.807368    2567 factory.go:223] Registration of the crio container factory successfully
Sep 10 15:26:54.808195 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:54.807454    2567 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Sep 10 15:26:54.808195 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:54.807488    2567 factory.go:103] Registering Raw factory
Sep 10 15:26:54.808195 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:54.807517    2567 manager.go:1196] Started watching for new ooms in manager
Sep 10 15:26:54.808195 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:54.808012    2567 manager.go:349] Starting recovery of all containers
Sep 10 15:26:54.812071 ip-10-0-128-241 kubenswrapper[2567]: E0910 15:26:54.812043    2567 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-128-241.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Sep 10 15:26:54.812145 ip-10-0-128-241 kubenswrapper[2567]: E0910 15:26:54.812125    2567 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Sep 10 15:26:54.818723 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:54.818701    2567 manager.go:354] Recovery completed
Sep 10 15:26:54.825208 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:54.825190    2567 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 10 15:26:54.827097 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:54.827072    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-241.ec2.internal" event="NodeHasSufficientMemory"
Sep 10 15:26:54.828242 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:54.828217    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-241.ec2.internal" event="NodeHasNoDiskPressure"
Sep 10 15:26:54.828296 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:54.828258    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-241.ec2.internal" event="NodeHasSufficientPID"
Sep 10 15:26:54.828961 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:54.828942    2567 cpu_manager.go:222] "Starting CPU manager" policy="none"
Sep 10 15:26:54.828961 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:54.828959    2567 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Sep 10 15:26:54.829093 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:54.828978    2567 state_mem.go:36] "Initialized new in-memory state store"
Sep 10 15:26:54.829093 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:54.828960    2567 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 10 15:26:54.829255 ip-10-0-128-241 kubenswrapper[2567]: E0910 15:26:54.829166    2567 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-128-241.ec2.internal.18d3ff25cdc18003  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-128-241.ec2.internal,UID:ip-10-0-128-241.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-128-241.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-128-241.ec2.internal,},FirstTimestamp:2026-09-10 15:26:54.827102211 +0000 UTC m=+0.395333936,LastTimestamp:2026-09-10 15:26:54.827102211 +0000 UTC m=+0.395333936,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-128-241.ec2.internal,}"
Sep 10 15:26:54.829768 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:54.829747    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-241.ec2.internal" event="NodeHasSufficientMemory"
Sep 10 15:26:54.829851 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:54.829779    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-241.ec2.internal" event="NodeHasNoDiskPressure"
Sep 10 15:26:54.829851 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:54.829793    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-241.ec2.internal" event="NodeHasSufficientPID"
Sep 10 15:26:54.830357 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:54.830342    2567 policy_none.go:49] "None policy: Start"
Sep 10 15:26:54.830419 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:54.830363    2567 memory_manager.go:187] "Starting memorymanager" policy="None"
Sep 10 15:26:54.830419 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:54.830378    2567 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Sep 10 15:26:54.830612 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:54.830597    2567 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-sknct"
Sep 10 15:26:54.831470 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:54.831459    2567 policy_none.go:47] "Start"
Sep 10 15:26:54.839867 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:54.839851    2567 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-sknct"
Sep 10 15:26:54.840711 ip-10-0-128-241 kubenswrapper[2567]: E0910 15:26:54.840636    2567 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-128-241.ec2.internal.18d3ff25cdd2e668  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-128-241.ec2.internal,UID:ip-10-0-128-241.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-128-241.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-128-241.ec2.internal,},FirstTimestamp:2026-09-10 15:26:54.828242536 +0000 UTC m=+0.396474263,LastTimestamp:2026-09-10 15:26:54.828242536 +0000 UTC m=+0.396474263,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-128-241.ec2.internal,}"
Sep 10 15:26:54.855838 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:54.855823    2567 manager.go:337] "Starting Device Plugin manager"
Sep 10 15:26:54.869247 ip-10-0-128-241 kubenswrapper[2567]: E0910 15:26:54.855858    2567 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Sep 10 15:26:54.869247 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:54.855868    2567 server.go:87] "Starting device plugin registration server"
Sep 10 15:26:54.869247 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:54.856170    2567 eviction_manager.go:189] "Eviction manager: starting control loop"
Sep 10 15:26:54.869247 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:54.856188    2567 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Sep 10 15:26:54.869247 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:54.856301    2567 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Sep 10 15:26:54.869247 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:54.856382    2567 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Sep 10 15:26:54.869247 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:54.856391    2567 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Sep 10 15:26:54.869247 ip-10-0-128-241 kubenswrapper[2567]: E0910 15:26:54.856816    2567 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Sep 10 15:26:54.869247 ip-10-0-128-241 kubenswrapper[2567]: E0910 15:26:54.856855    2567 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-128-241.ec2.internal\" not found"
Sep 10 15:26:54.896259 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:54.896215    2567 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Sep 10 15:26:54.897478 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:54.897460    2567 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Sep 10 15:26:54.897548 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:54.897524    2567 status_manager.go:244] "Starting to sync pod status with apiserver"
Sep 10 15:26:54.897594 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:54.897557    2567 kubelet.go:2448] "Starting kubelet main sync loop"
Sep 10 15:26:54.897639 ip-10-0-128-241 kubenswrapper[2567]: E0910 15:26:54.897603    2567 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Sep 10 15:26:54.899822 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:54.899803    2567 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 10 15:26:54.956860 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:54.956819    2567 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 10 15:26:54.960424 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:54.960405    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-241.ec2.internal" event="NodeHasSufficientMemory"
Sep 10 15:26:54.960549 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:54.960436    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-241.ec2.internal" event="NodeHasNoDiskPressure"
Sep 10 15:26:54.960549 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:54.960446    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-241.ec2.internal" event="NodeHasSufficientPID"
Sep 10 15:26:54.960549 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:54.960468    2567 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-128-241.ec2.internal"
Sep 10 15:26:54.968224 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:54.968208    2567 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-128-241.ec2.internal"
Sep 10 15:26:54.968298 ip-10-0-128-241 kubenswrapper[2567]: E0910 15:26:54.968242    2567 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-128-241.ec2.internal\": node \"ip-10-0-128-241.ec2.internal\" not found"
Sep 10 15:26:54.987387 ip-10-0-128-241 kubenswrapper[2567]: E0910 15:26:54.987312    2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-241.ec2.internal\" not found"
Sep 10 15:26:54.998423 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:54.998377    2567 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-241.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-128-241.ec2.internal"]
Sep 10 15:26:54.998533 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:54.998489    2567 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 10 15:26:54.999617 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:54.999599    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-241.ec2.internal" event="NodeHasSufficientMemory"
Sep 10 15:26:54.999731 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:54.999633    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-241.ec2.internal" event="NodeHasNoDiskPressure"
Sep 10 15:26:54.999731 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:54.999650    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-241.ec2.internal" event="NodeHasSufficientPID"
Sep 10 15:26:55.000908 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:55.000894    2567 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 10 15:26:55.001017 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:55.000996    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-241.ec2.internal"
Sep 10 15:26:55.001108 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:55.001044    2567 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 10 15:26:55.001750 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:55.001732    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-241.ec2.internal" event="NodeHasSufficientMemory"
Sep 10 15:26:55.001829 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:55.001761    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-241.ec2.internal" event="NodeHasNoDiskPressure"
Sep 10 15:26:55.001829 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:55.001739    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-241.ec2.internal" event="NodeHasSufficientMemory"
Sep 10 15:26:55.001829 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:55.001779    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-241.ec2.internal" event="NodeHasSufficientPID"
Sep 10 15:26:55.001829 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:55.001806    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-241.ec2.internal" event="NodeHasNoDiskPressure"
Sep 10 15:26:55.001829 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:55.001822    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-241.ec2.internal" event="NodeHasSufficientPID"
Sep 10 15:26:55.003080 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:55.003066    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-241.ec2.internal"
Sep 10 15:26:55.003133 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:55.003095    2567 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 10 15:26:55.003845 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:55.003829    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-241.ec2.internal" event="NodeHasSufficientMemory"
Sep 10 15:26:55.003917 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:55.003856    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-241.ec2.internal" event="NodeHasNoDiskPressure"
Sep 10 15:26:55.003917 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:55.003872    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-241.ec2.internal" event="NodeHasSufficientPID"
Sep 10 15:26:55.028954 ip-10-0-128-241 kubenswrapper[2567]: E0910 15:26:55.028928    2567 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-128-241.ec2.internal\" not found" node="ip-10-0-128-241.ec2.internal"
Sep 10 15:26:55.033357 ip-10-0-128-241 kubenswrapper[2567]: E0910 15:26:55.033342    2567 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-128-241.ec2.internal\" not found" node="ip-10-0-128-241.ec2.internal"
Sep 10 15:26:55.087608 ip-10-0-128-241 kubenswrapper[2567]: E0910 15:26:55.087579    2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-241.ec2.internal\" not found"
Sep 10 15:26:55.106934 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:55.106907    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/7cc72b205a190f6d5d54c7ca1053da96-config\") pod \"kube-apiserver-proxy-ip-10-0-128-241.ec2.internal\" (UID: \"7cc72b205a190f6d5d54c7ca1053da96\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-241.ec2.internal"
Sep 10 15:26:55.107021 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:55.106938    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/e18dcc5e79fca5bc2558326e4e703c37-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-241.ec2.internal\" (UID: \"e18dcc5e79fca5bc2558326e4e703c37\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-241.ec2.internal"
Sep 10 15:26:55.107021 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:55.106955    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e18dcc5e79fca5bc2558326e4e703c37-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-241.ec2.internal\" (UID: \"e18dcc5e79fca5bc2558326e4e703c37\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-241.ec2.internal"
Sep 10 15:26:55.188094 ip-10-0-128-241 kubenswrapper[2567]: E0910 15:26:55.188067    2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-241.ec2.internal\" not found"
Sep 10 15:26:55.207407 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:55.207374    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/7cc72b205a190f6d5d54c7ca1053da96-config\") pod \"kube-apiserver-proxy-ip-10-0-128-241.ec2.internal\" (UID: \"7cc72b205a190f6d5d54c7ca1053da96\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-241.ec2.internal"
Sep 10 15:26:55.207407 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:55.207411    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/e18dcc5e79fca5bc2558326e4e703c37-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-241.ec2.internal\" (UID: \"e18dcc5e79fca5bc2558326e4e703c37\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-241.ec2.internal"
Sep 10 15:26:55.207519 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:55.207427    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e18dcc5e79fca5bc2558326e4e703c37-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-241.ec2.internal\" (UID: \"e18dcc5e79fca5bc2558326e4e703c37\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-241.ec2.internal"
Sep 10 15:26:55.207519 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:55.207468    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/7cc72b205a190f6d5d54c7ca1053da96-config\") pod \"kube-apiserver-proxy-ip-10-0-128-241.ec2.internal\" (UID: \"7cc72b205a190f6d5d54c7ca1053da96\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-241.ec2.internal"
Sep 10 15:26:55.207519 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:55.207488    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e18dcc5e79fca5bc2558326e4e703c37-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-241.ec2.internal\" (UID: \"e18dcc5e79fca5bc2558326e4e703c37\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-241.ec2.internal"
Sep 10 15:26:55.207519 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:55.207458    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/e18dcc5e79fca5bc2558326e4e703c37-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-241.ec2.internal\" (UID: \"e18dcc5e79fca5bc2558326e4e703c37\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-241.ec2.internal"
Sep 10 15:26:55.288716 ip-10-0-128-241 kubenswrapper[2567]: E0910 15:26:55.288649    2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-241.ec2.internal\" not found"
Sep 10 15:26:55.332118 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:55.332096    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-241.ec2.internal"
Sep 10 15:26:55.335924 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:55.335904    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-241.ec2.internal"
Sep 10 15:26:55.389803 ip-10-0-128-241 kubenswrapper[2567]: E0910 15:26:55.389772    2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-241.ec2.internal\" not found"
Sep 10 15:26:55.490404 ip-10-0-128-241 kubenswrapper[2567]: E0910 15:26:55.490381    2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-241.ec2.internal\" not found"
Sep 10 15:26:55.596586 ip-10-0-128-241 kubenswrapper[2567]: E0910 15:26:55.596497    2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-241.ec2.internal\" not found"
Sep 10 15:26:55.696658 ip-10-0-128-241 kubenswrapper[2567]: E0910 15:26:55.696623    2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-241.ec2.internal\" not found"
Sep 10 15:26:55.748794 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:55.748761    2567 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Sep 10 15:26:55.749204 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:55.748934    2567 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 10 15:26:55.797051 ip-10-0-128-241 kubenswrapper[2567]: E0910 15:26:55.797019    2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-241.ec2.internal\" not found"
Sep 10 15:26:55.843624 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:55.843571    2567 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-09-09 15:21:55 +0000 UTC" deadline="2028-02-07 10:45:33.588231774 +0000 UTC"
Sep 10 15:26:55.843624 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:55.843617    2567 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12355h18m37.744619388s"
Sep 10 15:26:55.883668 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:55.883603    2567 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 10 15:26:55.897761 ip-10-0-128-241 kubenswrapper[2567]: E0910 15:26:55.897734    2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-241.ec2.internal\" not found"
Sep 10 15:26:55.998124 ip-10-0-128-241 kubenswrapper[2567]: E0910 15:26:55.998088    2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-241.ec2.internal\" not found"
Sep 10 15:26:56.022169 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:56.022136    2567 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 10 15:26:56.023389 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:56.023370    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-241.ec2.internal" event="NodeHasSufficientMemory"
Sep 10 15:26:56.023502 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:56.023406    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-241.ec2.internal" event="NodeHasNoDiskPressure"
Sep 10 15:26:56.023502 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:56.023422    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-241.ec2.internal" event="NodeHasSufficientPID"
Sep 10 15:26:56.082297 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:56.082256    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7cc72b205a190f6d5d54c7ca1053da96.slice/crio-9512f356bdad65be5f43249ba31b475995b83f7f071cd7871517f25380847f80 WatchSource:0}: Error finding container 9512f356bdad65be5f43249ba31b475995b83f7f071cd7871517f25380847f80: Status 404 returned error can't find the container with id 9512f356bdad65be5f43249ba31b475995b83f7f071cd7871517f25380847f80
Sep 10 15:26:56.082702 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:56.082682    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode18dcc5e79fca5bc2558326e4e703c37.slice/crio-1b29632eaa729c57aa1bdd0f61275c31afae93fd0e0cd880818b383157eea809 WatchSource:0}: Error finding container 1b29632eaa729c57aa1bdd0f61275c31afae93fd0e0cd880818b383157eea809: Status 404 returned error can't find the container with id 1b29632eaa729c57aa1bdd0f61275c31afae93fd0e0cd880818b383157eea809
Sep 10 15:26:56.087219 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:56.087205    2567 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 10 15:26:56.099277 ip-10-0-128-241 kubenswrapper[2567]: E0910 15:26:56.099252    2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-241.ec2.internal\" not found"
Sep 10 15:26:56.187853 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:56.187772    2567 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 10 15:26:56.205146 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:56.205114    2567 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-241.ec2.internal"
Sep 10 15:26:56.221009 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:56.220986    2567 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 10 15:26:56.222329 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:56.222304    2567 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-241.ec2.internal"
Sep 10 15:26:56.230009 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:56.229993    2567 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 10 15:26:56.295753 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:56.295727    2567 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 10 15:26:56.679872 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:56.679786    2567 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 10 15:26:56.789176 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:56.789139    2567 apiserver.go:52] "Watching apiserver"
Sep 10 15:26:56.800819 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:56.800786    2567 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Sep 10 15:26:56.802375 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:56.802349    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/node-ca-g8rkv","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-241.ec2.internal","openshift-network-operator/iptables-alerter-9b7ts","openshift-ovn-kubernetes/ovnkube-node-bctsx","kube-system/global-pull-secret-syncer-fgx9b","kube-system/konnectivity-agent-pzrzz","kube-system/kube-apiserver-proxy-ip-10-0-128-241.ec2.internal","openshift-cluster-node-tuning-operator/tuned-xt7qs","openshift-multus/multus-additional-cni-plugins-82drl","openshift-multus/multus-gqngc","openshift-multus/network-metrics-daemon-zxpwj","openshift-network-diagnostics/network-check-target-2f7vt","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-krx2c","openshift-dns/node-resolver-rctzd"]
Sep 10 15:26:56.803888 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:56.803861    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-gqngc"
Sep 10 15:26:56.805077 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:56.805024    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zxpwj"
Sep 10 15:26:56.805077 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:56.805048    2567 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Sep 10 15:26:56.805228 ip-10-0-128-241 kubenswrapper[2567]: E0910 15:26:56.805102    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zxpwj" podUID="f91d1ea8-2355-4b51-b736-448a89aba131"
Sep 10 15:26:56.806371 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:56.806350    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-9b7ts"
Sep 10 15:26:56.806469 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:56.806444    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Sep 10 15:26:56.806599 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:56.806576    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Sep 10 15:26:56.806739 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:56.806712    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Sep 10 15:26:56.806739 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:56.806585    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-hlkng\""
Sep 10 15:26:56.807095 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:56.807072    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Sep 10 15:26:56.807447 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:56.807432    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-bctsx"
Sep 10 15:26:56.809007 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:56.808987    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Sep 10 15:26:56.809106 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:56.809007    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Sep 10 15:26:56.809377 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:56.809356    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Sep 10 15:26:56.809516 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:56.809498    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-rctzd"
Sep 10 15:26:56.809580 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:56.809360    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-v978l\""
Sep 10 15:26:56.809859 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:56.809838    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Sep 10 15:26:56.810373 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:56.810351    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Sep 10 15:26:56.810472 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:56.810390    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Sep 10 15:26:56.810472 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:56.810409    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Sep 10 15:26:56.810982 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:56.810962    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-pzrzz"
Sep 10 15:26:56.811396 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:56.811370    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-vx8kw\""
Sep 10 15:26:56.811497 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:56.811400    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Sep 10 15:26:56.811497 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:56.811482    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Sep 10 15:26:56.811826 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:56.811811    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Sep 10 15:26:56.811895 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:56.811827    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-q6dsw\""
Sep 10 15:26:56.811895 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:56.811811    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Sep 10 15:26:56.812904 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:56.812613    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-82drl"
Sep 10 15:26:56.813916 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:56.813890    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-ww5jn\""
Sep 10 15:26:56.814008 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:56.813984    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Sep 10 15:26:56.814211 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:56.814195    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Sep 10 15:26:56.815234 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:56.815213    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Sep 10 15:26:56.815398 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:56.815316    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-xt7qs"
Sep 10 15:26:56.815521 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:56.815504    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-pl4xd\""
Sep 10 15:26:56.816351 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:56.815740    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Sep 10 15:26:56.816351 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:56.816225    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2f7vt"
Sep 10 15:26:56.816351 ip-10-0-128-241 kubenswrapper[2567]: E0910 15:26:56.816310    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2f7vt" podUID="e105e176-14c2-4093-946d-492879428ef7"
Sep 10 15:26:56.816731 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:56.816710    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/1b59aadf-17c2-49a9-90e4-c507ca7d21c2-cni-binary-copy\") pod \"multus-gqngc\" (UID: \"1b59aadf-17c2-49a9-90e4-c507ca7d21c2\") " pod="openshift-multus/multus-gqngc"
Sep 10 15:26:56.816796 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:56.816749    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1b59aadf-17c2-49a9-90e4-c507ca7d21c2-host-var-lib-kubelet\") pod \"multus-gqngc\" (UID: \"1b59aadf-17c2-49a9-90e4-c507ca7d21c2\") " pod="openshift-multus/multus-gqngc"
Sep 10 15:26:56.816796 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:56.816779    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/1b59aadf-17c2-49a9-90e4-c507ca7d21c2-multus-conf-dir\") pod \"multus-gqngc\" (UID: \"1b59aadf-17c2-49a9-90e4-c507ca7d21c2\") " pod="openshift-multus/multus-gqngc"
Sep 10 15:26:56.816905 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:56.816806    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2j2dm\" (UniqueName: \"kubernetes.io/projected/f91d1ea8-2355-4b51-b736-448a89aba131-kube-api-access-2j2dm\") pod \"network-metrics-daemon-zxpwj\" (UID: \"f91d1ea8-2355-4b51-b736-448a89aba131\") " pod="openshift-multus/network-metrics-daemon-zxpwj"
Sep 10 15:26:56.816905 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:56.816833    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/b2e911b2-e1d3-4eb1-85b8-61f6a12bd9ba-iptables-alerter-script\") pod \"iptables-alerter-9b7ts\" (UID: \"b2e911b2-e1d3-4eb1-85b8-61f6a12bd9ba\") " pod="openshift-network-operator/iptables-alerter-9b7ts"
Sep 10 15:26:56.816905 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:56.816850    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bfpq4\" (UniqueName: \"kubernetes.io/projected/b2e911b2-e1d3-4eb1-85b8-61f6a12bd9ba-kube-api-access-bfpq4\") pod \"iptables-alerter-9b7ts\" (UID: \"b2e911b2-e1d3-4eb1-85b8-61f6a12bd9ba\") " pod="openshift-network-operator/iptables-alerter-9b7ts"
Sep 10 15:26:56.816905 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:56.816867    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1b59aadf-17c2-49a9-90e4-c507ca7d21c2-multus-cni-dir\") pod \"multus-gqngc\" (UID: \"1b59aadf-17c2-49a9-90e4-c507ca7d21c2\") " pod="openshift-multus/multus-gqngc"
Sep 10 15:26:56.816905 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:56.816886    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/1b59aadf-17c2-49a9-90e4-c507ca7d21c2-cnibin\") pod \"multus-gqngc\" (UID: \"1b59aadf-17c2-49a9-90e4-c507ca7d21c2\") " pod="openshift-multus/multus-gqngc"
Sep 10 15:26:56.816905 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:56.816905    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/1b59aadf-17c2-49a9-90e4-c507ca7d21c2-multus-socket-dir-parent\") pod \"multus-gqngc\" (UID: \"1b59aadf-17c2-49a9-90e4-c507ca7d21c2\") " pod="openshift-multus/multus-gqngc"
Sep 10 15:26:56.817230 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:56.816919    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/1b59aadf-17c2-49a9-90e4-c507ca7d21c2-host-run-k8s-cni-cncf-io\") pod \"multus-gqngc\" (UID: \"1b59aadf-17c2-49a9-90e4-c507ca7d21c2\") " pod="openshift-multus/multus-gqngc"
Sep 10 15:26:56.817230 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:56.816935    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/1b59aadf-17c2-49a9-90e4-c507ca7d21c2-host-var-lib-cni-multus\") pod \"multus-gqngc\" (UID: \"1b59aadf-17c2-49a9-90e4-c507ca7d21c2\") " pod="openshift-multus/multus-gqngc"
Sep 10 15:26:56.817230 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:56.816957    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/1b59aadf-17c2-49a9-90e4-c507ca7d21c2-hostroot\") pod \"multus-gqngc\" (UID: \"1b59aadf-17c2-49a9-90e4-c507ca7d21c2\") " pod="openshift-multus/multus-gqngc"
Sep 10 15:26:56.817230 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:56.816977    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1b59aadf-17c2-49a9-90e4-c507ca7d21c2-etc-kubernetes\") pod \"multus-gqngc\" (UID: \"1b59aadf-17c2-49a9-90e4-c507ca7d21c2\") " pod="openshift-multus/multus-gqngc"
Sep 10 15:26:56.817230 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:56.816995    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1b59aadf-17c2-49a9-90e4-c507ca7d21c2-system-cni-dir\") pod \"multus-gqngc\" (UID: \"1b59aadf-17c2-49a9-90e4-c507ca7d21c2\") " pod="openshift-multus/multus-gqngc"
Sep 10 15:26:56.817230 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:56.817017    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/1b59aadf-17c2-49a9-90e4-c507ca7d21c2-host-var-lib-cni-bin\") pod \"multus-gqngc\" (UID: \"1b59aadf-17c2-49a9-90e4-c507ca7d21c2\") " pod="openshift-multus/multus-gqngc"
Sep 10 15:26:56.817230 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:56.817036    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/1b59aadf-17c2-49a9-90e4-c507ca7d21c2-multus-daemon-config\") pod \"multus-gqngc\" (UID: \"1b59aadf-17c2-49a9-90e4-c507ca7d21c2\") " pod="openshift-multus/multus-gqngc"
Sep 10 15:26:56.817230 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:56.817059    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5cscg\" (UniqueName: \"kubernetes.io/projected/1b59aadf-17c2-49a9-90e4-c507ca7d21c2-kube-api-access-5cscg\") pod \"multus-gqngc\" (UID: \"1b59aadf-17c2-49a9-90e4-c507ca7d21c2\") " pod="openshift-multus/multus-gqngc"
Sep 10 15:26:56.817230 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:56.817083    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f91d1ea8-2355-4b51-b736-448a89aba131-metrics-certs\") pod \"network-metrics-daemon-zxpwj\" (UID: \"f91d1ea8-2355-4b51-b736-448a89aba131\") " pod="openshift-multus/network-metrics-daemon-zxpwj"
Sep 10 15:26:56.817230 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:56.817146    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/b2e911b2-e1d3-4eb1-85b8-61f6a12bd9ba-host-slash\") pod \"iptables-alerter-9b7ts\" (UID: \"b2e911b2-e1d3-4eb1-85b8-61f6a12bd9ba\") " pod="openshift-network-operator/iptables-alerter-9b7ts"
Sep 10 15:26:56.817230 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:56.817209    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/1b59aadf-17c2-49a9-90e4-c507ca7d21c2-host-run-netns\") pod \"multus-gqngc\" (UID: \"1b59aadf-17c2-49a9-90e4-c507ca7d21c2\") " pod="openshift-multus/multus-gqngc"
Sep 10 15:26:56.817807 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:56.817426    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/1b59aadf-17c2-49a9-90e4-c507ca7d21c2-host-run-multus-certs\") pod \"multus-gqngc\" (UID: \"1b59aadf-17c2-49a9-90e4-c507ca7d21c2\") " pod="openshift-multus/multus-gqngc"
Sep 10 15:26:56.817807 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:56.817616    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fgx9b"
Sep 10 15:26:56.817807 ip-10-0-128-241 kubenswrapper[2567]: E0910 15:26:56.817676    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fgx9b" podUID="49bbf894-04df-427c-b71a-c108fbf25054"
Sep 10 15:26:56.817807 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:56.817775    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Sep 10 15:26:56.817807 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:56.817799    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Sep 10 15:26:56.818076 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:56.818061    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-p2njt\""
Sep 10 15:26:56.818903 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:56.818884    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-g8rkv"
Sep 10 15:26:56.820223 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:56.820202    2567 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 10 15:26:56.820223 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:56.820218    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-krx2c"
Sep 10 15:26:56.822128 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:56.821619    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Sep 10 15:26:56.822128 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:56.821682    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Sep 10 15:26:56.822128 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:56.821898    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-6cbtf\""
Sep 10 15:26:56.822128 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:56.821903    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Sep 10 15:26:56.822570 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:56.822550    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Sep 10 15:26:56.822665 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:56.822576    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Sep 10 15:26:56.822665 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:56.822590    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-kj8n2\""
Sep 10 15:26:56.823111 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:56.823092    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Sep 10 15:26:56.845436 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:56.845408    2567 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-zx59q"
Sep 10 15:26:56.855221 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:56.855198    2567 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-zx59q"
Sep 10 15:26:56.903343 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:56.903292    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-241.ec2.internal" event={"ID":"e18dcc5e79fca5bc2558326e4e703c37","Type":"ContainerStarted","Data":"1b29632eaa729c57aa1bdd0f61275c31afae93fd0e0cd880818b383157eea809"}
Sep 10 15:26:56.905016 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:56.904987    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-241.ec2.internal" event={"ID":"7cc72b205a190f6d5d54c7ca1053da96","Type":"ContainerStarted","Data":"9512f356bdad65be5f43249ba31b475995b83f7f071cd7871517f25380847f80"}
Sep 10 15:26:56.906424 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:56.906335    2567 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Sep 10 15:26:56.917858 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:56.917827    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bfpq4\" (UniqueName: \"kubernetes.io/projected/b2e911b2-e1d3-4eb1-85b8-61f6a12bd9ba-kube-api-access-bfpq4\") pod \"iptables-alerter-9b7ts\" (UID: \"b2e911b2-e1d3-4eb1-85b8-61f6a12bd9ba\") " pod="openshift-network-operator/iptables-alerter-9b7ts"
Sep 10 15:26:56.917965 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:56.917878    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/ccbffcf4-9a0d-445b-b05d-3bcf97e1866d-etc-modprobe-d\") pod \"tuned-xt7qs\" (UID: \"ccbffcf4-9a0d-445b-b05d-3bcf97e1866d\") " pod="openshift-cluster-node-tuning-operator/tuned-xt7qs"
Sep 10 15:26:56.917965 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:56.917908    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/831261f1-1d19-42bd-80c0-5173a7132c38-registration-dir\") pod \"aws-ebs-csi-driver-node-krx2c\" (UID: \"831261f1-1d19-42bd-80c0-5173a7132c38\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-krx2c"
Sep 10 15:26:56.917965 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:56.917938    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/1b59aadf-17c2-49a9-90e4-c507ca7d21c2-host-run-k8s-cni-cncf-io\") pod \"multus-gqngc\" (UID: \"1b59aadf-17c2-49a9-90e4-c507ca7d21c2\") " pod="openshift-multus/multus-gqngc"
Sep 10 15:26:56.918134 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:56.917963    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1b59aadf-17c2-49a9-90e4-c507ca7d21c2-etc-kubernetes\") pod \"multus-gqngc\" (UID: \"1b59aadf-17c2-49a9-90e4-c507ca7d21c2\") " pod="openshift-multus/multus-gqngc"
Sep 10 15:26:56.918134 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:56.917990    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5cscg\" (UniqueName: \"kubernetes.io/projected/1b59aadf-17c2-49a9-90e4-c507ca7d21c2-kube-api-access-5cscg\") pod \"multus-gqngc\" (UID: \"1b59aadf-17c2-49a9-90e4-c507ca7d21c2\") " pod="openshift-multus/multus-gqngc"
Sep 10 15:26:56.918134 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:56.918014    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/1b59aadf-17c2-49a9-90e4-c507ca7d21c2-host-run-k8s-cni-cncf-io\") pod \"multus-gqngc\" (UID: \"1b59aadf-17c2-49a9-90e4-c507ca7d21c2\") " pod="openshift-multus/multus-gqngc"
Sep 10 15:26:56.918134 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:56.918019    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c1a148df-5bb2-4059-9d8a-2715cc0b6f8c-host-run-ovn-kubernetes\") pod \"ovnkube-node-bctsx\" (UID: \"c1a148df-5bb2-4059-9d8a-2715cc0b6f8c\") " pod="openshift-ovn-kubernetes/ovnkube-node-bctsx"
Sep 10 15:26:56.918134 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:56.918083    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1b59aadf-17c2-49a9-90e4-c507ca7d21c2-etc-kubernetes\") pod \"multus-gqngc\" (UID: \"1b59aadf-17c2-49a9-90e4-c507ca7d21c2\") " pod="openshift-multus/multus-gqngc"
Sep 10 15:26:56.918134 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:56.918131    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/49bbf894-04df-427c-b71a-c108fbf25054-dbus\") pod \"global-pull-secret-syncer-fgx9b\" (UID: \"49bbf894-04df-427c-b71a-c108fbf25054\") " pod="kube-system/global-pull-secret-syncer-fgx9b"
Sep 10 15:26:56.918464 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:56.918179    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/831261f1-1d19-42bd-80c0-5173a7132c38-sys-fs\") pod \"aws-ebs-csi-driver-node-krx2c\" (UID: \"831261f1-1d19-42bd-80c0-5173a7132c38\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-krx2c"
Sep 10 15:26:56.918464 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:56.918210    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/c1a148df-5bb2-4059-9d8a-2715cc0b6f8c-ovn-node-metrics-cert\") pod \"ovnkube-node-bctsx\" (UID: \"c1a148df-5bb2-4059-9d8a-2715cc0b6f8c\") " pod="openshift-ovn-kubernetes/ovnkube-node-bctsx"
Sep 10 15:26:56.918464 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:56.918277    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fj2pp\" (UniqueName: \"kubernetes.io/projected/4dd1b47c-e497-45d2-a404-4ea3a2d8ec94-kube-api-access-fj2pp\") pod \"node-resolver-rctzd\" (UID: \"4dd1b47c-e497-45d2-a404-4ea3a2d8ec94\") " pod="openshift-dns/node-resolver-rctzd"
Sep 10 15:26:56.918464 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:56.918312    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b44e8918-d279-456a-b0cc-63526a3947c2-system-cni-dir\") pod \"multus-additional-cni-plugins-82drl\" (UID: \"b44e8918-d279-456a-b0cc-63526a3947c2\") " pod="openshift-multus/multus-additional-cni-plugins-82drl"
Sep 10 15:26:56.918464 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:56.918339    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2jrk5\" (UniqueName: \"kubernetes.io/projected/b44e8918-d279-456a-b0cc-63526a3947c2-kube-api-access-2jrk5\") pod \"multus-additional-cni-plugins-82drl\" (UID: \"b44e8918-d279-456a-b0cc-63526a3947c2\") " pod="openshift-multus/multus-additional-cni-plugins-82drl"
Sep 10 15:26:56.918464 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:56.918366    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/831261f1-1d19-42bd-80c0-5173a7132c38-device-dir\") pod \"aws-ebs-csi-driver-node-krx2c\" (UID: \"831261f1-1d19-42bd-80c0-5173a7132c38\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-krx2c"
Sep 10 15:26:56.918464 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:56.918398    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/b2e911b2-e1d3-4eb1-85b8-61f6a12bd9ba-iptables-alerter-script\") pod \"iptables-alerter-9b7ts\" (UID: \"b2e911b2-e1d3-4eb1-85b8-61f6a12bd9ba\") " pod="openshift-network-operator/iptables-alerter-9b7ts"
Sep 10 15:26:56.918464 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:56.918424    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c1a148df-5bb2-4059-9d8a-2715cc0b6f8c-run-openvswitch\") pod \"ovnkube-node-bctsx\" (UID: \"c1a148df-5bb2-4059-9d8a-2715cc0b6f8c\") " pod="openshift-ovn-kubernetes/ovnkube-node-bctsx"
Sep 10 15:26:56.918464 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:56.918456    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/1b59aadf-17c2-49a9-90e4-c507ca7d21c2-cnibin\") pod \"multus-gqngc\" (UID: \"1b59aadf-17c2-49a9-90e4-c507ca7d21c2\") " pod="openshift-multus/multus-gqngc"
Sep 10 15:26:56.918906 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:56.918481    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/1b59aadf-17c2-49a9-90e4-c507ca7d21c2-host-var-lib-cni-multus\") pod \"multus-gqngc\" (UID: \"1b59aadf-17c2-49a9-90e4-c507ca7d21c2\") " pod="openshift-multus/multus-gqngc"
Sep 10 15:26:56.918906 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:56.918507    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/1b59aadf-17c2-49a9-90e4-c507ca7d21c2-hostroot\") pod \"multus-gqngc\" (UID: \"1b59aadf-17c2-49a9-90e4-c507ca7d21c2\") " pod="openshift-multus/multus-gqngc"
Sep 10 15:26:56.918906 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:56.918532    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/b2e911b2-e1d3-4eb1-85b8-61f6a12bd9ba-host-slash\") pod \"iptables-alerter-9b7ts\" (UID: \"b2e911b2-e1d3-4eb1-85b8-61f6a12bd9ba\") " pod="openshift-network-operator/iptables-alerter-9b7ts"
Sep 10 15:26:56.918906 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:56.918548    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/1b59aadf-17c2-49a9-90e4-c507ca7d21c2-cnibin\") pod \"multus-gqngc\" (UID: \"1b59aadf-17c2-49a9-90e4-c507ca7d21c2\") " pod="openshift-multus/multus-gqngc"
Sep 10 15:26:56.918906 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:56.918557    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c1a148df-5bb2-4059-9d8a-2715cc0b6f8c-etc-openvswitch\") pod \"ovnkube-node-bctsx\" (UID: \"c1a148df-5bb2-4059-9d8a-2715cc0b6f8c\") " pod="openshift-ovn-kubernetes/ovnkube-node-bctsx"
Sep 10 15:26:56.918906 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:56.918594    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/c1a148df-5bb2-4059-9d8a-2715cc0b6f8c-run-ovn\") pod \"ovnkube-node-bctsx\" (UID: \"c1a148df-5bb2-4059-9d8a-2715cc0b6f8c\") " pod="openshift-ovn-kubernetes/ovnkube-node-bctsx"
Sep 10 15:26:56.918906 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:56.918625    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/1b59aadf-17c2-49a9-90e4-c507ca7d21c2-host-var-lib-cni-multus\") pod \"multus-gqngc\" (UID: \"1b59aadf-17c2-49a9-90e4-c507ca7d21c2\") " pod="openshift-multus/multus-gqngc"
Sep 10 15:26:56.918906 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:56.918625    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hhvx7\" (UniqueName: \"kubernetes.io/projected/c1a148df-5bb2-4059-9d8a-2715cc0b6f8c-kube-api-access-hhvx7\") pod \"ovnkube-node-bctsx\" (UID: \"c1a148df-5bb2-4059-9d8a-2715cc0b6f8c\") " pod="openshift-ovn-kubernetes/ovnkube-node-bctsx"
Sep 10 15:26:56.918906 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:56.918658    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/ccbffcf4-9a0d-445b-b05d-3bcf97e1866d-lib-modules\") pod \"tuned-xt7qs\" (UID: \"ccbffcf4-9a0d-445b-b05d-3bcf97e1866d\") " pod="openshift-cluster-node-tuning-operator/tuned-xt7qs"
Sep 10 15:26:56.918906 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:56.918684    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1b59aadf-17c2-49a9-90e4-c507ca7d21c2-system-cni-dir\") pod \"multus-gqngc\" (UID: \"1b59aadf-17c2-49a9-90e4-c507ca7d21c2\") " pod="openshift-multus/multus-gqngc"
Sep 10 15:26:56.918906 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:56.918689    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/1b59aadf-17c2-49a9-90e4-c507ca7d21c2-hostroot\") pod \"multus-gqngc\" (UID: \"1b59aadf-17c2-49a9-90e4-c507ca7d21c2\") " pod="openshift-multus/multus-gqngc"
Sep 10 15:26:56.918906 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:56.918713    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/ccbffcf4-9a0d-445b-b05d-3bcf97e1866d-etc-sysconfig\") pod \"tuned-xt7qs\" (UID: \"ccbffcf4-9a0d-445b-b05d-3bcf97e1866d\") " pod="openshift-cluster-node-tuning-operator/tuned-xt7qs"
Sep 10 15:26:56.918906 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:56.918738    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/ccbffcf4-9a0d-445b-b05d-3bcf97e1866d-host\") pod \"tuned-xt7qs\" (UID: \"ccbffcf4-9a0d-445b-b05d-3bcf97e1866d\") " pod="openshift-cluster-node-tuning-operator/tuned-xt7qs"
Sep 10 15:26:56.918906 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:56.918767    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4wmx2\" (UniqueName: \"kubernetes.io/projected/e105e176-14c2-4093-946d-492879428ef7-kube-api-access-4wmx2\") pod \"network-check-target-2f7vt\" (UID: \"e105e176-14c2-4093-946d-492879428ef7\") " pod="openshift-network-diagnostics/network-check-target-2f7vt"
Sep 10 15:26:56.918906 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:56.918809    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f6s58\" (UniqueName: \"kubernetes.io/projected/e9b09c08-aec9-41db-8386-e62731150ccc-kube-api-access-f6s58\") pod \"node-ca-g8rkv\" (UID: \"e9b09c08-aec9-41db-8386-e62731150ccc\") " pod="openshift-image-registry/node-ca-g8rkv"
Sep 10 15:26:56.918906 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:56.918835    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/831261f1-1d19-42bd-80c0-5173a7132c38-kubelet-dir\") pod \"aws-ebs-csi-driver-node-krx2c\" (UID: \"831261f1-1d19-42bd-80c0-5173a7132c38\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-krx2c"
Sep 10 15:26:56.918906 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:56.918861    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/1b59aadf-17c2-49a9-90e4-c507ca7d21c2-host-run-netns\") pod \"multus-gqngc\" (UID: \"1b59aadf-17c2-49a9-90e4-c507ca7d21c2\") " pod="openshift-multus/multus-gqngc"
Sep 10 15:26:56.919802 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:56.918886    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/c1a148df-5bb2-4059-9d8a-2715cc0b6f8c-host-run-netns\") pod \"ovnkube-node-bctsx\" (UID: \"c1a148df-5bb2-4059-9d8a-2715cc0b6f8c\") " pod="openshift-ovn-kubernetes/ovnkube-node-bctsx"
Sep 10 15:26:56.919802 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:56.918912    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4dd1b47c-e497-45d2-a404-4ea3a2d8ec94-tmp-dir\") pod \"node-resolver-rctzd\" (UID: \"4dd1b47c-e497-45d2-a404-4ea3a2d8ec94\") " pod="openshift-dns/node-resolver-rctzd"
Sep 10 15:26:56.919802 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:56.918937    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fkjzd\" (UniqueName: \"kubernetes.io/projected/ccbffcf4-9a0d-445b-b05d-3bcf97e1866d-kube-api-access-fkjzd\") pod \"tuned-xt7qs\" (UID: \"ccbffcf4-9a0d-445b-b05d-3bcf97e1866d\") " pod="openshift-cluster-node-tuning-operator/tuned-xt7qs"
Sep 10 15:26:56.919802 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:56.918966    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/49bbf894-04df-427c-b71a-c108fbf25054-original-pull-secret\") pod \"global-pull-secret-syncer-fgx9b\" (UID: \"49bbf894-04df-427c-b71a-c108fbf25054\") " pod="kube-system/global-pull-secret-syncer-fgx9b"
Sep 10 15:26:56.919802 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:56.918992    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/b44e8918-d279-456a-b0cc-63526a3947c2-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-82drl\" (UID: \"b44e8918-d279-456a-b0cc-63526a3947c2\") " pod="openshift-multus/multus-additional-cni-plugins-82drl"
Sep 10 15:26:56.919802 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:56.919018    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ccbffcf4-9a0d-445b-b05d-3bcf97e1866d-etc-kubernetes\") pod \"tuned-xt7qs\" (UID: \"ccbffcf4-9a0d-445b-b05d-3bcf97e1866d\") " pod="openshift-cluster-node-tuning-operator/tuned-xt7qs"
Sep 10 15:26:56.919802 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:56.919044    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/ccbffcf4-9a0d-445b-b05d-3bcf97e1866d-run\") pod \"tuned-xt7qs\" (UID: \"ccbffcf4-9a0d-445b-b05d-3bcf97e1866d\") " pod="openshift-cluster-node-tuning-operator/tuned-xt7qs"
Sep 10 15:26:56.919802 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:56.919067    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ccbffcf4-9a0d-445b-b05d-3bcf97e1866d-tmp\") pod \"tuned-xt7qs\" (UID: \"ccbffcf4-9a0d-445b-b05d-3bcf97e1866d\") " pod="openshift-cluster-node-tuning-operator/tuned-xt7qs"
Sep 10 15:26:56.919802 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:56.919092    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/49bbf894-04df-427c-b71a-c108fbf25054-kubelet-config\") pod \"global-pull-secret-syncer-fgx9b\" (UID: \"49bbf894-04df-427c-b71a-c108fbf25054\") " pod="kube-system/global-pull-secret-syncer-fgx9b"
Sep 10 15:26:56.919802 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:56.919130    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b44e8918-d279-456a-b0cc-63526a3947c2-cni-binary-copy\") pod \"multus-additional-cni-plugins-82drl\" (UID: \"b44e8918-d279-456a-b0cc-63526a3947c2\") " pod="openshift-multus/multus-additional-cni-plugins-82drl"
Sep 10 15:26:56.919802 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:56.919144    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/1b59aadf-17c2-49a9-90e4-c507ca7d21c2-host-run-netns\") pod \"multus-gqngc\" (UID: \"1b59aadf-17c2-49a9-90e4-c507ca7d21c2\") " pod="openshift-multus/multus-gqngc"
Sep 10 15:26:56.919802 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:56.919158    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1b59aadf-17c2-49a9-90e4-c507ca7d21c2-multus-cni-dir\") pod \"multus-gqngc\" (UID: \"1b59aadf-17c2-49a9-90e4-c507ca7d21c2\") " pod="openshift-multus/multus-gqngc"
Sep 10 15:26:56.919802 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:56.919191    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/1b59aadf-17c2-49a9-90e4-c507ca7d21c2-multus-socket-dir-parent\") pod \"multus-gqngc\" (UID: \"1b59aadf-17c2-49a9-90e4-c507ca7d21c2\") " pod="openshift-multus/multus-gqngc"
Sep 10 15:26:56.919802 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:56.919226    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/ccbffcf4-9a0d-445b-b05d-3bcf97e1866d-etc-sysctl-conf\") pod \"tuned-xt7qs\" (UID: \"ccbffcf4-9a0d-445b-b05d-3bcf97e1866d\") " pod="openshift-cluster-node-tuning-operator/tuned-xt7qs"
Sep 10 15:26:56.919802 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:56.919333    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/ccbffcf4-9a0d-445b-b05d-3bcf97e1866d-etc-systemd\") pod \"tuned-xt7qs\" (UID: \"ccbffcf4-9a0d-445b-b05d-3bcf97e1866d\") " pod="openshift-cluster-node-tuning-operator/tuned-xt7qs"
Sep 10 15:26:56.919802 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:56.919326    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/1b59aadf-17c2-49a9-90e4-c507ca7d21c2-multus-socket-dir-parent\") pod \"multus-gqngc\" (UID: \"1b59aadf-17c2-49a9-90e4-c507ca7d21c2\") " pod="openshift-multus/multus-gqngc"
Sep 10 15:26:56.919802 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:56.919369    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/ccbffcf4-9a0d-445b-b05d-3bcf97e1866d-etc-tuned\") pod \"tuned-xt7qs\" (UID: \"ccbffcf4-9a0d-445b-b05d-3bcf97e1866d\") " pod="openshift-cluster-node-tuning-operator/tuned-xt7qs"
Sep 10 15:26:56.920669 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:56.919385    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1b59aadf-17c2-49a9-90e4-c507ca7d21c2-system-cni-dir\") pod \"multus-gqngc\" (UID: \"1b59aadf-17c2-49a9-90e4-c507ca7d21c2\") " pod="openshift-multus/multus-gqngc"
Sep 10 15:26:56.920669 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:56.919399    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/1b59aadf-17c2-49a9-90e4-c507ca7d21c2-multus-daemon-config\") pod \"multus-gqngc\" (UID: \"1b59aadf-17c2-49a9-90e4-c507ca7d21c2\") " pod="openshift-multus/multus-gqngc"
Sep 10 15:26:56.920669 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:56.919405    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1b59aadf-17c2-49a9-90e4-c507ca7d21c2-multus-cni-dir\") pod \"multus-gqngc\" (UID: \"1b59aadf-17c2-49a9-90e4-c507ca7d21c2\") " pod="openshift-multus/multus-gqngc"
Sep 10 15:26:56.920669 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:56.919444    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/aa974d12-9161-4bc9-a1c7-0846241a3206-agent-certs\") pod \"konnectivity-agent-pzrzz\" (UID: \"aa974d12-9161-4bc9-a1c7-0846241a3206\") " pod="kube-system/konnectivity-agent-pzrzz"
Sep 10 15:26:56.920669 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:56.919478    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ccbffcf4-9a0d-445b-b05d-3bcf97e1866d-sys\") pod \"tuned-xt7qs\" (UID: \"ccbffcf4-9a0d-445b-b05d-3bcf97e1866d\") " pod="openshift-cluster-node-tuning-operator/tuned-xt7qs"
Sep 10 15:26:56.920669 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:56.919508    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/1b59aadf-17c2-49a9-90e4-c507ca7d21c2-host-run-multus-certs\") pod \"multus-gqngc\" (UID: \"1b59aadf-17c2-49a9-90e4-c507ca7d21c2\") " pod="openshift-multus/multus-gqngc"
Sep 10 15:26:56.920669 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:56.919542    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/c1a148df-5bb2-4059-9d8a-2715cc0b6f8c-host-slash\") pod \"ovnkube-node-bctsx\" (UID: \"c1a148df-5bb2-4059-9d8a-2715cc0b6f8c\") " pod="openshift-ovn-kubernetes/ovnkube-node-bctsx"
Sep 10 15:26:56.920669 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:56.919554    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/b2e911b2-e1d3-4eb1-85b8-61f6a12bd9ba-iptables-alerter-script\") pod \"iptables-alerter-9b7ts\" (UID: \"b2e911b2-e1d3-4eb1-85b8-61f6a12bd9ba\") " pod="openshift-network-operator/iptables-alerter-9b7ts"
Sep 10 15:26:56.920669 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:56.919567    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c1a148df-5bb2-4059-9d8a-2715cc0b6f8c-var-lib-openvswitch\") pod \"ovnkube-node-bctsx\" (UID: \"c1a148df-5bb2-4059-9d8a-2715cc0b6f8c\") " pod="openshift-ovn-kubernetes/ovnkube-node-bctsx"
Sep 10 15:26:56.920669 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:56.919572    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/1b59aadf-17c2-49a9-90e4-c507ca7d21c2-host-run-multus-certs\") pod \"multus-gqngc\" (UID: \"1b59aadf-17c2-49a9-90e4-c507ca7d21c2\") " pod="openshift-multus/multus-gqngc"
Sep 10 15:26:56.920669 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:56.919592    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/c1a148df-5bb2-4059-9d8a-2715cc0b6f8c-node-log\") pod \"ovnkube-node-bctsx\" (UID: \"c1a148df-5bb2-4059-9d8a-2715cc0b6f8c\") " pod="openshift-ovn-kubernetes/ovnkube-node-bctsx"
Sep 10 15:26:56.920669 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:56.919617    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/b2e911b2-e1d3-4eb1-85b8-61f6a12bd9ba-host-slash\") pod \"iptables-alerter-9b7ts\" (UID: \"b2e911b2-e1d3-4eb1-85b8-61f6a12bd9ba\") " pod="openshift-network-operator/iptables-alerter-9b7ts"
Sep 10 15:26:56.920669 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:56.919643    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/c1a148df-5bb2-4059-9d8a-2715cc0b6f8c-host-cni-netd\") pod \"ovnkube-node-bctsx\" (UID: \"c1a148df-5bb2-4059-9d8a-2715cc0b6f8c\") " pod="openshift-ovn-kubernetes/ovnkube-node-bctsx"
Sep 10 15:26:56.920669 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:56.919671    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c1a148df-5bb2-4059-9d8a-2715cc0b6f8c-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-bctsx\" (UID: \"c1a148df-5bb2-4059-9d8a-2715cc0b6f8c\") " pod="openshift-ovn-kubernetes/ovnkube-node-bctsx"
Sep 10 15:26:56.920669 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:56.919698    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/c1a148df-5bb2-4059-9d8a-2715cc0b6f8c-ovnkube-config\") pod \"ovnkube-node-bctsx\" (UID: \"c1a148df-5bb2-4059-9d8a-2715cc0b6f8c\") " pod="openshift-ovn-kubernetes/ovnkube-node-bctsx"
Sep 10 15:26:56.920669 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:56.919723    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/1b59aadf-17c2-49a9-90e4-c507ca7d21c2-multus-conf-dir\") pod \"multus-gqngc\" (UID: \"1b59aadf-17c2-49a9-90e4-c507ca7d21c2\") " pod="openshift-multus/multus-gqngc"
Sep 10 15:26:56.920669 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:56.919756    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/1b59aadf-17c2-49a9-90e4-c507ca7d21c2-multus-conf-dir\") pod \"multus-gqngc\" (UID: \"1b59aadf-17c2-49a9-90e4-c507ca7d21c2\") " pod="openshift-multus/multus-gqngc"
Sep 10 15:26:56.921564 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:56.919783    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/c1a148df-5bb2-4059-9d8a-2715cc0b6f8c-host-kubelet\") pod \"ovnkube-node-bctsx\" (UID: \"c1a148df-5bb2-4059-9d8a-2715cc0b6f8c\") " pod="openshift-ovn-kubernetes/ovnkube-node-bctsx"
Sep 10 15:26:56.921564 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:56.919805    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/c1a148df-5bb2-4059-9d8a-2715cc0b6f8c-ovnkube-script-lib\") pod \"ovnkube-node-bctsx\" (UID: \"c1a148df-5bb2-4059-9d8a-2715cc0b6f8c\") " pod="openshift-ovn-kubernetes/ovnkube-node-bctsx"
Sep 10 15:26:56.921564 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:56.919821    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/4dd1b47c-e497-45d2-a404-4ea3a2d8ec94-hosts-file\") pod \"node-resolver-rctzd\" (UID: \"4dd1b47c-e497-45d2-a404-4ea3a2d8ec94\") " pod="openshift-dns/node-resolver-rctzd"
Sep 10 15:26:56.921564 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:56.919874    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/e9b09c08-aec9-41db-8386-e62731150ccc-host\") pod \"node-ca-g8rkv\" (UID: \"e9b09c08-aec9-41db-8386-e62731150ccc\") " pod="openshift-image-registry/node-ca-g8rkv"
Sep 10 15:26:56.921564 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:56.919904    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/e9b09c08-aec9-41db-8386-e62731150ccc-serviceca\") pod \"node-ca-g8rkv\" (UID: \"e9b09c08-aec9-41db-8386-e62731150ccc\") " pod="openshift-image-registry/node-ca-g8rkv"
Sep 10 15:26:56.921564 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:56.919947    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f91d1ea8-2355-4b51-b736-448a89aba131-metrics-certs\") pod \"network-metrics-daemon-zxpwj\" (UID: \"f91d1ea8-2355-4b51-b736-448a89aba131\") " pod="openshift-multus/network-metrics-daemon-zxpwj"
Sep 10 15:26:56.921564 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:56.919972    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/c1a148df-5bb2-4059-9d8a-2715cc0b6f8c-host-cni-bin\") pod \"ovnkube-node-bctsx\" (UID: \"c1a148df-5bb2-4059-9d8a-2715cc0b6f8c\") " pod="openshift-ovn-kubernetes/ovnkube-node-bctsx"
Sep 10 15:26:56.921564 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:56.919974    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/1b59aadf-17c2-49a9-90e4-c507ca7d21c2-multus-daemon-config\") pod \"multus-gqngc\" (UID: \"1b59aadf-17c2-49a9-90e4-c507ca7d21c2\") " pod="openshift-multus/multus-gqngc"
Sep 10 15:26:56.921564 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:56.919998    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/831261f1-1d19-42bd-80c0-5173a7132c38-etc-selinux\") pod \"aws-ebs-csi-driver-node-krx2c\" (UID: \"831261f1-1d19-42bd-80c0-5173a7132c38\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-krx2c"
Sep 10 15:26:56.921564 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:56.920030    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/1b59aadf-17c2-49a9-90e4-c507ca7d21c2-host-var-lib-cni-bin\") pod \"multus-gqngc\" (UID: \"1b59aadf-17c2-49a9-90e4-c507ca7d21c2\") " pod="openshift-multus/multus-gqngc"
Sep 10 15:26:56.921564 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:56.920056    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/c1a148df-5bb2-4059-9d8a-2715cc0b6f8c-systemd-units\") pod \"ovnkube-node-bctsx\" (UID: \"c1a148df-5bb2-4059-9d8a-2715cc0b6f8c\") " pod="openshift-ovn-kubernetes/ovnkube-node-bctsx"
Sep 10 15:26:56.921564 ip-10-0-128-241 kubenswrapper[2567]: E0910 15:26:56.920076    2567 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 15:26:56.921564 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:56.920080    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/c1a148df-5bb2-4059-9d8a-2715cc0b6f8c-run-systemd\") pod \"ovnkube-node-bctsx\" (UID: \"c1a148df-5bb2-4059-9d8a-2715cc0b6f8c\") " pod="openshift-ovn-kubernetes/ovnkube-node-bctsx"
Sep 10 15:26:56.921564 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:56.920136    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/1b59aadf-17c2-49a9-90e4-c507ca7d21c2-host-var-lib-cni-bin\") pod \"multus-gqngc\" (UID: \"1b59aadf-17c2-49a9-90e4-c507ca7d21c2\") " pod="openshift-multus/multus-gqngc"
Sep 10 15:26:56.921564 ip-10-0-128-241 kubenswrapper[2567]: E0910 15:26:56.920152    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f91d1ea8-2355-4b51-b736-448a89aba131-metrics-certs podName:f91d1ea8-2355-4b51-b736-448a89aba131 nodeName:}" failed. No retries permitted until 2026-09-10 15:26:57.420130073 +0000 UTC m=+2.988361801 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f91d1ea8-2355-4b51-b736-448a89aba131-metrics-certs") pod "network-metrics-daemon-zxpwj" (UID: "f91d1ea8-2355-4b51-b736-448a89aba131") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 15:26:56.921564 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:56.920194    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/c1a148df-5bb2-4059-9d8a-2715cc0b6f8c-log-socket\") pod \"ovnkube-node-bctsx\" (UID: \"c1a148df-5bb2-4059-9d8a-2715cc0b6f8c\") " pod="openshift-ovn-kubernetes/ovnkube-node-bctsx"
Sep 10 15:26:56.921564 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:56.920220    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/c1a148df-5bb2-4059-9d8a-2715cc0b6f8c-env-overrides\") pod \"ovnkube-node-bctsx\" (UID: \"c1a148df-5bb2-4059-9d8a-2715cc0b6f8c\") " pod="openshift-ovn-kubernetes/ovnkube-node-bctsx"
Sep 10 15:26:56.922533 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:56.920244    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/aa974d12-9161-4bc9-a1c7-0846241a3206-konnectivity-ca\") pod \"konnectivity-agent-pzrzz\" (UID: \"aa974d12-9161-4bc9-a1c7-0846241a3206\") " pod="kube-system/konnectivity-agent-pzrzz"
Sep 10 15:26:56.922533 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:56.920285    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/ccbffcf4-9a0d-445b-b05d-3bcf97e1866d-etc-sysctl-d\") pod \"tuned-xt7qs\" (UID: \"ccbffcf4-9a0d-445b-b05d-3bcf97e1866d\") " pod="openshift-cluster-node-tuning-operator/tuned-xt7qs"
Sep 10 15:26:56.922533 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:56.920310    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ccbffcf4-9a0d-445b-b05d-3bcf97e1866d-var-lib-kubelet\") pod \"tuned-xt7qs\" (UID: \"ccbffcf4-9a0d-445b-b05d-3bcf97e1866d\") " pod="openshift-cluster-node-tuning-operator/tuned-xt7qs"
Sep 10 15:26:56.922533 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:56.920335    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b44e8918-d279-456a-b0cc-63526a3947c2-cnibin\") pod \"multus-additional-cni-plugins-82drl\" (UID: \"b44e8918-d279-456a-b0cc-63526a3947c2\") " pod="openshift-multus/multus-additional-cni-plugins-82drl"
Sep 10 15:26:56.922533 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:56.920360    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b44e8918-d279-456a-b0cc-63526a3947c2-tuning-conf-dir\") pod \"multus-additional-cni-plugins-82drl\" (UID: \"b44e8918-d279-456a-b0cc-63526a3947c2\") " pod="openshift-multus/multus-additional-cni-plugins-82drl"
Sep 10 15:26:56.922533 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:56.920386    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/b44e8918-d279-456a-b0cc-63526a3947c2-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-82drl\" (UID: \"b44e8918-d279-456a-b0cc-63526a3947c2\") " pod="openshift-multus/multus-additional-cni-plugins-82drl"
Sep 10 15:26:56.922533 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:56.920429    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/831261f1-1d19-42bd-80c0-5173a7132c38-socket-dir\") pod \"aws-ebs-csi-driver-node-krx2c\" (UID: \"831261f1-1d19-42bd-80c0-5173a7132c38\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-krx2c"
Sep 10 15:26:56.922533 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:56.920461    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xxd9z\" (UniqueName: \"kubernetes.io/projected/831261f1-1d19-42bd-80c0-5173a7132c38-kube-api-access-xxd9z\") pod \"aws-ebs-csi-driver-node-krx2c\" (UID: \"831261f1-1d19-42bd-80c0-5173a7132c38\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-krx2c"
Sep 10 15:26:56.922533 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:56.920497    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/1b59aadf-17c2-49a9-90e4-c507ca7d21c2-cni-binary-copy\") pod \"multus-gqngc\" (UID: \"1b59aadf-17c2-49a9-90e4-c507ca7d21c2\") " pod="openshift-multus/multus-gqngc"
Sep 10 15:26:56.922533 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:56.920535    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1b59aadf-17c2-49a9-90e4-c507ca7d21c2-host-var-lib-kubelet\") pod \"multus-gqngc\" (UID: \"1b59aadf-17c2-49a9-90e4-c507ca7d21c2\") " pod="openshift-multus/multus-gqngc"
Sep 10 15:26:56.922533 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:56.920562    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2j2dm\" (UniqueName: \"kubernetes.io/projected/f91d1ea8-2355-4b51-b736-448a89aba131-kube-api-access-2j2dm\") pod \"network-metrics-daemon-zxpwj\" (UID: \"f91d1ea8-2355-4b51-b736-448a89aba131\") " pod="openshift-multus/network-metrics-daemon-zxpwj"
Sep 10 15:26:56.922533 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:56.920844    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1b59aadf-17c2-49a9-90e4-c507ca7d21c2-host-var-lib-kubelet\") pod \"multus-gqngc\" (UID: \"1b59aadf-17c2-49a9-90e4-c507ca7d21c2\") " pod="openshift-multus/multus-gqngc"
Sep 10 15:26:56.922533 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:56.920964    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/1b59aadf-17c2-49a9-90e4-c507ca7d21c2-cni-binary-copy\") pod \"multus-gqngc\" (UID: \"1b59aadf-17c2-49a9-90e4-c507ca7d21c2\") " pod="openshift-multus/multus-gqngc"
Sep 10 15:26:56.927909 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:56.927882    2567 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Sep 10 15:26:56.933408 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:56.933171    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bfpq4\" (UniqueName: \"kubernetes.io/projected/b2e911b2-e1d3-4eb1-85b8-61f6a12bd9ba-kube-api-access-bfpq4\") pod \"iptables-alerter-9b7ts\" (UID: \"b2e911b2-e1d3-4eb1-85b8-61f6a12bd9ba\") " pod="openshift-network-operator/iptables-alerter-9b7ts"
Sep 10 15:26:56.933512 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:56.933188    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2j2dm\" (UniqueName: \"kubernetes.io/projected/f91d1ea8-2355-4b51-b736-448a89aba131-kube-api-access-2j2dm\") pod \"network-metrics-daemon-zxpwj\" (UID: \"f91d1ea8-2355-4b51-b736-448a89aba131\") " pod="openshift-multus/network-metrics-daemon-zxpwj"
Sep 10 15:26:56.934033 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:56.933988    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5cscg\" (UniqueName: \"kubernetes.io/projected/1b59aadf-17c2-49a9-90e4-c507ca7d21c2-kube-api-access-5cscg\") pod \"multus-gqngc\" (UID: \"1b59aadf-17c2-49a9-90e4-c507ca7d21c2\") " pod="openshift-multus/multus-gqngc"
Sep 10 15:26:57.021780 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:57.021746    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/b44e8918-d279-456a-b0cc-63526a3947c2-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-82drl\" (UID: \"b44e8918-d279-456a-b0cc-63526a3947c2\") " pod="openshift-multus/multus-additional-cni-plugins-82drl"
Sep 10 15:26:57.021958 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:57.021792    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ccbffcf4-9a0d-445b-b05d-3bcf97e1866d-etc-kubernetes\") pod \"tuned-xt7qs\" (UID: \"ccbffcf4-9a0d-445b-b05d-3bcf97e1866d\") " pod="openshift-cluster-node-tuning-operator/tuned-xt7qs"
Sep 10 15:26:57.021958 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:57.021819    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/ccbffcf4-9a0d-445b-b05d-3bcf97e1866d-run\") pod \"tuned-xt7qs\" (UID: \"ccbffcf4-9a0d-445b-b05d-3bcf97e1866d\") " pod="openshift-cluster-node-tuning-operator/tuned-xt7qs"
Sep 10 15:26:57.021958 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:57.021863    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ccbffcf4-9a0d-445b-b05d-3bcf97e1866d-tmp\") pod \"tuned-xt7qs\" (UID: \"ccbffcf4-9a0d-445b-b05d-3bcf97e1866d\") " pod="openshift-cluster-node-tuning-operator/tuned-xt7qs"
Sep 10 15:26:57.021958 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:57.021888    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/49bbf894-04df-427c-b71a-c108fbf25054-kubelet-config\") pod \"global-pull-secret-syncer-fgx9b\" (UID: \"49bbf894-04df-427c-b71a-c108fbf25054\") " pod="kube-system/global-pull-secret-syncer-fgx9b"
Sep 10 15:26:57.021958 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:57.021913    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b44e8918-d279-456a-b0cc-63526a3947c2-cni-binary-copy\") pod \"multus-additional-cni-plugins-82drl\" (UID: \"b44e8918-d279-456a-b0cc-63526a3947c2\") " pod="openshift-multus/multus-additional-cni-plugins-82drl"
Sep 10 15:26:57.021958 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:57.021913    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ccbffcf4-9a0d-445b-b05d-3bcf97e1866d-etc-kubernetes\") pod \"tuned-xt7qs\" (UID: \"ccbffcf4-9a0d-445b-b05d-3bcf97e1866d\") " pod="openshift-cluster-node-tuning-operator/tuned-xt7qs"
Sep 10 15:26:57.021958 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:57.021943    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/ccbffcf4-9a0d-445b-b05d-3bcf97e1866d-etc-sysctl-conf\") pod \"tuned-xt7qs\" (UID: \"ccbffcf4-9a0d-445b-b05d-3bcf97e1866d\") " pod="openshift-cluster-node-tuning-operator/tuned-xt7qs"
Sep 10 15:26:57.022361 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:57.021968    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/ccbffcf4-9a0d-445b-b05d-3bcf97e1866d-etc-systemd\") pod \"tuned-xt7qs\" (UID: \"ccbffcf4-9a0d-445b-b05d-3bcf97e1866d\") " pod="openshift-cluster-node-tuning-operator/tuned-xt7qs"
Sep 10 15:26:57.022361 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:57.021988    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/ccbffcf4-9a0d-445b-b05d-3bcf97e1866d-etc-tuned\") pod \"tuned-xt7qs\" (UID: \"ccbffcf4-9a0d-445b-b05d-3bcf97e1866d\") " pod="openshift-cluster-node-tuning-operator/tuned-xt7qs"
Sep 10 15:26:57.022361 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:57.022027    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/aa974d12-9161-4bc9-a1c7-0846241a3206-agent-certs\") pod \"konnectivity-agent-pzrzz\" (UID: \"aa974d12-9161-4bc9-a1c7-0846241a3206\") " pod="kube-system/konnectivity-agent-pzrzz"
Sep 10 15:26:57.022361 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:57.022048    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ccbffcf4-9a0d-445b-b05d-3bcf97e1866d-sys\") pod \"tuned-xt7qs\" (UID: \"ccbffcf4-9a0d-445b-b05d-3bcf97e1866d\") " pod="openshift-cluster-node-tuning-operator/tuned-xt7qs"
Sep 10 15:26:57.022361 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:57.022064    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/ccbffcf4-9a0d-445b-b05d-3bcf97e1866d-run\") pod \"tuned-xt7qs\" (UID: \"ccbffcf4-9a0d-445b-b05d-3bcf97e1866d\") " pod="openshift-cluster-node-tuning-operator/tuned-xt7qs"
Sep 10 15:26:57.022361 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:57.022076    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/c1a148df-5bb2-4059-9d8a-2715cc0b6f8c-host-slash\") pod \"ovnkube-node-bctsx\" (UID: \"c1a148df-5bb2-4059-9d8a-2715cc0b6f8c\") " pod="openshift-ovn-kubernetes/ovnkube-node-bctsx"
Sep 10 15:26:57.022361 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:57.022104    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c1a148df-5bb2-4059-9d8a-2715cc0b6f8c-var-lib-openvswitch\") pod \"ovnkube-node-bctsx\" (UID: \"c1a148df-5bb2-4059-9d8a-2715cc0b6f8c\") " pod="openshift-ovn-kubernetes/ovnkube-node-bctsx"
Sep 10 15:26:57.022361 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:57.022119    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/ccbffcf4-9a0d-445b-b05d-3bcf97e1866d-etc-systemd\") pod \"tuned-xt7qs\" (UID: \"ccbffcf4-9a0d-445b-b05d-3bcf97e1866d\") " pod="openshift-cluster-node-tuning-operator/tuned-xt7qs"
Sep 10 15:26:57.022361 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:57.022130    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/c1a148df-5bb2-4059-9d8a-2715cc0b6f8c-node-log\") pod \"ovnkube-node-bctsx\" (UID: \"c1a148df-5bb2-4059-9d8a-2715cc0b6f8c\") " pod="openshift-ovn-kubernetes/ovnkube-node-bctsx"
Sep 10 15:26:57.022361 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:57.022155    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/c1a148df-5bb2-4059-9d8a-2715cc0b6f8c-host-cni-netd\") pod \"ovnkube-node-bctsx\" (UID: \"c1a148df-5bb2-4059-9d8a-2715cc0b6f8c\") " pod="openshift-ovn-kubernetes/ovnkube-node-bctsx"
Sep 10 15:26:57.022361 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:57.022172    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/49bbf894-04df-427c-b71a-c108fbf25054-kubelet-config\") pod \"global-pull-secret-syncer-fgx9b\" (UID: \"49bbf894-04df-427c-b71a-c108fbf25054\") " pod="kube-system/global-pull-secret-syncer-fgx9b"
Sep 10 15:26:57.022361 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:57.022182    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c1a148df-5bb2-4059-9d8a-2715cc0b6f8c-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-bctsx\" (UID: \"c1a148df-5bb2-4059-9d8a-2715cc0b6f8c\") " pod="openshift-ovn-kubernetes/ovnkube-node-bctsx"
Sep 10 15:26:57.022361 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:57.022208    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/c1a148df-5bb2-4059-9d8a-2715cc0b6f8c-ovnkube-config\") pod \"ovnkube-node-bctsx\" (UID: \"c1a148df-5bb2-4059-9d8a-2715cc0b6f8c\") " pod="openshift-ovn-kubernetes/ovnkube-node-bctsx"
Sep 10 15:26:57.022361 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:57.022233    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/c1a148df-5bb2-4059-9d8a-2715cc0b6f8c-host-kubelet\") pod \"ovnkube-node-bctsx\" (UID: \"c1a148df-5bb2-4059-9d8a-2715cc0b6f8c\") " pod="openshift-ovn-kubernetes/ovnkube-node-bctsx"
Sep 10 15:26:57.022361 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:57.022258    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/c1a148df-5bb2-4059-9d8a-2715cc0b6f8c-ovnkube-script-lib\") pod \"ovnkube-node-bctsx\" (UID: \"c1a148df-5bb2-4059-9d8a-2715cc0b6f8c\") " pod="openshift-ovn-kubernetes/ovnkube-node-bctsx"
Sep 10 15:26:57.022361 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:57.022303    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/4dd1b47c-e497-45d2-a404-4ea3a2d8ec94-hosts-file\") pod \"node-resolver-rctzd\" (UID: \"4dd1b47c-e497-45d2-a404-4ea3a2d8ec94\") " pod="openshift-dns/node-resolver-rctzd"
Sep 10 15:26:57.022361 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:57.022331    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/e9b09c08-aec9-41db-8386-e62731150ccc-host\") pod \"node-ca-g8rkv\" (UID: \"e9b09c08-aec9-41db-8386-e62731150ccc\") " pod="openshift-image-registry/node-ca-g8rkv"
Sep 10 15:26:57.023230 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:57.022336    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/b44e8918-d279-456a-b0cc-63526a3947c2-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-82drl\" (UID: \"b44e8918-d279-456a-b0cc-63526a3947c2\") " pod="openshift-multus/multus-additional-cni-plugins-82drl"
Sep 10 15:26:57.023230 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:57.022358    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/e9b09c08-aec9-41db-8386-e62731150ccc-serviceca\") pod \"node-ca-g8rkv\" (UID: \"e9b09c08-aec9-41db-8386-e62731150ccc\") " pod="openshift-image-registry/node-ca-g8rkv"
Sep 10 15:26:57.023230 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:57.022388    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/c1a148df-5bb2-4059-9d8a-2715cc0b6f8c-node-log\") pod \"ovnkube-node-bctsx\" (UID: \"c1a148df-5bb2-4059-9d8a-2715cc0b6f8c\") " pod="openshift-ovn-kubernetes/ovnkube-node-bctsx"
Sep 10 15:26:57.023230 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:57.022400    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/c1a148df-5bb2-4059-9d8a-2715cc0b6f8c-host-cni-bin\") pod \"ovnkube-node-bctsx\" (UID: \"c1a148df-5bb2-4059-9d8a-2715cc0b6f8c\") " pod="openshift-ovn-kubernetes/ovnkube-node-bctsx"
Sep 10 15:26:57.023230 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:57.022407    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/ccbffcf4-9a0d-445b-b05d-3bcf97e1866d-etc-sysctl-conf\") pod \"tuned-xt7qs\" (UID: \"ccbffcf4-9a0d-445b-b05d-3bcf97e1866d\") " pod="openshift-cluster-node-tuning-operator/tuned-xt7qs"
Sep 10 15:26:57.023230 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:57.022428    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/4dd1b47c-e497-45d2-a404-4ea3a2d8ec94-hosts-file\") pod \"node-resolver-rctzd\" (UID: \"4dd1b47c-e497-45d2-a404-4ea3a2d8ec94\") " pod="openshift-dns/node-resolver-rctzd"
Sep 10 15:26:57.023230 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:57.022428    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/c1a148df-5bb2-4059-9d8a-2715cc0b6f8c-host-cni-netd\") pod \"ovnkube-node-bctsx\" (UID: \"c1a148df-5bb2-4059-9d8a-2715cc0b6f8c\") " pod="openshift-ovn-kubernetes/ovnkube-node-bctsx"
Sep 10 15:26:57.023230 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:57.022442    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/c1a148df-5bb2-4059-9d8a-2715cc0b6f8c-host-cni-bin\") pod \"ovnkube-node-bctsx\" (UID: \"c1a148df-5bb2-4059-9d8a-2715cc0b6f8c\") " pod="openshift-ovn-kubernetes/ovnkube-node-bctsx"
Sep 10 15:26:57.023230 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:57.022448    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/831261f1-1d19-42bd-80c0-5173a7132c38-etc-selinux\") pod \"aws-ebs-csi-driver-node-krx2c\" (UID: \"831261f1-1d19-42bd-80c0-5173a7132c38\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-krx2c"
Sep 10 15:26:57.023230 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:57.022479    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/e9b09c08-aec9-41db-8386-e62731150ccc-host\") pod \"node-ca-g8rkv\" (UID: \"e9b09c08-aec9-41db-8386-e62731150ccc\") " pod="openshift-image-registry/node-ca-g8rkv"
Sep 10 15:26:57.023230 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:57.022493    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/c1a148df-5bb2-4059-9d8a-2715cc0b6f8c-systemd-units\") pod \"ovnkube-node-bctsx\" (UID: \"c1a148df-5bb2-4059-9d8a-2715cc0b6f8c\") " pod="openshift-ovn-kubernetes/ovnkube-node-bctsx"
Sep 10 15:26:57.023230 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:57.022524    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/c1a148df-5bb2-4059-9d8a-2715cc0b6f8c-run-systemd\") pod \"ovnkube-node-bctsx\" (UID: \"c1a148df-5bb2-4059-9d8a-2715cc0b6f8c\") " pod="openshift-ovn-kubernetes/ovnkube-node-bctsx"
Sep 10 15:26:57.023230 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:57.022539    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ccbffcf4-9a0d-445b-b05d-3bcf97e1866d-sys\") pod \"tuned-xt7qs\" (UID: \"ccbffcf4-9a0d-445b-b05d-3bcf97e1866d\") " pod="openshift-cluster-node-tuning-operator/tuned-xt7qs"
Sep 10 15:26:57.023230 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:57.022550    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/c1a148df-5bb2-4059-9d8a-2715cc0b6f8c-log-socket\") pod \"ovnkube-node-bctsx\" (UID: \"c1a148df-5bb2-4059-9d8a-2715cc0b6f8c\") " pod="openshift-ovn-kubernetes/ovnkube-node-bctsx"
Sep 10 15:26:57.023230 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:57.022577    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/c1a148df-5bb2-4059-9d8a-2715cc0b6f8c-env-overrides\") pod \"ovnkube-node-bctsx\" (UID: \"c1a148df-5bb2-4059-9d8a-2715cc0b6f8c\") " pod="openshift-ovn-kubernetes/ovnkube-node-bctsx"
Sep 10 15:26:57.023230 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:57.022585    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/c1a148df-5bb2-4059-9d8a-2715cc0b6f8c-systemd-units\") pod \"ovnkube-node-bctsx\" (UID: \"c1a148df-5bb2-4059-9d8a-2715cc0b6f8c\") " pod="openshift-ovn-kubernetes/ovnkube-node-bctsx"
Sep 10 15:26:57.023230 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:57.022600    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/c1a148df-5bb2-4059-9d8a-2715cc0b6f8c-host-slash\") pod \"ovnkube-node-bctsx\" (UID: \"c1a148df-5bb2-4059-9d8a-2715cc0b6f8c\") " pod="openshift-ovn-kubernetes/ovnkube-node-bctsx"
Sep 10 15:26:57.023230 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:57.022603    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/aa974d12-9161-4bc9-a1c7-0846241a3206-konnectivity-ca\") pod \"konnectivity-agent-pzrzz\" (UID: \"aa974d12-9161-4bc9-a1c7-0846241a3206\") " pod="kube-system/konnectivity-agent-pzrzz"
Sep 10 15:26:57.024154 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:57.022629    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/c1a148df-5bb2-4059-9d8a-2715cc0b6f8c-host-kubelet\") pod \"ovnkube-node-bctsx\" (UID: \"c1a148df-5bb2-4059-9d8a-2715cc0b6f8c\") " pod="openshift-ovn-kubernetes/ovnkube-node-bctsx"
Sep 10 15:26:57.024154 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:57.022637    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/ccbffcf4-9a0d-445b-b05d-3bcf97e1866d-etc-sysctl-d\") pod \"tuned-xt7qs\" (UID: \"ccbffcf4-9a0d-445b-b05d-3bcf97e1866d\") " pod="openshift-cluster-node-tuning-operator/tuned-xt7qs"
Sep 10 15:26:57.024154 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:57.022664    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ccbffcf4-9a0d-445b-b05d-3bcf97e1866d-var-lib-kubelet\") pod \"tuned-xt7qs\" (UID: \"ccbffcf4-9a0d-445b-b05d-3bcf97e1866d\") " pod="openshift-cluster-node-tuning-operator/tuned-xt7qs"
Sep 10 15:26:57.024154 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:57.022692    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b44e8918-d279-456a-b0cc-63526a3947c2-cni-binary-copy\") pod \"multus-additional-cni-plugins-82drl\" (UID: \"b44e8918-d279-456a-b0cc-63526a3947c2\") " pod="openshift-multus/multus-additional-cni-plugins-82drl"
Sep 10 15:26:57.024154 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:57.022694    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b44e8918-d279-456a-b0cc-63526a3947c2-cnibin\") pod \"multus-additional-cni-plugins-82drl\" (UID: \"b44e8918-d279-456a-b0cc-63526a3947c2\") " pod="openshift-multus/multus-additional-cni-plugins-82drl"
Sep 10 15:26:57.024154 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:57.022726    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b44e8918-d279-456a-b0cc-63526a3947c2-cnibin\") pod \"multus-additional-cni-plugins-82drl\" (UID: \"b44e8918-d279-456a-b0cc-63526a3947c2\") " pod="openshift-multus/multus-additional-cni-plugins-82drl"
Sep 10 15:26:57.024154 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:57.022746    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b44e8918-d279-456a-b0cc-63526a3947c2-tuning-conf-dir\") pod \"multus-additional-cni-plugins-82drl\" (UID: \"b44e8918-d279-456a-b0cc-63526a3947c2\") " pod="openshift-multus/multus-additional-cni-plugins-82drl"
Sep 10 15:26:57.024154 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:57.022776    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c1a148df-5bb2-4059-9d8a-2715cc0b6f8c-var-lib-openvswitch\") pod \"ovnkube-node-bctsx\" (UID: \"c1a148df-5bb2-4059-9d8a-2715cc0b6f8c\") " pod="openshift-ovn-kubernetes/ovnkube-node-bctsx"
Sep 10 15:26:57.024154 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:57.022777    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/b44e8918-d279-456a-b0cc-63526a3947c2-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-82drl\" (UID: \"b44e8918-d279-456a-b0cc-63526a3947c2\") " pod="openshift-multus/multus-additional-cni-plugins-82drl"
Sep 10 15:26:57.024154 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:57.022807    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/831261f1-1d19-42bd-80c0-5173a7132c38-socket-dir\") pod \"aws-ebs-csi-driver-node-krx2c\" (UID: \"831261f1-1d19-42bd-80c0-5173a7132c38\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-krx2c"
Sep 10 15:26:57.024154 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:57.022832    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xxd9z\" (UniqueName: \"kubernetes.io/projected/831261f1-1d19-42bd-80c0-5173a7132c38-kube-api-access-xxd9z\") pod \"aws-ebs-csi-driver-node-krx2c\" (UID: \"831261f1-1d19-42bd-80c0-5173a7132c38\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-krx2c"
Sep 10 15:26:57.024154 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:57.022865    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/ccbffcf4-9a0d-445b-b05d-3bcf97e1866d-etc-modprobe-d\") pod \"tuned-xt7qs\" (UID: \"ccbffcf4-9a0d-445b-b05d-3bcf97e1866d\") " pod="openshift-cluster-node-tuning-operator/tuned-xt7qs"
Sep 10 15:26:57.024154 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:57.022875    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/ccbffcf4-9a0d-445b-b05d-3bcf97e1866d-etc-sysctl-d\") pod \"tuned-xt7qs\" (UID: \"ccbffcf4-9a0d-445b-b05d-3bcf97e1866d\") " pod="openshift-cluster-node-tuning-operator/tuned-xt7qs"
Sep 10 15:26:57.024154 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:57.022890    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/831261f1-1d19-42bd-80c0-5173a7132c38-registration-dir\") pod \"aws-ebs-csi-driver-node-krx2c\" (UID: \"831261f1-1d19-42bd-80c0-5173a7132c38\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-krx2c"
Sep 10 15:26:57.024154 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:57.022923    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c1a148df-5bb2-4059-9d8a-2715cc0b6f8c-host-run-ovn-kubernetes\") pod \"ovnkube-node-bctsx\" (UID: \"c1a148df-5bb2-4059-9d8a-2715cc0b6f8c\") " pod="openshift-ovn-kubernetes/ovnkube-node-bctsx"
Sep 10 15:26:57.024154 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:57.022494    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c1a148df-5bb2-4059-9d8a-2715cc0b6f8c-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-bctsx\" (UID: \"c1a148df-5bb2-4059-9d8a-2715cc0b6f8c\") " pod="openshift-ovn-kubernetes/ovnkube-node-bctsx"
Sep 10 15:26:57.024154 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:57.022926    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/e9b09c08-aec9-41db-8386-e62731150ccc-serviceca\") pod \"node-ca-g8rkv\" (UID: \"e9b09c08-aec9-41db-8386-e62731150ccc\") " pod="openshift-image-registry/node-ca-g8rkv"
Sep 10 15:26:57.025103 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:57.022949    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/49bbf894-04df-427c-b71a-c108fbf25054-dbus\") pod \"global-pull-secret-syncer-fgx9b\" (UID: \"49bbf894-04df-427c-b71a-c108fbf25054\") " pod="kube-system/global-pull-secret-syncer-fgx9b"
Sep 10 15:26:57.025103 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:57.022976    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/831261f1-1d19-42bd-80c0-5173a7132c38-sys-fs\") pod \"aws-ebs-csi-driver-node-krx2c\" (UID: \"831261f1-1d19-42bd-80c0-5173a7132c38\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-krx2c"
Sep 10 15:26:57.025103 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:57.022992    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/c1a148df-5bb2-4059-9d8a-2715cc0b6f8c-run-systemd\") pod \"ovnkube-node-bctsx\" (UID: \"c1a148df-5bb2-4059-9d8a-2715cc0b6f8c\") " pod="openshift-ovn-kubernetes/ovnkube-node-bctsx"
Sep 10 15:26:57.025103 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:57.023000    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/c1a148df-5bb2-4059-9d8a-2715cc0b6f8c-ovn-node-metrics-cert\") pod \"ovnkube-node-bctsx\" (UID: \"c1a148df-5bb2-4059-9d8a-2715cc0b6f8c\") " pod="openshift-ovn-kubernetes/ovnkube-node-bctsx"
Sep 10 15:26:57.025103 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:57.023027    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fj2pp\" (UniqueName: \"kubernetes.io/projected/4dd1b47c-e497-45d2-a404-4ea3a2d8ec94-kube-api-access-fj2pp\") pod \"node-resolver-rctzd\" (UID: \"4dd1b47c-e497-45d2-a404-4ea3a2d8ec94\") " pod="openshift-dns/node-resolver-rctzd"
Sep 10 15:26:57.025103 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:57.023078    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b44e8918-d279-456a-b0cc-63526a3947c2-system-cni-dir\") pod \"multus-additional-cni-plugins-82drl\" (UID: \"b44e8918-d279-456a-b0cc-63526a3947c2\") " pod="openshift-multus/multus-additional-cni-plugins-82drl"
Sep 10 15:26:57.025103 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:57.023107    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2jrk5\" (UniqueName: \"kubernetes.io/projected/b44e8918-d279-456a-b0cc-63526a3947c2-kube-api-access-2jrk5\") pod \"multus-additional-cni-plugins-82drl\" (UID: \"b44e8918-d279-456a-b0cc-63526a3947c2\") " pod="openshift-multus/multus-additional-cni-plugins-82drl"
Sep 10 15:26:57.025103 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:57.023134    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/831261f1-1d19-42bd-80c0-5173a7132c38-device-dir\") pod \"aws-ebs-csi-driver-node-krx2c\" (UID: \"831261f1-1d19-42bd-80c0-5173a7132c38\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-krx2c"
Sep 10 15:26:57.025103 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:57.023150    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/aa974d12-9161-4bc9-a1c7-0846241a3206-konnectivity-ca\") pod \"konnectivity-agent-pzrzz\" (UID: \"aa974d12-9161-4bc9-a1c7-0846241a3206\") " pod="kube-system/konnectivity-agent-pzrzz"
Sep 10 15:26:57.025103 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:57.023165    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c1a148df-5bb2-4059-9d8a-2715cc0b6f8c-run-openvswitch\") pod \"ovnkube-node-bctsx\" (UID: \"c1a148df-5bb2-4059-9d8a-2715cc0b6f8c\") " pod="openshift-ovn-kubernetes/ovnkube-node-bctsx"
Sep 10 15:26:57.025103 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:57.023190    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/c1a148df-5bb2-4059-9d8a-2715cc0b6f8c-ovnkube-script-lib\") pod \"ovnkube-node-bctsx\" (UID: \"c1a148df-5bb2-4059-9d8a-2715cc0b6f8c\") " pod="openshift-ovn-kubernetes/ovnkube-node-bctsx"
Sep 10 15:26:57.025103 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:57.023204    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c1a148df-5bb2-4059-9d8a-2715cc0b6f8c-run-openvswitch\") pod \"ovnkube-node-bctsx\" (UID: \"c1a148df-5bb2-4059-9d8a-2715cc0b6f8c\") " pod="openshift-ovn-kubernetes/ovnkube-node-bctsx"
Sep 10 15:26:57.025103 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:57.023207    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c1a148df-5bb2-4059-9d8a-2715cc0b6f8c-etc-openvswitch\") pod \"ovnkube-node-bctsx\" (UID: \"c1a148df-5bb2-4059-9d8a-2715cc0b6f8c\") " pod="openshift-ovn-kubernetes/ovnkube-node-bctsx"
Sep 10 15:26:57.025103 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:57.023250    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c1a148df-5bb2-4059-9d8a-2715cc0b6f8c-etc-openvswitch\") pod \"ovnkube-node-bctsx\" (UID: \"c1a148df-5bb2-4059-9d8a-2715cc0b6f8c\") " pod="openshift-ovn-kubernetes/ovnkube-node-bctsx"
Sep 10 15:26:57.025103 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:57.023255    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/c1a148df-5bb2-4059-9d8a-2715cc0b6f8c-run-ovn\") pod \"ovnkube-node-bctsx\" (UID: \"c1a148df-5bb2-4059-9d8a-2715cc0b6f8c\") " pod="openshift-ovn-kubernetes/ovnkube-node-bctsx"
Sep 10 15:26:57.025103 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:57.023288    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hhvx7\" (UniqueName: \"kubernetes.io/projected/c1a148df-5bb2-4059-9d8a-2715cc0b6f8c-kube-api-access-hhvx7\") pod \"ovnkube-node-bctsx\" (UID: \"c1a148df-5bb2-4059-9d8a-2715cc0b6f8c\") " pod="openshift-ovn-kubernetes/ovnkube-node-bctsx"
Sep 10 15:26:57.025103 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:57.023293    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/c1a148df-5bb2-4059-9d8a-2715cc0b6f8c-run-ovn\") pod \"ovnkube-node-bctsx\" (UID: \"c1a148df-5bb2-4059-9d8a-2715cc0b6f8c\") " pod="openshift-ovn-kubernetes/ovnkube-node-bctsx"
Sep 10 15:26:57.025974 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:57.022550    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/831261f1-1d19-42bd-80c0-5173a7132c38-etc-selinux\") pod \"aws-ebs-csi-driver-node-krx2c\" (UID: \"831261f1-1d19-42bd-80c0-5173a7132c38\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-krx2c"
Sep 10 15:26:57.025974 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:57.023390    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/c1a148df-5bb2-4059-9d8a-2715cc0b6f8c-ovnkube-config\") pod \"ovnkube-node-bctsx\" (UID: \"c1a148df-5bb2-4059-9d8a-2715cc0b6f8c\") " pod="openshift-ovn-kubernetes/ovnkube-node-bctsx"
Sep 10 15:26:57.025974 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:57.023441    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/c1a148df-5bb2-4059-9d8a-2715cc0b6f8c-log-socket\") pod \"ovnkube-node-bctsx\" (UID: \"c1a148df-5bb2-4059-9d8a-2715cc0b6f8c\") " pod="openshift-ovn-kubernetes/ovnkube-node-bctsx"
Sep 10 15:26:57.025974 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:57.023485    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b44e8918-d279-456a-b0cc-63526a3947c2-tuning-conf-dir\") pod \"multus-additional-cni-plugins-82drl\" (UID: \"b44e8918-d279-456a-b0cc-63526a3947c2\") " pod="openshift-multus/multus-additional-cni-plugins-82drl"
Sep 10 15:26:57.025974 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:57.023502    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/49bbf894-04df-427c-b71a-c108fbf25054-dbus\") pod \"global-pull-secret-syncer-fgx9b\" (UID: \"49bbf894-04df-427c-b71a-c108fbf25054\") " pod="kube-system/global-pull-secret-syncer-fgx9b"
Sep 10 15:26:57.025974 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:57.023547    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/831261f1-1d19-42bd-80c0-5173a7132c38-sys-fs\") pod \"aws-ebs-csi-driver-node-krx2c\" (UID: \"831261f1-1d19-42bd-80c0-5173a7132c38\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-krx2c"
Sep 10 15:26:57.025974 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:57.023939    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/c1a148df-5bb2-4059-9d8a-2715cc0b6f8c-env-overrides\") pod \"ovnkube-node-bctsx\" (UID: \"c1a148df-5bb2-4059-9d8a-2715cc0b6f8c\") " pod="openshift-ovn-kubernetes/ovnkube-node-bctsx"
Sep 10 15:26:57.025974 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:57.023987    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/b44e8918-d279-456a-b0cc-63526a3947c2-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-82drl\" (UID: \"b44e8918-d279-456a-b0cc-63526a3947c2\") " pod="openshift-multus/multus-additional-cni-plugins-82drl"
Sep 10 15:26:57.025974 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:57.024057    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/ccbffcf4-9a0d-445b-b05d-3bcf97e1866d-etc-modprobe-d\") pod \"tuned-xt7qs\" (UID: \"ccbffcf4-9a0d-445b-b05d-3bcf97e1866d\") " pod="openshift-cluster-node-tuning-operator/tuned-xt7qs"
Sep 10 15:26:57.025974 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:57.024057    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/831261f1-1d19-42bd-80c0-5173a7132c38-registration-dir\") pod \"aws-ebs-csi-driver-node-krx2c\" (UID: \"831261f1-1d19-42bd-80c0-5173a7132c38\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-krx2c"
Sep 10 15:26:57.025974 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:57.024090    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c1a148df-5bb2-4059-9d8a-2715cc0b6f8c-host-run-ovn-kubernetes\") pod \"ovnkube-node-bctsx\" (UID: \"c1a148df-5bb2-4059-9d8a-2715cc0b6f8c\") " pod="openshift-ovn-kubernetes/ovnkube-node-bctsx"
Sep 10 15:26:57.025974 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:57.022924    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ccbffcf4-9a0d-445b-b05d-3bcf97e1866d-var-lib-kubelet\") pod \"tuned-xt7qs\" (UID: \"ccbffcf4-9a0d-445b-b05d-3bcf97e1866d\") " pod="openshift-cluster-node-tuning-operator/tuned-xt7qs"
Sep 10 15:26:57.025974 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:57.024175    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/831261f1-1d19-42bd-80c0-5173a7132c38-socket-dir\") pod \"aws-ebs-csi-driver-node-krx2c\" (UID: \"831261f1-1d19-42bd-80c0-5173a7132c38\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-krx2c"
Sep 10 15:26:57.025974 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:57.024224    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b44e8918-d279-456a-b0cc-63526a3947c2-system-cni-dir\") pod \"multus-additional-cni-plugins-82drl\" (UID: \"b44e8918-d279-456a-b0cc-63526a3947c2\") " pod="openshift-multus/multus-additional-cni-plugins-82drl"
Sep 10 15:26:57.025974 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:57.024247    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/831261f1-1d19-42bd-80c0-5173a7132c38-device-dir\") pod \"aws-ebs-csi-driver-node-krx2c\" (UID: \"831261f1-1d19-42bd-80c0-5173a7132c38\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-krx2c"
Sep 10 15:26:57.025974 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:57.024256    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/ccbffcf4-9a0d-445b-b05d-3bcf97e1866d-lib-modules\") pod \"tuned-xt7qs\" (UID: \"ccbffcf4-9a0d-445b-b05d-3bcf97e1866d\") " pod="openshift-cluster-node-tuning-operator/tuned-xt7qs"
Sep 10 15:26:57.025974 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:57.024282    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/ccbffcf4-9a0d-445b-b05d-3bcf97e1866d-etc-sysconfig\") pod \"tuned-xt7qs\" (UID: \"ccbffcf4-9a0d-445b-b05d-3bcf97e1866d\") " pod="openshift-cluster-node-tuning-operator/tuned-xt7qs"
Sep 10 15:26:57.026903 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:57.024324    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/ccbffcf4-9a0d-445b-b05d-3bcf97e1866d-host\") pod \"tuned-xt7qs\" (UID: \"ccbffcf4-9a0d-445b-b05d-3bcf97e1866d\") " pod="openshift-cluster-node-tuning-operator/tuned-xt7qs"
Sep 10 15:26:57.026903 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:57.024349    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4wmx2\" (UniqueName: \"kubernetes.io/projected/e105e176-14c2-4093-946d-492879428ef7-kube-api-access-4wmx2\") pod \"network-check-target-2f7vt\" (UID: \"e105e176-14c2-4093-946d-492879428ef7\") " pod="openshift-network-diagnostics/network-check-target-2f7vt"
Sep 10 15:26:57.026903 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:57.024376    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f6s58\" (UniqueName: \"kubernetes.io/projected/e9b09c08-aec9-41db-8386-e62731150ccc-kube-api-access-f6s58\") pod \"node-ca-g8rkv\" (UID: \"e9b09c08-aec9-41db-8386-e62731150ccc\") " pod="openshift-image-registry/node-ca-g8rkv"
Sep 10 15:26:57.026903 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:57.024381    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/ccbffcf4-9a0d-445b-b05d-3bcf97e1866d-lib-modules\") pod \"tuned-xt7qs\" (UID: \"ccbffcf4-9a0d-445b-b05d-3bcf97e1866d\") " pod="openshift-cluster-node-tuning-operator/tuned-xt7qs"
Sep 10 15:26:57.026903 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:57.024392    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/ccbffcf4-9a0d-445b-b05d-3bcf97e1866d-etc-sysconfig\") pod \"tuned-xt7qs\" (UID: \"ccbffcf4-9a0d-445b-b05d-3bcf97e1866d\") " pod="openshift-cluster-node-tuning-operator/tuned-xt7qs"
Sep 10 15:26:57.026903 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:57.024401    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/831261f1-1d19-42bd-80c0-5173a7132c38-kubelet-dir\") pod \"aws-ebs-csi-driver-node-krx2c\" (UID: \"831261f1-1d19-42bd-80c0-5173a7132c38\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-krx2c"
Sep 10 15:26:57.026903 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:57.024441    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/c1a148df-5bb2-4059-9d8a-2715cc0b6f8c-host-run-netns\") pod \"ovnkube-node-bctsx\" (UID: \"c1a148df-5bb2-4059-9d8a-2715cc0b6f8c\") " pod="openshift-ovn-kubernetes/ovnkube-node-bctsx"
Sep 10 15:26:57.026903 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:57.024471    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4dd1b47c-e497-45d2-a404-4ea3a2d8ec94-tmp-dir\") pod \"node-resolver-rctzd\" (UID: \"4dd1b47c-e497-45d2-a404-4ea3a2d8ec94\") " pod="openshift-dns/node-resolver-rctzd"
Sep 10 15:26:57.026903 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:57.024499    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fkjzd\" (UniqueName: \"kubernetes.io/projected/ccbffcf4-9a0d-445b-b05d-3bcf97e1866d-kube-api-access-fkjzd\") pod \"tuned-xt7qs\" (UID: \"ccbffcf4-9a0d-445b-b05d-3bcf97e1866d\") " pod="openshift-cluster-node-tuning-operator/tuned-xt7qs"
Sep 10 15:26:57.026903 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:57.024526    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/49bbf894-04df-427c-b71a-c108fbf25054-original-pull-secret\") pod \"global-pull-secret-syncer-fgx9b\" (UID: \"49bbf894-04df-427c-b71a-c108fbf25054\") " pod="kube-system/global-pull-secret-syncer-fgx9b"
Sep 10 15:26:57.026903 ip-10-0-128-241 kubenswrapper[2567]: E0910 15:26:57.024628    2567 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 10 15:26:57.026903 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:57.024661    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/ccbffcf4-9a0d-445b-b05d-3bcf97e1866d-host\") pod \"tuned-xt7qs\" (UID: \"ccbffcf4-9a0d-445b-b05d-3bcf97e1866d\") " pod="openshift-cluster-node-tuning-operator/tuned-xt7qs"
Sep 10 15:26:57.026903 ip-10-0-128-241 kubenswrapper[2567]: E0910 15:26:57.024708    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/49bbf894-04df-427c-b71a-c108fbf25054-original-pull-secret podName:49bbf894-04df-427c-b71a-c108fbf25054 nodeName:}" failed. No retries permitted until 2026-09-10 15:26:57.524688521 +0000 UTC m=+3.092920233 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/49bbf894-04df-427c-b71a-c108fbf25054-original-pull-secret") pod "global-pull-secret-syncer-fgx9b" (UID: "49bbf894-04df-427c-b71a-c108fbf25054") : object "kube-system"/"original-pull-secret" not registered
Sep 10 15:26:57.026903 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:57.024712    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/831261f1-1d19-42bd-80c0-5173a7132c38-kubelet-dir\") pod \"aws-ebs-csi-driver-node-krx2c\" (UID: \"831261f1-1d19-42bd-80c0-5173a7132c38\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-krx2c"
Sep 10 15:26:57.026903 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:57.024759    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/c1a148df-5bb2-4059-9d8a-2715cc0b6f8c-host-run-netns\") pod \"ovnkube-node-bctsx\" (UID: \"c1a148df-5bb2-4059-9d8a-2715cc0b6f8c\") " pod="openshift-ovn-kubernetes/ovnkube-node-bctsx"
Sep 10 15:26:57.026903 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:57.024773    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4dd1b47c-e497-45d2-a404-4ea3a2d8ec94-tmp-dir\") pod \"node-resolver-rctzd\" (UID: \"4dd1b47c-e497-45d2-a404-4ea3a2d8ec94\") " pod="openshift-dns/node-resolver-rctzd"
Sep 10 15:26:57.026903 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:57.026662    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/c1a148df-5bb2-4059-9d8a-2715cc0b6f8c-ovn-node-metrics-cert\") pod \"ovnkube-node-bctsx\" (UID: \"c1a148df-5bb2-4059-9d8a-2715cc0b6f8c\") " pod="openshift-ovn-kubernetes/ovnkube-node-bctsx"
Sep 10 15:26:57.027764 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:57.027558    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/ccbffcf4-9a0d-445b-b05d-3bcf97e1866d-etc-tuned\") pod \"tuned-xt7qs\" (UID: \"ccbffcf4-9a0d-445b-b05d-3bcf97e1866d\") " pod="openshift-cluster-node-tuning-operator/tuned-xt7qs"
Sep 10 15:26:57.027822 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:57.027781    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ccbffcf4-9a0d-445b-b05d-3bcf97e1866d-tmp\") pod \"tuned-xt7qs\" (UID: \"ccbffcf4-9a0d-445b-b05d-3bcf97e1866d\") " pod="openshift-cluster-node-tuning-operator/tuned-xt7qs"
Sep 10 15:26:57.029088 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:57.029042    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/aa974d12-9161-4bc9-a1c7-0846241a3206-agent-certs\") pod \"konnectivity-agent-pzrzz\" (UID: \"aa974d12-9161-4bc9-a1c7-0846241a3206\") " pod="kube-system/konnectivity-agent-pzrzz"
Sep 10 15:26:57.036024 ip-10-0-128-241 kubenswrapper[2567]: E0910 15:26:57.035791    2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 10 15:26:57.036024 ip-10-0-128-241 kubenswrapper[2567]: E0910 15:26:57.035816    2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 10 15:26:57.036024 ip-10-0-128-241 kubenswrapper[2567]: E0910 15:26:57.035832    2567 projected.go:196] Error preparing data for projected volume kube-api-access-4wmx2 for pod openshift-network-diagnostics/network-check-target-2f7vt: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 15:26:57.036024 ip-10-0-128-241 kubenswrapper[2567]: E0910 15:26:57.035904    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e105e176-14c2-4093-946d-492879428ef7-kube-api-access-4wmx2 podName:e105e176-14c2-4093-946d-492879428ef7 nodeName:}" failed. No retries permitted until 2026-09-10 15:26:57.535883929 +0000 UTC m=+3.104115660 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-4wmx2" (UniqueName: "kubernetes.io/projected/e105e176-14c2-4093-946d-492879428ef7-kube-api-access-4wmx2") pod "network-check-target-2f7vt" (UID: "e105e176-14c2-4093-946d-492879428ef7") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 15:26:57.044797 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:57.044496    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fkjzd\" (UniqueName: \"kubernetes.io/projected/ccbffcf4-9a0d-445b-b05d-3bcf97e1866d-kube-api-access-fkjzd\") pod \"tuned-xt7qs\" (UID: \"ccbffcf4-9a0d-445b-b05d-3bcf97e1866d\") " pod="openshift-cluster-node-tuning-operator/tuned-xt7qs"
Sep 10 15:26:57.044797 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:57.044644    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f6s58\" (UniqueName: \"kubernetes.io/projected/e9b09c08-aec9-41db-8386-e62731150ccc-kube-api-access-f6s58\") pod \"node-ca-g8rkv\" (UID: \"e9b09c08-aec9-41db-8386-e62731150ccc\") " pod="openshift-image-registry/node-ca-g8rkv"
Sep 10 15:26:57.044797 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:57.044696    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xxd9z\" (UniqueName: \"kubernetes.io/projected/831261f1-1d19-42bd-80c0-5173a7132c38-kube-api-access-xxd9z\") pod \"aws-ebs-csi-driver-node-krx2c\" (UID: \"831261f1-1d19-42bd-80c0-5173a7132c38\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-krx2c"
Sep 10 15:26:57.045015 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:57.044938    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fj2pp\" (UniqueName: \"kubernetes.io/projected/4dd1b47c-e497-45d2-a404-4ea3a2d8ec94-kube-api-access-fj2pp\") pod \"node-resolver-rctzd\" (UID: \"4dd1b47c-e497-45d2-a404-4ea3a2d8ec94\") " pod="openshift-dns/node-resolver-rctzd"
Sep 10 15:26:57.047554 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:57.045150    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2jrk5\" (UniqueName: \"kubernetes.io/projected/b44e8918-d279-456a-b0cc-63526a3947c2-kube-api-access-2jrk5\") pod \"multus-additional-cni-plugins-82drl\" (UID: \"b44e8918-d279-456a-b0cc-63526a3947c2\") " pod="openshift-multus/multus-additional-cni-plugins-82drl"
Sep 10 15:26:57.047554 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:57.045154    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hhvx7\" (UniqueName: \"kubernetes.io/projected/c1a148df-5bb2-4059-9d8a-2715cc0b6f8c-kube-api-access-hhvx7\") pod \"ovnkube-node-bctsx\" (UID: \"c1a148df-5bb2-4059-9d8a-2715cc0b6f8c\") " pod="openshift-ovn-kubernetes/ovnkube-node-bctsx"
Sep 10 15:26:57.116909 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:57.116856    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-gqngc"
Sep 10 15:26:57.125611 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:57.125584    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-9b7ts"
Sep 10 15:26:57.135094 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:57.135072    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-bctsx"
Sep 10 15:26:57.135828 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:57.135794    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1b59aadf_17c2_49a9_90e4_c507ca7d21c2.slice/crio-86981284bbcf51c9f1c55324e22e0a06a95a7168bf30a6e8fe3f97316a6c14fa WatchSource:0}: Error finding container 86981284bbcf51c9f1c55324e22e0a06a95a7168bf30a6e8fe3f97316a6c14fa: Status 404 returned error can't find the container with id 86981284bbcf51c9f1c55324e22e0a06a95a7168bf30a6e8fe3f97316a6c14fa
Sep 10 15:26:57.143268 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:57.143220    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-rctzd"
Sep 10 15:26:57.143834 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:57.143810    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb2e911b2_e1d3_4eb1_85b8_61f6a12bd9ba.slice/crio-e4c4dbb1551f473de44a0205a03f95e48f8fdea8288b37b21978d90332969a19 WatchSource:0}: Error finding container e4c4dbb1551f473de44a0205a03f95e48f8fdea8288b37b21978d90332969a19: Status 404 returned error can't find the container with id e4c4dbb1551f473de44a0205a03f95e48f8fdea8288b37b21978d90332969a19
Sep 10 15:26:57.147948 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:57.147918    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc1a148df_5bb2_4059_9d8a_2715cc0b6f8c.slice/crio-257b51bd5569a3af7bf471d2017f9de3d0322daafa7f343726feaaa8c8fa18f2 WatchSource:0}: Error finding container 257b51bd5569a3af7bf471d2017f9de3d0322daafa7f343726feaaa8c8fa18f2: Status 404 returned error can't find the container with id 257b51bd5569a3af7bf471d2017f9de3d0322daafa7f343726feaaa8c8fa18f2
Sep 10 15:26:57.151797 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:57.151776    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-pzrzz"
Sep 10 15:26:57.163123 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:57.163100    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-82drl"
Sep 10 15:26:57.172166 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:57.172147    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-xt7qs"
Sep 10 15:26:57.180979 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:57.180952    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-g8rkv"
Sep 10 15:26:57.188822 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:57.188763    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-krx2c"
Sep 10 15:26:57.428575 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:57.428538    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f91d1ea8-2355-4b51-b736-448a89aba131-metrics-certs\") pod \"network-metrics-daemon-zxpwj\" (UID: \"f91d1ea8-2355-4b51-b736-448a89aba131\") " pod="openshift-multus/network-metrics-daemon-zxpwj"
Sep 10 15:26:57.428757 ip-10-0-128-241 kubenswrapper[2567]: E0910 15:26:57.428703    2567 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 15:26:57.428823 ip-10-0-128-241 kubenswrapper[2567]: E0910 15:26:57.428784    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f91d1ea8-2355-4b51-b736-448a89aba131-metrics-certs podName:f91d1ea8-2355-4b51-b736-448a89aba131 nodeName:}" failed. No retries permitted until 2026-09-10 15:26:58.428765355 +0000 UTC m=+3.996997064 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f91d1ea8-2355-4b51-b736-448a89aba131-metrics-certs") pod "network-metrics-daemon-zxpwj" (UID: "f91d1ea8-2355-4b51-b736-448a89aba131") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 15:26:57.529612 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:57.529535    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/49bbf894-04df-427c-b71a-c108fbf25054-original-pull-secret\") pod \"global-pull-secret-syncer-fgx9b\" (UID: \"49bbf894-04df-427c-b71a-c108fbf25054\") " pod="kube-system/global-pull-secret-syncer-fgx9b"
Sep 10 15:26:57.529777 ip-10-0-128-241 kubenswrapper[2567]: E0910 15:26:57.529696    2567 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 10 15:26:57.529777 ip-10-0-128-241 kubenswrapper[2567]: E0910 15:26:57.529769    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/49bbf894-04df-427c-b71a-c108fbf25054-original-pull-secret podName:49bbf894-04df-427c-b71a-c108fbf25054 nodeName:}" failed. No retries permitted until 2026-09-10 15:26:58.529749111 +0000 UTC m=+4.097980836 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/49bbf894-04df-427c-b71a-c108fbf25054-original-pull-secret") pod "global-pull-secret-syncer-fgx9b" (UID: "49bbf894-04df-427c-b71a-c108fbf25054") : object "kube-system"/"original-pull-secret" not registered
Sep 10 15:26:57.630306 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:57.630265    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4wmx2\" (UniqueName: \"kubernetes.io/projected/e105e176-14c2-4093-946d-492879428ef7-kube-api-access-4wmx2\") pod \"network-check-target-2f7vt\" (UID: \"e105e176-14c2-4093-946d-492879428ef7\") " pod="openshift-network-diagnostics/network-check-target-2f7vt"
Sep 10 15:26:57.630482 ip-10-0-128-241 kubenswrapper[2567]: E0910 15:26:57.630417    2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 10 15:26:57.630482 ip-10-0-128-241 kubenswrapper[2567]: E0910 15:26:57.630434    2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 10 15:26:57.630482 ip-10-0-128-241 kubenswrapper[2567]: E0910 15:26:57.630444    2567 projected.go:196] Error preparing data for projected volume kube-api-access-4wmx2 for pod openshift-network-diagnostics/network-check-target-2f7vt: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 15:26:57.630668 ip-10-0-128-241 kubenswrapper[2567]: E0910 15:26:57.630500    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e105e176-14c2-4093-946d-492879428ef7-kube-api-access-4wmx2 podName:e105e176-14c2-4093-946d-492879428ef7 nodeName:}" failed. No retries permitted until 2026-09-10 15:26:58.630485744 +0000 UTC m=+4.198717458 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-4wmx2" (UniqueName: "kubernetes.io/projected/e105e176-14c2-4093-946d-492879428ef7-kube-api-access-4wmx2") pod "network-check-target-2f7vt" (UID: "e105e176-14c2-4093-946d-492879428ef7") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 15:26:57.857991 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:57.857894    2567 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-09 15:21:57 +0000 UTC" deadline="2028-02-12 09:12:34.919467567 +0000 UTC"
Sep 10 15:26:57.857991 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:57.857944    2567 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12473h45m37.061535263s"
Sep 10 15:26:57.898269 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:57.898235    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fgx9b"
Sep 10 15:26:57.898453 ip-10-0-128-241 kubenswrapper[2567]: E0910 15:26:57.898378    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fgx9b" podUID="49bbf894-04df-427c-b71a-c108fbf25054"
Sep 10 15:26:57.908478 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:57.908443    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-bctsx" event={"ID":"c1a148df-5bb2-4059-9d8a-2715cc0b6f8c","Type":"ContainerStarted","Data":"257b51bd5569a3af7bf471d2017f9de3d0322daafa7f343726feaaa8c8fa18f2"}
Sep 10 15:26:57.909828 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:57.909801    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-9b7ts" event={"ID":"b2e911b2-e1d3-4eb1-85b8-61f6a12bd9ba","Type":"ContainerStarted","Data":"e4c4dbb1551f473de44a0205a03f95e48f8fdea8288b37b21978d90332969a19"}
Sep 10 15:26:57.911124 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:57.911100    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-gqngc" event={"ID":"1b59aadf-17c2-49a9-90e4-c507ca7d21c2","Type":"ContainerStarted","Data":"86981284bbcf51c9f1c55324e22e0a06a95a7168bf30a6e8fe3f97316a6c14fa"}
Sep 10 15:26:58.224525 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:58.224493    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4dd1b47c_e497_45d2_a404_4ea3a2d8ec94.slice/crio-356f1d1e280a59f7230798448d32dea9a39b8e6dc5b17bad23b5536b04dd6898 WatchSource:0}: Error finding container 356f1d1e280a59f7230798448d32dea9a39b8e6dc5b17bad23b5536b04dd6898: Status 404 returned error can't find the container with id 356f1d1e280a59f7230798448d32dea9a39b8e6dc5b17bad23b5536b04dd6898
Sep 10 15:26:58.237460 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:58.237408    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podccbffcf4_9a0d_445b_b05d_3bcf97e1866d.slice/crio-4e9465b7ed71e36fddedd68ece5498121cf682f13eca5d1ae457838ba3dceb6f WatchSource:0}: Error finding container 4e9465b7ed71e36fddedd68ece5498121cf682f13eca5d1ae457838ba3dceb6f: Status 404 returned error can't find the container with id 4e9465b7ed71e36fddedd68ece5498121cf682f13eca5d1ae457838ba3dceb6f
Sep 10 15:26:58.239323 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:58.239286    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaa974d12_9161_4bc9_a1c7_0846241a3206.slice/crio-c20f11a538a5d25c7df71fae632ab91286079fe52ab710efcf29c5b4b4abfe52 WatchSource:0}: Error finding container c20f11a538a5d25c7df71fae632ab91286079fe52ab710efcf29c5b4b4abfe52: Status 404 returned error can't find the container with id c20f11a538a5d25c7df71fae632ab91286079fe52ab710efcf29c5b4b4abfe52
Sep 10 15:26:58.239534 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:58.239491    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod831261f1_1d19_42bd_80c0_5173a7132c38.slice/crio-55e6683ec55d1bff2673fe3b06baae8747ab050ee80d799f6f7ee2ca3b10742a WatchSource:0}: Error finding container 55e6683ec55d1bff2673fe3b06baae8747ab050ee80d799f6f7ee2ca3b10742a: Status 404 returned error can't find the container with id 55e6683ec55d1bff2673fe3b06baae8747ab050ee80d799f6f7ee2ca3b10742a
Sep 10 15:26:58.240880 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:58.240847    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb44e8918_d279_456a_b0cc_63526a3947c2.slice/crio-b5d79cbb6063a2d279efd87036e0ff954a3e42899c615b1179087b13061c960c WatchSource:0}: Error finding container b5d79cbb6063a2d279efd87036e0ff954a3e42899c615b1179087b13061c960c: Status 404 returned error can't find the container with id b5d79cbb6063a2d279efd87036e0ff954a3e42899c615b1179087b13061c960c
Sep 10 15:26:58.241677 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:26:58.241648    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode9b09c08_aec9_41db_8386_e62731150ccc.slice/crio-99ee803a58d9c559a20c638247293ba2e2c105af5c9648c149c7c0416f08c50d WatchSource:0}: Error finding container 99ee803a58d9c559a20c638247293ba2e2c105af5c9648c149c7c0416f08c50d: Status 404 returned error can't find the container with id 99ee803a58d9c559a20c638247293ba2e2c105af5c9648c149c7c0416f08c50d
Sep 10 15:26:58.435923 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:58.435882    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f91d1ea8-2355-4b51-b736-448a89aba131-metrics-certs\") pod \"network-metrics-daemon-zxpwj\" (UID: \"f91d1ea8-2355-4b51-b736-448a89aba131\") " pod="openshift-multus/network-metrics-daemon-zxpwj"
Sep 10 15:26:58.436111 ip-10-0-128-241 kubenswrapper[2567]: E0910 15:26:58.436041    2567 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 15:26:58.436162 ip-10-0-128-241 kubenswrapper[2567]: E0910 15:26:58.436141    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f91d1ea8-2355-4b51-b736-448a89aba131-metrics-certs podName:f91d1ea8-2355-4b51-b736-448a89aba131 nodeName:}" failed. No retries permitted until 2026-09-10 15:27:00.436095655 +0000 UTC m=+6.004327369 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f91d1ea8-2355-4b51-b736-448a89aba131-metrics-certs") pod "network-metrics-daemon-zxpwj" (UID: "f91d1ea8-2355-4b51-b736-448a89aba131") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 15:26:58.536990 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:58.536956    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/49bbf894-04df-427c-b71a-c108fbf25054-original-pull-secret\") pod \"global-pull-secret-syncer-fgx9b\" (UID: \"49bbf894-04df-427c-b71a-c108fbf25054\") " pod="kube-system/global-pull-secret-syncer-fgx9b"
Sep 10 15:26:58.537167 ip-10-0-128-241 kubenswrapper[2567]: E0910 15:26:58.537142    2567 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 10 15:26:58.537244 ip-10-0-128-241 kubenswrapper[2567]: E0910 15:26:58.537227    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/49bbf894-04df-427c-b71a-c108fbf25054-original-pull-secret podName:49bbf894-04df-427c-b71a-c108fbf25054 nodeName:}" failed. No retries permitted until 2026-09-10 15:27:00.537205932 +0000 UTC m=+6.105437661 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/49bbf894-04df-427c-b71a-c108fbf25054-original-pull-secret") pod "global-pull-secret-syncer-fgx9b" (UID: "49bbf894-04df-427c-b71a-c108fbf25054") : object "kube-system"/"original-pull-secret" not registered
Sep 10 15:26:58.638476 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:58.637793    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4wmx2\" (UniqueName: \"kubernetes.io/projected/e105e176-14c2-4093-946d-492879428ef7-kube-api-access-4wmx2\") pod \"network-check-target-2f7vt\" (UID: \"e105e176-14c2-4093-946d-492879428ef7\") " pod="openshift-network-diagnostics/network-check-target-2f7vt"
Sep 10 15:26:58.638476 ip-10-0-128-241 kubenswrapper[2567]: E0910 15:26:58.637984    2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 10 15:26:58.638476 ip-10-0-128-241 kubenswrapper[2567]: E0910 15:26:58.638004    2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 10 15:26:58.638476 ip-10-0-128-241 kubenswrapper[2567]: E0910 15:26:58.638018    2567 projected.go:196] Error preparing data for projected volume kube-api-access-4wmx2 for pod openshift-network-diagnostics/network-check-target-2f7vt: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 15:26:58.638476 ip-10-0-128-241 kubenswrapper[2567]: E0910 15:26:58.638081    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e105e176-14c2-4093-946d-492879428ef7-kube-api-access-4wmx2 podName:e105e176-14c2-4093-946d-492879428ef7 nodeName:}" failed. No retries permitted until 2026-09-10 15:27:00.638061683 +0000 UTC m=+6.206293404 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-4wmx2" (UniqueName: "kubernetes.io/projected/e105e176-14c2-4093-946d-492879428ef7-kube-api-access-4wmx2") pod "network-check-target-2f7vt" (UID: "e105e176-14c2-4093-946d-492879428ef7") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 15:26:58.860872 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:58.858513    2567 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-09 15:21:57 +0000 UTC" deadline="2028-02-23 10:11:50.528995367 +0000 UTC"
Sep 10 15:26:58.860872 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:58.858551    2567 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12738h44m51.670449708s"
Sep 10 15:26:58.898876 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:58.898843    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2f7vt"
Sep 10 15:26:58.899057 ip-10-0-128-241 kubenswrapper[2567]: E0910 15:26:58.898981    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2f7vt" podUID="e105e176-14c2-4093-946d-492879428ef7"
Sep 10 15:26:58.899551 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:58.899526    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zxpwj"
Sep 10 15:26:58.899682 ip-10-0-128-241 kubenswrapper[2567]: E0910 15:26:58.899658    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zxpwj" podUID="f91d1ea8-2355-4b51-b736-448a89aba131"
Sep 10 15:26:58.931328 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:58.931279    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-241.ec2.internal" event={"ID":"e18dcc5e79fca5bc2558326e4e703c37","Type":"ContainerDied","Data":"0c979bb61f3cc7a01b7706ea6354aed5d85f7fca98daefe1bcad4af8b62287e2"}
Sep 10 15:26:58.931491 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:58.931358    2567 generic.go:356] "Generic (PLEG): container finished" podID="e18dcc5e79fca5bc2558326e4e703c37" containerID="0c979bb61f3cc7a01b7706ea6354aed5d85f7fca98daefe1bcad4af8b62287e2" exitCode=0
Sep 10 15:26:58.935058 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:58.935028    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-241.ec2.internal" event={"ID":"7cc72b205a190f6d5d54c7ca1053da96","Type":"ContainerStarted","Data":"ceeea3dc5857282834dfe052ae569485eea540ff2a536ab46a502bc85f7be380"}
Sep 10 15:26:58.939634 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:58.939610    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-g8rkv" event={"ID":"e9b09c08-aec9-41db-8386-e62731150ccc","Type":"ContainerStarted","Data":"99ee803a58d9c559a20c638247293ba2e2c105af5c9648c149c7c0416f08c50d"}
Sep 10 15:26:58.943908 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:58.943884    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-82drl" event={"ID":"b44e8918-d279-456a-b0cc-63526a3947c2","Type":"ContainerStarted","Data":"b5d79cbb6063a2d279efd87036e0ff954a3e42899c615b1179087b13061c960c"}
Sep 10 15:26:58.969093 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:58.969059    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-pzrzz" event={"ID":"aa974d12-9161-4bc9-a1c7-0846241a3206","Type":"ContainerStarted","Data":"c20f11a538a5d25c7df71fae632ab91286079fe52ab710efcf29c5b4b4abfe52"}
Sep 10 15:26:58.974020 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:58.973948    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-241.ec2.internal" podStartSLOduration=2.9739297000000002 podStartE2EDuration="2.9739297s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 15:26:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 15:26:58.973549757 +0000 UTC m=+4.541781491" watchObservedRunningTime="2026-09-10 15:26:58.9739297 +0000 UTC m=+4.542161435"
Sep 10 15:26:58.987493 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:58.987454    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-krx2c" event={"ID":"831261f1-1d19-42bd-80c0-5173a7132c38","Type":"ContainerStarted","Data":"55e6683ec55d1bff2673fe3b06baae8747ab050ee80d799f6f7ee2ca3b10742a"}
Sep 10 15:26:58.998246 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:58.998209    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-xt7qs" event={"ID":"ccbffcf4-9a0d-445b-b05d-3bcf97e1866d","Type":"ContainerStarted","Data":"4e9465b7ed71e36fddedd68ece5498121cf682f13eca5d1ae457838ba3dceb6f"}
Sep 10 15:26:59.000862 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:59.000826    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-rctzd" event={"ID":"4dd1b47c-e497-45d2-a404-4ea3a2d8ec94","Type":"ContainerStarted","Data":"356f1d1e280a59f7230798448d32dea9a39b8e6dc5b17bad23b5536b04dd6898"}
Sep 10 15:26:59.898666 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:26:59.898579    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fgx9b"
Sep 10 15:26:59.899262 ip-10-0-128-241 kubenswrapper[2567]: E0910 15:26:59.898735    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fgx9b" podUID="49bbf894-04df-427c-b71a-c108fbf25054"
Sep 10 15:27:00.016817 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:27:00.016572    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-241.ec2.internal" event={"ID":"e18dcc5e79fca5bc2558326e4e703c37","Type":"ContainerStarted","Data":"ca9e4f6a9e3f4adbc65ee4a69f88f8845a97727b7378c0b444956cb6a2e71955"}
Sep 10 15:27:00.453028 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:27:00.452986    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f91d1ea8-2355-4b51-b736-448a89aba131-metrics-certs\") pod \"network-metrics-daemon-zxpwj\" (UID: \"f91d1ea8-2355-4b51-b736-448a89aba131\") " pod="openshift-multus/network-metrics-daemon-zxpwj"
Sep 10 15:27:00.453233 ip-10-0-128-241 kubenswrapper[2567]: E0910 15:27:00.453158    2567 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 15:27:00.453233 ip-10-0-128-241 kubenswrapper[2567]: E0910 15:27:00.453228    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f91d1ea8-2355-4b51-b736-448a89aba131-metrics-certs podName:f91d1ea8-2355-4b51-b736-448a89aba131 nodeName:}" failed. No retries permitted until 2026-09-10 15:27:04.453207007 +0000 UTC m=+10.021438722 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f91d1ea8-2355-4b51-b736-448a89aba131-metrics-certs") pod "network-metrics-daemon-zxpwj" (UID: "f91d1ea8-2355-4b51-b736-448a89aba131") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 15:27:00.554192 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:27:00.554147    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/49bbf894-04df-427c-b71a-c108fbf25054-original-pull-secret\") pod \"global-pull-secret-syncer-fgx9b\" (UID: \"49bbf894-04df-427c-b71a-c108fbf25054\") " pod="kube-system/global-pull-secret-syncer-fgx9b"
Sep 10 15:27:00.554399 ip-10-0-128-241 kubenswrapper[2567]: E0910 15:27:00.554334    2567 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 10 15:27:00.554480 ip-10-0-128-241 kubenswrapper[2567]: E0910 15:27:00.554404    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/49bbf894-04df-427c-b71a-c108fbf25054-original-pull-secret podName:49bbf894-04df-427c-b71a-c108fbf25054 nodeName:}" failed. No retries permitted until 2026-09-10 15:27:04.554385668 +0000 UTC m=+10.122617383 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/49bbf894-04df-427c-b71a-c108fbf25054-original-pull-secret") pod "global-pull-secret-syncer-fgx9b" (UID: "49bbf894-04df-427c-b71a-c108fbf25054") : object "kube-system"/"original-pull-secret" not registered
Sep 10 15:27:00.655170 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:27:00.655129    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4wmx2\" (UniqueName: \"kubernetes.io/projected/e105e176-14c2-4093-946d-492879428ef7-kube-api-access-4wmx2\") pod \"network-check-target-2f7vt\" (UID: \"e105e176-14c2-4093-946d-492879428ef7\") " pod="openshift-network-diagnostics/network-check-target-2f7vt"
Sep 10 15:27:00.655355 ip-10-0-128-241 kubenswrapper[2567]: E0910 15:27:00.655344    2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 10 15:27:00.655426 ip-10-0-128-241 kubenswrapper[2567]: E0910 15:27:00.655370    2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 10 15:27:00.655426 ip-10-0-128-241 kubenswrapper[2567]: E0910 15:27:00.655383    2567 projected.go:196] Error preparing data for projected volume kube-api-access-4wmx2 for pod openshift-network-diagnostics/network-check-target-2f7vt: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 15:27:00.655554 ip-10-0-128-241 kubenswrapper[2567]: E0910 15:27:00.655443    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e105e176-14c2-4093-946d-492879428ef7-kube-api-access-4wmx2 podName:e105e176-14c2-4093-946d-492879428ef7 nodeName:}" failed. No retries permitted until 2026-09-10 15:27:04.65542451 +0000 UTC m=+10.223656226 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-4wmx2" (UniqueName: "kubernetes.io/projected/e105e176-14c2-4093-946d-492879428ef7-kube-api-access-4wmx2") pod "network-check-target-2f7vt" (UID: "e105e176-14c2-4093-946d-492879428ef7") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 15:27:00.900355 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:27:00.899436    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zxpwj"
Sep 10 15:27:00.900355 ip-10-0-128-241 kubenswrapper[2567]: E0910 15:27:00.899645    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zxpwj" podUID="f91d1ea8-2355-4b51-b736-448a89aba131"
Sep 10 15:27:00.900355 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:27:00.900136    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2f7vt"
Sep 10 15:27:00.900355 ip-10-0-128-241 kubenswrapper[2567]: E0910 15:27:00.900257    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2f7vt" podUID="e105e176-14c2-4093-946d-492879428ef7"
Sep 10 15:27:01.898629 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:27:01.898575    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fgx9b"
Sep 10 15:27:01.898819 ip-10-0-128-241 kubenswrapper[2567]: E0910 15:27:01.898721    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fgx9b" podUID="49bbf894-04df-427c-b71a-c108fbf25054"
Sep 10 15:27:02.899431 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:27:02.899397    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2f7vt"
Sep 10 15:27:02.899944 ip-10-0-128-241 kubenswrapper[2567]: E0910 15:27:02.899547    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2f7vt" podUID="e105e176-14c2-4093-946d-492879428ef7"
Sep 10 15:27:02.901548 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:27:02.901385    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zxpwj"
Sep 10 15:27:02.901548 ip-10-0-128-241 kubenswrapper[2567]: E0910 15:27:02.901508    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zxpwj" podUID="f91d1ea8-2355-4b51-b736-448a89aba131"
Sep 10 15:27:03.898070 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:27:03.897979    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fgx9b"
Sep 10 15:27:03.898279 ip-10-0-128-241 kubenswrapper[2567]: E0910 15:27:03.898131    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fgx9b" podUID="49bbf894-04df-427c-b71a-c108fbf25054"
Sep 10 15:27:04.488557 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:27:04.488297    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f91d1ea8-2355-4b51-b736-448a89aba131-metrics-certs\") pod \"network-metrics-daemon-zxpwj\" (UID: \"f91d1ea8-2355-4b51-b736-448a89aba131\") " pod="openshift-multus/network-metrics-daemon-zxpwj"
Sep 10 15:27:04.488557 ip-10-0-128-241 kubenswrapper[2567]: E0910 15:27:04.488479    2567 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 15:27:04.488557 ip-10-0-128-241 kubenswrapper[2567]: E0910 15:27:04.488553    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f91d1ea8-2355-4b51-b736-448a89aba131-metrics-certs podName:f91d1ea8-2355-4b51-b736-448a89aba131 nodeName:}" failed. No retries permitted until 2026-09-10 15:27:12.48853178 +0000 UTC m=+18.056763496 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f91d1ea8-2355-4b51-b736-448a89aba131-metrics-certs") pod "network-metrics-daemon-zxpwj" (UID: "f91d1ea8-2355-4b51-b736-448a89aba131") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 15:27:04.589068 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:27:04.589020    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/49bbf894-04df-427c-b71a-c108fbf25054-original-pull-secret\") pod \"global-pull-secret-syncer-fgx9b\" (UID: \"49bbf894-04df-427c-b71a-c108fbf25054\") " pod="kube-system/global-pull-secret-syncer-fgx9b"
Sep 10 15:27:04.589269 ip-10-0-128-241 kubenswrapper[2567]: E0910 15:27:04.589188    2567 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 10 15:27:04.589332 ip-10-0-128-241 kubenswrapper[2567]: E0910 15:27:04.589287    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/49bbf894-04df-427c-b71a-c108fbf25054-original-pull-secret podName:49bbf894-04df-427c-b71a-c108fbf25054 nodeName:}" failed. No retries permitted until 2026-09-10 15:27:12.589264312 +0000 UTC m=+18.157496021 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/49bbf894-04df-427c-b71a-c108fbf25054-original-pull-secret") pod "global-pull-secret-syncer-fgx9b" (UID: "49bbf894-04df-427c-b71a-c108fbf25054") : object "kube-system"/"original-pull-secret" not registered
Sep 10 15:27:04.689925 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:27:04.689885    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4wmx2\" (UniqueName: \"kubernetes.io/projected/e105e176-14c2-4093-946d-492879428ef7-kube-api-access-4wmx2\") pod \"network-check-target-2f7vt\" (UID: \"e105e176-14c2-4093-946d-492879428ef7\") " pod="openshift-network-diagnostics/network-check-target-2f7vt"
Sep 10 15:27:04.690124 ip-10-0-128-241 kubenswrapper[2567]: E0910 15:27:04.690073    2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 10 15:27:04.690124 ip-10-0-128-241 kubenswrapper[2567]: E0910 15:27:04.690100    2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 10 15:27:04.690124 ip-10-0-128-241 kubenswrapper[2567]: E0910 15:27:04.690115    2567 projected.go:196] Error preparing data for projected volume kube-api-access-4wmx2 for pod openshift-network-diagnostics/network-check-target-2f7vt: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 15:27:04.690319 ip-10-0-128-241 kubenswrapper[2567]: E0910 15:27:04.690183    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e105e176-14c2-4093-946d-492879428ef7-kube-api-access-4wmx2 podName:e105e176-14c2-4093-946d-492879428ef7 nodeName:}" failed. No retries permitted until 2026-09-10 15:27:12.690164496 +0000 UTC m=+18.258396211 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-4wmx2" (UniqueName: "kubernetes.io/projected/e105e176-14c2-4093-946d-492879428ef7-kube-api-access-4wmx2") pod "network-check-target-2f7vt" (UID: "e105e176-14c2-4093-946d-492879428ef7") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 15:27:04.899691 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:27:04.899617    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2f7vt"
Sep 10 15:27:04.899837 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:27:04.899699    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zxpwj"
Sep 10 15:27:04.899837 ip-10-0-128-241 kubenswrapper[2567]: E0910 15:27:04.899777    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2f7vt" podUID="e105e176-14c2-4093-946d-492879428ef7"
Sep 10 15:27:04.899945 ip-10-0-128-241 kubenswrapper[2567]: E0910 15:27:04.899914    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zxpwj" podUID="f91d1ea8-2355-4b51-b736-448a89aba131"
Sep 10 15:27:05.898685 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:27:05.898516    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fgx9b"
Sep 10 15:27:05.899107 ip-10-0-128-241 kubenswrapper[2567]: E0910 15:27:05.898789    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fgx9b" podUID="49bbf894-04df-427c-b71a-c108fbf25054"
Sep 10 15:27:06.898215 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:27:06.898102    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zxpwj"
Sep 10 15:27:06.898439 ip-10-0-128-241 kubenswrapper[2567]: E0910 15:27:06.898256    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zxpwj" podUID="f91d1ea8-2355-4b51-b736-448a89aba131"
Sep 10 15:27:06.898641 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:27:06.898616    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2f7vt"
Sep 10 15:27:06.898749 ip-10-0-128-241 kubenswrapper[2567]: E0910 15:27:06.898721    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2f7vt" podUID="e105e176-14c2-4093-946d-492879428ef7"
Sep 10 15:27:07.898489 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:27:07.898428    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fgx9b"
Sep 10 15:27:07.898672 ip-10-0-128-241 kubenswrapper[2567]: E0910 15:27:07.898565    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fgx9b" podUID="49bbf894-04df-427c-b71a-c108fbf25054"
Sep 10 15:27:08.898394 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:27:08.898358    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2f7vt"
Sep 10 15:27:08.898833 ip-10-0-128-241 kubenswrapper[2567]: E0910 15:27:08.898497    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2f7vt" podUID="e105e176-14c2-4093-946d-492879428ef7"
Sep 10 15:27:08.898833 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:27:08.898537    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zxpwj"
Sep 10 15:27:08.898833 ip-10-0-128-241 kubenswrapper[2567]: E0910 15:27:08.898636    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zxpwj" podUID="f91d1ea8-2355-4b51-b736-448a89aba131"
Sep 10 15:27:09.897911 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:27:09.897871    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fgx9b"
Sep 10 15:27:09.898067 ip-10-0-128-241 kubenswrapper[2567]: E0910 15:27:09.897986    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fgx9b" podUID="49bbf894-04df-427c-b71a-c108fbf25054"
Sep 10 15:27:10.898211 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:27:10.898176    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2f7vt"
Sep 10 15:27:10.898688 ip-10-0-128-241 kubenswrapper[2567]: E0910 15:27:10.898297    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2f7vt" podUID="e105e176-14c2-4093-946d-492879428ef7"
Sep 10 15:27:10.898688 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:27:10.898348    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zxpwj"
Sep 10 15:27:10.898688 ip-10-0-128-241 kubenswrapper[2567]: E0910 15:27:10.898463    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zxpwj" podUID="f91d1ea8-2355-4b51-b736-448a89aba131"
Sep 10 15:27:11.898461 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:27:11.898426    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fgx9b"
Sep 10 15:27:11.898983 ip-10-0-128-241 kubenswrapper[2567]: E0910 15:27:11.898567    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fgx9b" podUID="49bbf894-04df-427c-b71a-c108fbf25054"
Sep 10 15:27:12.552645 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:27:12.552601    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f91d1ea8-2355-4b51-b736-448a89aba131-metrics-certs\") pod \"network-metrics-daemon-zxpwj\" (UID: \"f91d1ea8-2355-4b51-b736-448a89aba131\") " pod="openshift-multus/network-metrics-daemon-zxpwj"
Sep 10 15:27:12.552848 ip-10-0-128-241 kubenswrapper[2567]: E0910 15:27:12.552749    2567 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 15:27:12.552848 ip-10-0-128-241 kubenswrapper[2567]: E0910 15:27:12.552823    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f91d1ea8-2355-4b51-b736-448a89aba131-metrics-certs podName:f91d1ea8-2355-4b51-b736-448a89aba131 nodeName:}" failed. No retries permitted until 2026-09-10 15:27:28.552806301 +0000 UTC m=+34.121038027 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f91d1ea8-2355-4b51-b736-448a89aba131-metrics-certs") pod "network-metrics-daemon-zxpwj" (UID: "f91d1ea8-2355-4b51-b736-448a89aba131") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 15:27:12.653974 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:27:12.653936    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/49bbf894-04df-427c-b71a-c108fbf25054-original-pull-secret\") pod \"global-pull-secret-syncer-fgx9b\" (UID: \"49bbf894-04df-427c-b71a-c108fbf25054\") " pod="kube-system/global-pull-secret-syncer-fgx9b"
Sep 10 15:27:12.654140 ip-10-0-128-241 kubenswrapper[2567]: E0910 15:27:12.654090    2567 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 10 15:27:12.654193 ip-10-0-128-241 kubenswrapper[2567]: E0910 15:27:12.654161    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/49bbf894-04df-427c-b71a-c108fbf25054-original-pull-secret podName:49bbf894-04df-427c-b71a-c108fbf25054 nodeName:}" failed. No retries permitted until 2026-09-10 15:27:28.65413951 +0000 UTC m=+34.222371232 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/49bbf894-04df-427c-b71a-c108fbf25054-original-pull-secret") pod "global-pull-secret-syncer-fgx9b" (UID: "49bbf894-04df-427c-b71a-c108fbf25054") : object "kube-system"/"original-pull-secret" not registered
Sep 10 15:27:12.755243 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:27:12.755194    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4wmx2\" (UniqueName: \"kubernetes.io/projected/e105e176-14c2-4093-946d-492879428ef7-kube-api-access-4wmx2\") pod \"network-check-target-2f7vt\" (UID: \"e105e176-14c2-4093-946d-492879428ef7\") " pod="openshift-network-diagnostics/network-check-target-2f7vt"
Sep 10 15:27:12.755383 ip-10-0-128-241 kubenswrapper[2567]: E0910 15:27:12.755324    2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 10 15:27:12.755383 ip-10-0-128-241 kubenswrapper[2567]: E0910 15:27:12.755338    2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 10 15:27:12.755383 ip-10-0-128-241 kubenswrapper[2567]: E0910 15:27:12.755347    2567 projected.go:196] Error preparing data for projected volume kube-api-access-4wmx2 for pod openshift-network-diagnostics/network-check-target-2f7vt: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 15:27:12.755494 ip-10-0-128-241 kubenswrapper[2567]: E0910 15:27:12.755395    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e105e176-14c2-4093-946d-492879428ef7-kube-api-access-4wmx2 podName:e105e176-14c2-4093-946d-492879428ef7 nodeName:}" failed. No retries permitted until 2026-09-10 15:27:28.755383196 +0000 UTC m=+34.323614916 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-4wmx2" (UniqueName: "kubernetes.io/projected/e105e176-14c2-4093-946d-492879428ef7-kube-api-access-4wmx2") pod "network-check-target-2f7vt" (UID: "e105e176-14c2-4093-946d-492879428ef7") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 15:27:12.898004 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:27:12.897930    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2f7vt"
Sep 10 15:27:12.898004 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:27:12.897931    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zxpwj"
Sep 10 15:27:12.898205 ip-10-0-128-241 kubenswrapper[2567]: E0910 15:27:12.898097    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zxpwj" podUID="f91d1ea8-2355-4b51-b736-448a89aba131"
Sep 10 15:27:12.898298 ip-10-0-128-241 kubenswrapper[2567]: E0910 15:27:12.898256    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2f7vt" podUID="e105e176-14c2-4093-946d-492879428ef7"
Sep 10 15:27:13.898496 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:27:13.898462    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fgx9b"
Sep 10 15:27:13.898964 ip-10-0-128-241 kubenswrapper[2567]: E0910 15:27:13.898573    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fgx9b" podUID="49bbf894-04df-427c-b71a-c108fbf25054"
Sep 10 15:27:14.899410 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:27:14.899378    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2f7vt"
Sep 10 15:27:14.899802 ip-10-0-128-241 kubenswrapper[2567]: E0910 15:27:14.899473    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2f7vt" podUID="e105e176-14c2-4093-946d-492879428ef7"
Sep 10 15:27:14.899802 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:27:14.899703    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zxpwj"
Sep 10 15:27:14.899923 ip-10-0-128-241 kubenswrapper[2567]: E0910 15:27:14.899820    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zxpwj" podUID="f91d1ea8-2355-4b51-b736-448a89aba131"
Sep 10 15:27:15.897938 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:27:15.897898    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fgx9b"
Sep 10 15:27:15.898118 ip-10-0-128-241 kubenswrapper[2567]: E0910 15:27:15.898032    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fgx9b" podUID="49bbf894-04df-427c-b71a-c108fbf25054"
Sep 10 15:27:16.898532 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:27:16.898262    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2f7vt"
Sep 10 15:27:16.899294 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:27:16.898306    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zxpwj"
Sep 10 15:27:16.899294 ip-10-0-128-241 kubenswrapper[2567]: E0910 15:27:16.898669    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2f7vt" podUID="e105e176-14c2-4093-946d-492879428ef7"
Sep 10 15:27:16.899294 ip-10-0-128-241 kubenswrapper[2567]: E0910 15:27:16.898802    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zxpwj" podUID="f91d1ea8-2355-4b51-b736-448a89aba131"
Sep 10 15:27:17.055881 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:27:17.055803    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-bctsx" event={"ID":"c1a148df-5bb2-4059-9d8a-2715cc0b6f8c","Type":"ContainerStarted","Data":"49ab74c43d1c64d3230e091270b6c41cdb65abefa442455305d5324a9a9ccba6"}
Sep 10 15:27:17.055881 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:27:17.055881    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-bctsx" event={"ID":"c1a148df-5bb2-4059-9d8a-2715cc0b6f8c","Type":"ContainerStarted","Data":"bb57325b07b998315b5bfcfd42fedf9a07ec79bc48a55730104bdf1e862634db"}
Sep 10 15:27:17.056073 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:27:17.055926    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-bctsx" event={"ID":"c1a148df-5bb2-4059-9d8a-2715cc0b6f8c","Type":"ContainerStarted","Data":"4ba82865a6641cb22da811bcdaef5daf9bde5adf2944bdb336d51a0f77ff808d"}
Sep 10 15:27:17.056073 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:27:17.055959    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-bctsx" event={"ID":"c1a148df-5bb2-4059-9d8a-2715cc0b6f8c","Type":"ContainerStarted","Data":"9641b176ffb1973711f27ea9a2ae8e58893489ae828cad9a1799d87074def4be"}
Sep 10 15:27:17.056073 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:27:17.056008    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-bctsx" event={"ID":"c1a148df-5bb2-4059-9d8a-2715cc0b6f8c","Type":"ContainerStarted","Data":"69fe0597f707d576dab70a549f827d030110bb9151f8c634859164e9c6bae001"}
Sep 10 15:27:17.056073 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:27:17.056039    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-bctsx" event={"ID":"c1a148df-5bb2-4059-9d8a-2715cc0b6f8c","Type":"ContainerStarted","Data":"74902a4b170ffc5a4e825dd47abf2ff24f5dcb850851b80e10d264dcc398b96c"}
Sep 10 15:27:17.057496 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:27:17.057470    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-g8rkv" event={"ID":"e9b09c08-aec9-41db-8386-e62731150ccc","Type":"ContainerStarted","Data":"f22bc09e9247639bd2e069551cf3d5fe7704a86d0d499965408487abf0515f8b"}
Sep 10 15:27:17.059188 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:27:17.059166    2567 generic.go:356] "Generic (PLEG): container finished" podID="b44e8918-d279-456a-b0cc-63526a3947c2" containerID="d0042148456db3c7f2ee218e37e585f80558856647ddcd3b19cba8f2bdc62456" exitCode=0
Sep 10 15:27:17.059299 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:27:17.059253    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-82drl" event={"ID":"b44e8918-d279-456a-b0cc-63526a3947c2","Type":"ContainerDied","Data":"d0042148456db3c7f2ee218e37e585f80558856647ddcd3b19cba8f2bdc62456"}
Sep 10 15:27:17.060877 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:27:17.060796    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-pzrzz" event={"ID":"aa974d12-9161-4bc9-a1c7-0846241a3206","Type":"ContainerStarted","Data":"4bb048f98ce5b3b987118929b2f05a7b9a156216156122957d171211b08a4e43"}
Sep 10 15:27:17.062819 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:27:17.062782    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-gqngc" event={"ID":"1b59aadf-17c2-49a9-90e4-c507ca7d21c2","Type":"ContainerStarted","Data":"de7a98b670c7e12fcb2a2280f6d04fa8db8dd4daddd23b9fcc972f28fec644d4"}
Sep 10 15:27:17.064373 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:27:17.064350    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-krx2c" event={"ID":"831261f1-1d19-42bd-80c0-5173a7132c38","Type":"ContainerStarted","Data":"8b21e55879f8c746f1cfde475b1c11fa42b29269659dad50b000ab734b21bd2e"}
Sep 10 15:27:17.066016 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:27:17.065986    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-xt7qs" event={"ID":"ccbffcf4-9a0d-445b-b05d-3bcf97e1866d","Type":"ContainerStarted","Data":"78f26a26357bef6dd47e952f76ec6c04966c4f9a24a221c439951da4ab8badbc"}
Sep 10 15:27:17.067802 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:27:17.067780    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-rctzd" event={"ID":"4dd1b47c-e497-45d2-a404-4ea3a2d8ec94","Type":"ContainerStarted","Data":"f83cd3e8395957ebce0db712849cc29b75a1ced41ed98ed34a867a32cf9d2088"}
Sep 10 15:27:17.074748 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:27:17.074704    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-g8rkv" podStartSLOduration=3.278499309 podStartE2EDuration="21.074687748s" totalImagesPullingTime="17.796188439s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 15:26:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 15:27:17.073981506 +0000 UTC m=+22.642213238" watchObservedRunningTime="2026-09-10 15:27:17.074687748 +0000 UTC m=+22.642919482"
Sep 10 15:27:17.074841 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:27:17.074796    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-241.ec2.internal" podStartSLOduration=21.074788724 podStartE2EDuration="21.074788724s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 15:26:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 15:27:00.035460462 +0000 UTC m=+5.603692196" watchObservedRunningTime="2026-09-10 15:27:17.074788724 +0000 UTC m=+22.643020458"
Sep 10 15:27:17.090980 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:27:17.090928    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-pzrzz" podStartSLOduration=4.289909113 podStartE2EDuration="22.090894046s" totalImagesPullingTime="17.800984933s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 15:26:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 15:27:17.090196361 +0000 UTC m=+22.658428091" watchObservedRunningTime="2026-09-10 15:27:17.090894046 +0000 UTC m=+22.659125780"
Sep 10 15:27:17.134818 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:27:17.134765    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-xt7qs" podStartSLOduration=4.33237209 podStartE2EDuration="22.134748828s" totalImagesPullingTime="17.802376738s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 15:26:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 15:27:17.134321155 +0000 UTC m=+22.702552882" watchObservedRunningTime="2026-09-10 15:27:17.134748828 +0000 UTC m=+22.702980558"
Sep 10 15:27:17.156462 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:27:17.156398    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-gqngc" podStartSLOduration=3.20247224 podStartE2EDuration="22.156371929s" totalImagesPullingTime="18.953899689s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 15:26:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 15:27:17.155431791 +0000 UTC m=+22.723663521" watchObservedRunningTime="2026-09-10 15:27:17.156371929 +0000 UTC m=+22.724603662"
Sep 10 15:27:17.184265 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:27:17.184199    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-rctzd" podStartSLOduration=4.358468644 podStartE2EDuration="22.184183484s" totalImagesPullingTime="17.82571484s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 15:26:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 15:27:17.183675193 +0000 UTC m=+22.751906922" watchObservedRunningTime="2026-09-10 15:27:17.184183484 +0000 UTC m=+22.752415214"
Sep 10 15:27:17.337254 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:27:17.337207    2567 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Sep 10 15:27:17.409539 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:27:17.409507    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-pzrzz"
Sep 10 15:27:17.409968 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:27:17.409952    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-pzrzz"
Sep 10 15:27:17.868918 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:27:17.868820    2567 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-09-10T15:27:17.337227271Z","UUID":"655dce5d-31e9-4513-8c0d-48347fa4c9e9","Handler":null,"Name":"","Endpoint":""}
Sep 10 15:27:17.870511 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:27:17.870488    2567 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Sep 10 15:27:17.870633 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:27:17.870520    2567 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Sep 10 15:27:17.897768 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:27:17.897740    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fgx9b"
Sep 10 15:27:17.897899 ip-10-0-128-241 kubenswrapper[2567]: E0910 15:27:17.897841    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fgx9b" podUID="49bbf894-04df-427c-b71a-c108fbf25054"
Sep 10 15:27:18.073074 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:27:18.073035    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-krx2c" event={"ID":"831261f1-1d19-42bd-80c0-5173a7132c38","Type":"ContainerStarted","Data":"6e44b445011ea570474087186e3f040842a6e12b436c7b2937e9b0ae04662234"}
Sep 10 15:27:18.074799 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:27:18.074765    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-9b7ts" event={"ID":"b2e911b2-e1d3-4eb1-85b8-61f6a12bd9ba","Type":"ContainerStarted","Data":"0674d9b7863edb69f85e097bc936e94236ea50d2f27c1e7ef4319ea073e98b9e"}
Sep 10 15:27:18.075755 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:27:18.075733    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-pzrzz"
Sep 10 15:27:18.076175 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:27:18.076156    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-pzrzz"
Sep 10 15:27:18.109725 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:27:18.109681    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-9b7ts" podStartSLOduration=4.215510788 podStartE2EDuration="23.10966758s" totalImagesPullingTime="18.894156792s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 15:26:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 15:27:18.092009435 +0000 UTC m=+23.660241164" watchObservedRunningTime="2026-09-10 15:27:18.10966758 +0000 UTC m=+23.677899310"
Sep 10 15:27:18.898213 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:27:18.898138    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2f7vt"
Sep 10 15:27:18.898406 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:27:18.898144    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zxpwj"
Sep 10 15:27:18.898406 ip-10-0-128-241 kubenswrapper[2567]: E0910 15:27:18.898271    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2f7vt" podUID="e105e176-14c2-4093-946d-492879428ef7"
Sep 10 15:27:18.898406 ip-10-0-128-241 kubenswrapper[2567]: E0910 15:27:18.898339    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zxpwj" podUID="f91d1ea8-2355-4b51-b736-448a89aba131"
Sep 10 15:27:19.080725 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:27:19.080683    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-bctsx" event={"ID":"c1a148df-5bb2-4059-9d8a-2715cc0b6f8c","Type":"ContainerStarted","Data":"d8482168f1698572ab9aee4f3ec0ad6d8b7f1a9b1e00518dbcfef6e869fe4cc9"}
Sep 10 15:27:19.082916 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:27:19.082890    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-krx2c" event={"ID":"831261f1-1d19-42bd-80c0-5173a7132c38","Type":"ContainerStarted","Data":"4d7110b51c69b5ba5db71312bb4cde6794d0a340e7aed2e9c3d0c357fd8e0f93"}
Sep 10 15:27:19.105550 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:27:19.105495    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-krx2c" podStartSLOduration=3.049591682 podStartE2EDuration="23.105479315s" totalImagesPullingTime="20.055887633s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 15:26:56 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 15:27:19.104744259 +0000 UTC m=+24.672975990" watchObservedRunningTime="2026-09-10 15:27:19.105479315 +0000 UTC m=+24.673711045"
Sep 10 15:27:19.898128 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:27:19.898093    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fgx9b"
Sep 10 15:27:19.898361 ip-10-0-128-241 kubenswrapper[2567]: E0910 15:27:19.898241    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fgx9b" podUID="49bbf894-04df-427c-b71a-c108fbf25054"
Sep 10 15:27:20.897833 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:27:20.897773    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2f7vt"
Sep 10 15:27:20.897833 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:27:20.897815    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zxpwj"
Sep 10 15:27:20.898629 ip-10-0-128-241 kubenswrapper[2567]: E0910 15:27:20.897914    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2f7vt" podUID="e105e176-14c2-4093-946d-492879428ef7"
Sep 10 15:27:20.898629 ip-10-0-128-241 kubenswrapper[2567]: E0910 15:27:20.897997    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zxpwj" podUID="f91d1ea8-2355-4b51-b736-448a89aba131"
Sep 10 15:27:21.898240 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:27:21.898192    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fgx9b"
Sep 10 15:27:21.898801 ip-10-0-128-241 kubenswrapper[2567]: E0910 15:27:21.898338    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fgx9b" podUID="49bbf894-04df-427c-b71a-c108fbf25054"
Sep 10 15:27:22.898807 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:27:22.898630    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2f7vt"
Sep 10 15:27:22.899600 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:27:22.898696    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zxpwj"
Sep 10 15:27:22.899600 ip-10-0-128-241 kubenswrapper[2567]: E0910 15:27:22.898874    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2f7vt" podUID="e105e176-14c2-4093-946d-492879428ef7"
Sep 10 15:27:22.899600 ip-10-0-128-241 kubenswrapper[2567]: E0910 15:27:22.898949    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zxpwj" podUID="f91d1ea8-2355-4b51-b736-448a89aba131"
Sep 10 15:27:23.101065 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:27:23.101031    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-bctsx" event={"ID":"c1a148df-5bb2-4059-9d8a-2715cc0b6f8c","Type":"ContainerStarted","Data":"097ae84952e341145ad2a52376040bacbbabf46811a7494311d11d8eaa652e51"}
Sep 10 15:27:23.101384 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:27:23.101322    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-bctsx"
Sep 10 15:27:23.101513 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:27:23.101411    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-bctsx"
Sep 10 15:27:23.101513 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:27:23.101465    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-bctsx"
Sep 10 15:27:23.103107 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:27:23.103082    2567 generic.go:356] "Generic (PLEG): container finished" podID="b44e8918-d279-456a-b0cc-63526a3947c2" containerID="93f5ffdbccfd35b5754b57221b30fff8264b0d13328e55e9a2ab2ef8af7c1a60" exitCode=0
Sep 10 15:27:23.103215 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:27:23.103127    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-82drl" event={"ID":"b44e8918-d279-456a-b0cc-63526a3947c2","Type":"ContainerDied","Data":"93f5ffdbccfd35b5754b57221b30fff8264b0d13328e55e9a2ab2ef8af7c1a60"}
Sep 10 15:27:23.116551 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:27:23.116521    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-bctsx"
Sep 10 15:27:23.116648 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:27:23.116630    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-bctsx"
Sep 10 15:27:23.129343 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:27:23.129302    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-bctsx" podStartSLOduration=9.182950918 podStartE2EDuration="28.129289681s" totalImagesPullingTime="18.946338763s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 15:26:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 15:27:23.127974453 +0000 UTC m=+28.696206184" watchObservedRunningTime="2026-09-10 15:27:23.129289681 +0000 UTC m=+28.697521410"
Sep 10 15:27:23.898135 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:27:23.898105    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fgx9b"
Sep 10 15:27:23.898355 ip-10-0-128-241 kubenswrapper[2567]: E0910 15:27:23.898272    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fgx9b" podUID="49bbf894-04df-427c-b71a-c108fbf25054"
Sep 10 15:27:24.068632 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:27:24.068599    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-fgx9b"]
Sep 10 15:27:24.071650 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:27:24.071615    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-2f7vt"]
Sep 10 15:27:24.071787 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:27:24.071722    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2f7vt"
Sep 10 15:27:24.071855 ip-10-0-128-241 kubenswrapper[2567]: E0910 15:27:24.071819    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2f7vt" podUID="e105e176-14c2-4093-946d-492879428ef7"
Sep 10 15:27:24.072569 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:27:24.072321    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-zxpwj"]
Sep 10 15:27:24.072569 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:27:24.072424    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zxpwj"
Sep 10 15:27:24.072569 ip-10-0-128-241 kubenswrapper[2567]: E0910 15:27:24.072511    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zxpwj" podUID="f91d1ea8-2355-4b51-b736-448a89aba131"
Sep 10 15:27:24.106039 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:27:24.106010    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fgx9b"
Sep 10 15:27:24.106210 ip-10-0-128-241 kubenswrapper[2567]: E0910 15:27:24.106150    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fgx9b" podUID="49bbf894-04df-427c-b71a-c108fbf25054"
Sep 10 15:27:25.111199 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:27:25.110951    2567 generic.go:356] "Generic (PLEG): container finished" podID="b44e8918-d279-456a-b0cc-63526a3947c2" containerID="b94914361eb2815b7d87614ae20b3807384d9da2dfbb0279c7abd16d139bde55" exitCode=0
Sep 10 15:27:25.111199 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:27:25.111029    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-82drl" event={"ID":"b44e8918-d279-456a-b0cc-63526a3947c2","Type":"ContainerDied","Data":"b94914361eb2815b7d87614ae20b3807384d9da2dfbb0279c7abd16d139bde55"}
Sep 10 15:27:25.897901 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:27:25.897875    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zxpwj"
Sep 10 15:27:25.898095 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:27:25.897978    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2f7vt"
Sep 10 15:27:25.898095 ip-10-0-128-241 kubenswrapper[2567]: E0910 15:27:25.897981    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zxpwj" podUID="f91d1ea8-2355-4b51-b736-448a89aba131"
Sep 10 15:27:25.898095 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:27:25.897983    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fgx9b"
Sep 10 15:27:25.898261 ip-10-0-128-241 kubenswrapper[2567]: E0910 15:27:25.898086    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2f7vt" podUID="e105e176-14c2-4093-946d-492879428ef7"
Sep 10 15:27:25.898261 ip-10-0-128-241 kubenswrapper[2567]: E0910 15:27:25.898184    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fgx9b" podUID="49bbf894-04df-427c-b71a-c108fbf25054"
Sep 10 15:27:27.117840 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:27:27.117805    2567 generic.go:356] "Generic (PLEG): container finished" podID="b44e8918-d279-456a-b0cc-63526a3947c2" containerID="43149a46a6c2668bcac02b1c39125f6921e7092e3357bf767cf51e61514df361" exitCode=0
Sep 10 15:27:27.118219 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:27:27.117856    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-82drl" event={"ID":"b44e8918-d279-456a-b0cc-63526a3947c2","Type":"ContainerDied","Data":"43149a46a6c2668bcac02b1c39125f6921e7092e3357bf767cf51e61514df361"}
Sep 10 15:27:27.898577 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:27:27.898537    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2f7vt"
Sep 10 15:27:27.898577 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:27:27.898580    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zxpwj"
Sep 10 15:27:27.898810 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:27:27.898577    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fgx9b"
Sep 10 15:27:27.898810 ip-10-0-128-241 kubenswrapper[2567]: E0910 15:27:27.898661    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2f7vt" podUID="e105e176-14c2-4093-946d-492879428ef7"
Sep 10 15:27:27.898810 ip-10-0-128-241 kubenswrapper[2567]: E0910 15:27:27.898738    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zxpwj" podUID="f91d1ea8-2355-4b51-b736-448a89aba131"
Sep 10 15:27:27.898810 ip-10-0-128-241 kubenswrapper[2567]: E0910 15:27:27.898791    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fgx9b" podUID="49bbf894-04df-427c-b71a-c108fbf25054"
Sep 10 15:27:28.573637 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:27:28.573611    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f91d1ea8-2355-4b51-b736-448a89aba131-metrics-certs\") pod \"network-metrics-daemon-zxpwj\" (UID: \"f91d1ea8-2355-4b51-b736-448a89aba131\") " pod="openshift-multus/network-metrics-daemon-zxpwj"
Sep 10 15:27:28.573995 ip-10-0-128-241 kubenswrapper[2567]: E0910 15:27:28.573769    2567 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 15:27:28.573995 ip-10-0-128-241 kubenswrapper[2567]: E0910 15:27:28.573847    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f91d1ea8-2355-4b51-b736-448a89aba131-metrics-certs podName:f91d1ea8-2355-4b51-b736-448a89aba131 nodeName:}" failed. No retries permitted until 2026-09-10 15:28:00.573825351 +0000 UTC m=+66.142057065 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f91d1ea8-2355-4b51-b736-448a89aba131-metrics-certs") pod "network-metrics-daemon-zxpwj" (UID: "f91d1ea8-2355-4b51-b736-448a89aba131") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 15:27:28.674704 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:27:28.674668    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/49bbf894-04df-427c-b71a-c108fbf25054-original-pull-secret\") pod \"global-pull-secret-syncer-fgx9b\" (UID: \"49bbf894-04df-427c-b71a-c108fbf25054\") " pod="kube-system/global-pull-secret-syncer-fgx9b"
Sep 10 15:27:28.674855 ip-10-0-128-241 kubenswrapper[2567]: E0910 15:27:28.674774    2567 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 10 15:27:28.674855 ip-10-0-128-241 kubenswrapper[2567]: E0910 15:27:28.674823    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/49bbf894-04df-427c-b71a-c108fbf25054-original-pull-secret podName:49bbf894-04df-427c-b71a-c108fbf25054 nodeName:}" failed. No retries permitted until 2026-09-10 15:28:00.674809466 +0000 UTC m=+66.243041174 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/49bbf894-04df-427c-b71a-c108fbf25054-original-pull-secret") pod "global-pull-secret-syncer-fgx9b" (UID: "49bbf894-04df-427c-b71a-c108fbf25054") : object "kube-system"/"original-pull-secret" not registered
Sep 10 15:27:28.743092 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:27:28.743061    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-241.ec2.internal" event="NodeReady"
Sep 10 15:27:28.743282 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:27:28.743219    2567 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Sep 10 15:27:28.775276 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:27:28.775224    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4wmx2\" (UniqueName: \"kubernetes.io/projected/e105e176-14c2-4093-946d-492879428ef7-kube-api-access-4wmx2\") pod \"network-check-target-2f7vt\" (UID: \"e105e176-14c2-4093-946d-492879428ef7\") " pod="openshift-network-diagnostics/network-check-target-2f7vt"
Sep 10 15:27:28.775427 ip-10-0-128-241 kubenswrapper[2567]: E0910 15:27:28.775386    2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 10 15:27:28.775427 ip-10-0-128-241 kubenswrapper[2567]: E0910 15:27:28.775408    2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 10 15:27:28.775427 ip-10-0-128-241 kubenswrapper[2567]: E0910 15:27:28.775417    2567 projected.go:196] Error preparing data for projected volume kube-api-access-4wmx2 for pod openshift-network-diagnostics/network-check-target-2f7vt: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 15:27:28.775585 ip-10-0-128-241 kubenswrapper[2567]: E0910 15:27:28.775463    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e105e176-14c2-4093-946d-492879428ef7-kube-api-access-4wmx2 podName:e105e176-14c2-4093-946d-492879428ef7 nodeName:}" failed. No retries permitted until 2026-09-10 15:28:00.775450016 +0000 UTC m=+66.343681729 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-4wmx2" (UniqueName: "kubernetes.io/projected/e105e176-14c2-4093-946d-492879428ef7-kube-api-access-4wmx2") pod "network-check-target-2f7vt" (UID: "e105e176-14c2-4093-946d-492879428ef7") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 15:27:28.784242 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:27:28.784212    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-659bc4b685-mpbnt"]
Sep 10 15:27:28.787470 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:27:28.787451    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-659bc4b685-mpbnt"
Sep 10 15:27:28.789870 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:27:28.789848    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Sep 10 15:27:28.789977 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:27:28.789884    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Sep 10 15:27:28.789977 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:27:28.789970    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Sep 10 15:27:28.790087 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:27:28.790056    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-f2kws\""
Sep 10 15:27:28.797740 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:27:28.797719    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Sep 10 15:27:28.801473 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:27:28.801450    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-4k8rd"]
Sep 10 15:27:28.805491 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:27:28.805473    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-659bc4b685-mpbnt"]
Sep 10 15:27:28.805632 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:27:28.805608    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-4k8rd"
Sep 10 15:27:28.805722 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:27:28.805634    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-h8h5g"]
Sep 10 15:27:28.808405 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:27:28.807889    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Sep 10 15:27:28.808405 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:27:28.808204    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-w5mt4\""
Sep 10 15:27:28.808405 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:27:28.808277    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Sep 10 15:27:28.809216 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:27:28.809195    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-h8h5g"
Sep 10 15:27:28.811358 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:27:28.811341    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Sep 10 15:27:28.811633 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:27:28.811614    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Sep 10 15:27:28.811731 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:27:28.811675    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-8ztg8\""
Sep 10 15:27:28.811731 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:27:28.811715    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Sep 10 15:27:28.815461 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:27:28.815438    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-4k8rd"]
Sep 10 15:27:28.818482 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:27:28.818463    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-h8h5g"]
Sep 10 15:27:28.876325 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:27:28.876246    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/5ef1f4eb-6292-4a77-8515-48e1671d8d13-ca-trust-extracted\") pod \"image-registry-659bc4b685-mpbnt\" (UID: \"5ef1f4eb-6292-4a77-8515-48e1671d8d13\") " pod="openshift-image-registry/image-registry-659bc4b685-mpbnt"
Sep 10 15:27:28.876325 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:27:28.876304    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/5ef1f4eb-6292-4a77-8515-48e1671d8d13-registry-certificates\") pod \"image-registry-659bc4b685-mpbnt\" (UID: \"5ef1f4eb-6292-4a77-8515-48e1671d8d13\") " pod="openshift-image-registry/image-registry-659bc4b685-mpbnt"
Sep 10 15:27:28.876534 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:27:28.876339    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/5ef1f4eb-6292-4a77-8515-48e1671d8d13-installation-pull-secrets\") pod \"image-registry-659bc4b685-mpbnt\" (UID: \"5ef1f4eb-6292-4a77-8515-48e1671d8d13\") " pod="openshift-image-registry/image-registry-659bc4b685-mpbnt"
Sep 10 15:27:28.876534 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:27:28.876389    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/5ef1f4eb-6292-4a77-8515-48e1671d8d13-bound-sa-token\") pod \"image-registry-659bc4b685-mpbnt\" (UID: \"5ef1f4eb-6292-4a77-8515-48e1671d8d13\") " pod="openshift-image-registry/image-registry-659bc4b685-mpbnt"
Sep 10 15:27:28.876534 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:27:28.876476    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5ef1f4eb-6292-4a77-8515-48e1671d8d13-registry-tls\") pod \"image-registry-659bc4b685-mpbnt\" (UID: \"5ef1f4eb-6292-4a77-8515-48e1671d8d13\") " pod="openshift-image-registry/image-registry-659bc4b685-mpbnt"
Sep 10 15:27:28.876534 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:27:28.876504    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rzs4q\" (UniqueName: \"kubernetes.io/projected/5ef1f4eb-6292-4a77-8515-48e1671d8d13-kube-api-access-rzs4q\") pod \"image-registry-659bc4b685-mpbnt\" (UID: \"5ef1f4eb-6292-4a77-8515-48e1671d8d13\") " pod="openshift-image-registry/image-registry-659bc4b685-mpbnt"
Sep 10 15:27:28.876688 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:27:28.876543    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/5ef1f4eb-6292-4a77-8515-48e1671d8d13-image-registry-private-configuration\") pod \"image-registry-659bc4b685-mpbnt\" (UID: \"5ef1f4eb-6292-4a77-8515-48e1671d8d13\") " pod="openshift-image-registry/image-registry-659bc4b685-mpbnt"
Sep 10 15:27:28.876688 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:27:28.876570    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/5ef1f4eb-6292-4a77-8515-48e1671d8d13-trusted-ca\") pod \"image-registry-659bc4b685-mpbnt\" (UID: \"5ef1f4eb-6292-4a77-8515-48e1671d8d13\") " pod="openshift-image-registry/image-registry-659bc4b685-mpbnt"
Sep 10 15:27:28.972293 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:27:28.972259    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:27:28.977115 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:27:28.977093    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/5ef1f4eb-6292-4a77-8515-48e1671d8d13-ca-trust-extracted\") pod \"image-registry-659bc4b685-mpbnt\" (UID: \"5ef1f4eb-6292-4a77-8515-48e1671d8d13\") " pod="openshift-image-registry/image-registry-659bc4b685-mpbnt"
Sep 10 15:27:28.977244 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:27:28.977122    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9818ae98-700a-4e5c-a4a5-24c0fc624542-tmp-dir\") pod \"dns-default-4k8rd\" (UID: \"9818ae98-700a-4e5c-a4a5-24c0fc624542\") " pod="openshift-dns/dns-default-4k8rd"
Sep 10 15:27:28.977244 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:27:28.977143    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9818ae98-700a-4e5c-a4a5-24c0fc624542-metrics-tls\") pod \"dns-default-4k8rd\" (UID: \"9818ae98-700a-4e5c-a4a5-24c0fc624542\") " pod="openshift-dns/dns-default-4k8rd"
Sep 10 15:27:28.977244 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:27:28.977178    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nqh5q\" (UniqueName: \"kubernetes.io/projected/b9ea6621-b91a-44bd-a38f-666177f4d26b-kube-api-access-nqh5q\") pod \"ingress-canary-h8h5g\" (UID: \"b9ea6621-b91a-44bd-a38f-666177f4d26b\") " pod="openshift-ingress-canary/ingress-canary-h8h5g"
Sep 10 15:27:28.977244 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:27:28.977197    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/5ef1f4eb-6292-4a77-8515-48e1671d8d13-registry-certificates\") pod \"image-registry-659bc4b685-mpbnt\" (UID: \"5ef1f4eb-6292-4a77-8515-48e1671d8d13\") " pod="openshift-image-registry/image-registry-659bc4b685-mpbnt"
Sep 10 15:27:28.977244 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:27:28.977214    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/5ef1f4eb-6292-4a77-8515-48e1671d8d13-installation-pull-secrets\") pod \"image-registry-659bc4b685-mpbnt\" (UID: \"5ef1f4eb-6292-4a77-8515-48e1671d8d13\") " pod="openshift-image-registry/image-registry-659bc4b685-mpbnt"
Sep 10 15:27:28.977496 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:27:28.977254    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/5ef1f4eb-6292-4a77-8515-48e1671d8d13-bound-sa-token\") pod \"image-registry-659bc4b685-mpbnt\" (UID: \"5ef1f4eb-6292-4a77-8515-48e1671d8d13\") " pod="openshift-image-registry/image-registry-659bc4b685-mpbnt"
Sep 10 15:27:28.977496 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:27:28.977381    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/9818ae98-700a-4e5c-a4a5-24c0fc624542-config-volume\") pod \"dns-default-4k8rd\" (UID: \"9818ae98-700a-4e5c-a4a5-24c0fc624542\") " pod="openshift-dns/dns-default-4k8rd"
Sep 10 15:27:28.977496 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:27:28.977405    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mghd8\" (UniqueName: \"kubernetes.io/projected/9818ae98-700a-4e5c-a4a5-24c0fc624542-kube-api-access-mghd8\") pod \"dns-default-4k8rd\" (UID: \"9818ae98-700a-4e5c-a4a5-24c0fc624542\") " pod="openshift-dns/dns-default-4k8rd"
Sep 10 15:27:28.977496 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:27:28.977437    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5ef1f4eb-6292-4a77-8515-48e1671d8d13-registry-tls\") pod \"image-registry-659bc4b685-mpbnt\" (UID: \"5ef1f4eb-6292-4a77-8515-48e1671d8d13\") " pod="openshift-image-registry/image-registry-659bc4b685-mpbnt"
Sep 10 15:27:28.977496 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:27:28.977466    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rzs4q\" (UniqueName: \"kubernetes.io/projected/5ef1f4eb-6292-4a77-8515-48e1671d8d13-kube-api-access-rzs4q\") pod \"image-registry-659bc4b685-mpbnt\" (UID: \"5ef1f4eb-6292-4a77-8515-48e1671d8d13\") " pod="openshift-image-registry/image-registry-659bc4b685-mpbnt"
Sep 10 15:27:28.977711 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:27:28.977503    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b9ea6621-b91a-44bd-a38f-666177f4d26b-cert\") pod \"ingress-canary-h8h5g\" (UID: \"b9ea6621-b91a-44bd-a38f-666177f4d26b\") " pod="openshift-ingress-canary/ingress-canary-h8h5g"
Sep 10 15:27:28.977711 ip-10-0-128-241 kubenswrapper[2567]: E0910 15:27:28.977576    2567 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 10 15:27:28.977711 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:27:28.977585    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/5ef1f4eb-6292-4a77-8515-48e1671d8d13-ca-trust-extracted\") pod \"image-registry-659bc4b685-mpbnt\" (UID: \"5ef1f4eb-6292-4a77-8515-48e1671d8d13\") " pod="openshift-image-registry/image-registry-659bc4b685-mpbnt"
Sep 10 15:27:28.977711 ip-10-0-128-241 kubenswrapper[2567]: E0910 15:27:28.977595    2567 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-659bc4b685-mpbnt: secret "image-registry-tls" not found
Sep 10 15:27:28.977711 ip-10-0-128-241 kubenswrapper[2567]: E0910 15:27:28.977659    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5ef1f4eb-6292-4a77-8515-48e1671d8d13-registry-tls podName:5ef1f4eb-6292-4a77-8515-48e1671d8d13 nodeName:}" failed. No retries permitted until 2026-09-10 15:27:29.47764111 +0000 UTC m=+35.045872822 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/5ef1f4eb-6292-4a77-8515-48e1671d8d13-registry-tls") pod "image-registry-659bc4b685-mpbnt" (UID: "5ef1f4eb-6292-4a77-8515-48e1671d8d13") : secret "image-registry-tls" not found
Sep 10 15:27:28.977711 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:27:28.977582    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/5ef1f4eb-6292-4a77-8515-48e1671d8d13-image-registry-private-configuration\") pod \"image-registry-659bc4b685-mpbnt\" (UID: \"5ef1f4eb-6292-4a77-8515-48e1671d8d13\") " pod="openshift-image-registry/image-registry-659bc4b685-mpbnt"
Sep 10 15:27:28.977711 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:27:28.977702    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/5ef1f4eb-6292-4a77-8515-48e1671d8d13-trusted-ca\") pod \"image-registry-659bc4b685-mpbnt\" (UID: \"5ef1f4eb-6292-4a77-8515-48e1671d8d13\") " pod="openshift-image-registry/image-registry-659bc4b685-mpbnt"
Sep 10 15:27:28.978033 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:27:28.977869    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/5ef1f4eb-6292-4a77-8515-48e1671d8d13-registry-certificates\") pod \"image-registry-659bc4b685-mpbnt\" (UID: \"5ef1f4eb-6292-4a77-8515-48e1671d8d13\") " pod="openshift-image-registry/image-registry-659bc4b685-mpbnt"
Sep 10 15:27:28.978636 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:27:28.978608    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/5ef1f4eb-6292-4a77-8515-48e1671d8d13-trusted-ca\") pod \"image-registry-659bc4b685-mpbnt\" (UID: \"5ef1f4eb-6292-4a77-8515-48e1671d8d13\") " pod="openshift-image-registry/image-registry-659bc4b685-mpbnt"
Sep 10 15:27:28.981961 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:27:28.981939    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/5ef1f4eb-6292-4a77-8515-48e1671d8d13-image-registry-private-configuration\") pod \"image-registry-659bc4b685-mpbnt\" (UID: \"5ef1f4eb-6292-4a77-8515-48e1671d8d13\") " pod="openshift-image-registry/image-registry-659bc4b685-mpbnt"
Sep 10 15:27:28.981961 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:27:28.981953    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/5ef1f4eb-6292-4a77-8515-48e1671d8d13-installation-pull-secrets\") pod \"image-registry-659bc4b685-mpbnt\" (UID: \"5ef1f4eb-6292-4a77-8515-48e1671d8d13\") " pod="openshift-image-registry/image-registry-659bc4b685-mpbnt"
Sep 10 15:27:28.988149 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:27:28.988127    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/5ef1f4eb-6292-4a77-8515-48e1671d8d13-bound-sa-token\") pod \"image-registry-659bc4b685-mpbnt\" (UID: \"5ef1f4eb-6292-4a77-8515-48e1671d8d13\") " pod="openshift-image-registry/image-registry-659bc4b685-mpbnt"
Sep 10 15:27:28.988421 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:27:28.988372    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rzs4q\" (UniqueName: \"kubernetes.io/projected/5ef1f4eb-6292-4a77-8515-48e1671d8d13-kube-api-access-rzs4q\") pod \"image-registry-659bc4b685-mpbnt\" (UID: \"5ef1f4eb-6292-4a77-8515-48e1671d8d13\") " pod="openshift-image-registry/image-registry-659bc4b685-mpbnt"
Sep 10 15:27:29.072393 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:27:29.072361    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:27:29.078263 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:27:29.078242    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/9818ae98-700a-4e5c-a4a5-24c0fc624542-config-volume\") pod \"dns-default-4k8rd\" (UID: \"9818ae98-700a-4e5c-a4a5-24c0fc624542\") " pod="openshift-dns/dns-default-4k8rd"
Sep 10 15:27:29.078356 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:27:29.078271    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mghd8\" (UniqueName: \"kubernetes.io/projected/9818ae98-700a-4e5c-a4a5-24c0fc624542-kube-api-access-mghd8\") pod \"dns-default-4k8rd\" (UID: \"9818ae98-700a-4e5c-a4a5-24c0fc624542\") " pod="openshift-dns/dns-default-4k8rd"
Sep 10 15:27:29.078356 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:27:29.078306    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b9ea6621-b91a-44bd-a38f-666177f4d26b-cert\") pod \"ingress-canary-h8h5g\" (UID: \"b9ea6621-b91a-44bd-a38f-666177f4d26b\") " pod="openshift-ingress-canary/ingress-canary-h8h5g"
Sep 10 15:27:29.078356 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:27:29.078347    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9818ae98-700a-4e5c-a4a5-24c0fc624542-tmp-dir\") pod \"dns-default-4k8rd\" (UID: \"9818ae98-700a-4e5c-a4a5-24c0fc624542\") " pod="openshift-dns/dns-default-4k8rd"
Sep 10 15:27:29.078497 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:27:29.078361    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9818ae98-700a-4e5c-a4a5-24c0fc624542-metrics-tls\") pod \"dns-default-4k8rd\" (UID: \"9818ae98-700a-4e5c-a4a5-24c0fc624542\") " pod="openshift-dns/dns-default-4k8rd"
Sep 10 15:27:29.078497 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:27:29.078379    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nqh5q\" (UniqueName: \"kubernetes.io/projected/b9ea6621-b91a-44bd-a38f-666177f4d26b-kube-api-access-nqh5q\") pod \"ingress-canary-h8h5g\" (UID: \"b9ea6621-b91a-44bd-a38f-666177f4d26b\") " pod="openshift-ingress-canary/ingress-canary-h8h5g"
Sep 10 15:27:29.078497 ip-10-0-128-241 kubenswrapper[2567]: E0910 15:27:29.078469    2567 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 10 15:27:29.078633 ip-10-0-128-241 kubenswrapper[2567]: E0910 15:27:29.078546    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9818ae98-700a-4e5c-a4a5-24c0fc624542-metrics-tls podName:9818ae98-700a-4e5c-a4a5-24c0fc624542 nodeName:}" failed. No retries permitted until 2026-09-10 15:27:29.578526905 +0000 UTC m=+35.146758626 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/9818ae98-700a-4e5c-a4a5-24c0fc624542-metrics-tls") pod "dns-default-4k8rd" (UID: "9818ae98-700a-4e5c-a4a5-24c0fc624542") : secret "dns-default-metrics-tls" not found
Sep 10 15:27:29.078825 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:27:29.078803    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9818ae98-700a-4e5c-a4a5-24c0fc624542-tmp-dir\") pod \"dns-default-4k8rd\" (UID: \"9818ae98-700a-4e5c-a4a5-24c0fc624542\") " pod="openshift-dns/dns-default-4k8rd"
Sep 10 15:27:29.078991 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:27:29.078972    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/9818ae98-700a-4e5c-a4a5-24c0fc624542-config-volume\") pod \"dns-default-4k8rd\" (UID: \"9818ae98-700a-4e5c-a4a5-24c0fc624542\") " pod="openshift-dns/dns-default-4k8rd"
Sep 10 15:27:29.080763 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:27:29.080742    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b9ea6621-b91a-44bd-a38f-666177f4d26b-cert\") pod \"ingress-canary-h8h5g\" (UID: \"b9ea6621-b91a-44bd-a38f-666177f4d26b\") " pod="openshift-ingress-canary/ingress-canary-h8h5g"
Sep 10 15:27:29.087851 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:27:29.087827    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mghd8\" (UniqueName: \"kubernetes.io/projected/9818ae98-700a-4e5c-a4a5-24c0fc624542-kube-api-access-mghd8\") pod \"dns-default-4k8rd\" (UID: \"9818ae98-700a-4e5c-a4a5-24c0fc624542\") " pod="openshift-dns/dns-default-4k8rd"
Sep 10 15:27:29.088271 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:27:29.088253    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nqh5q\" (UniqueName: \"kubernetes.io/projected/b9ea6621-b91a-44bd-a38f-666177f4d26b-kube-api-access-nqh5q\") pod \"ingress-canary-h8h5g\" (UID: \"b9ea6621-b91a-44bd-a38f-666177f4d26b\") " pod="openshift-ingress-canary/ingress-canary-h8h5g"
Sep 10 15:27:29.124916 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:27:29.124897    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-h8h5g"
Sep 10 15:27:29.416515 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:27:29.416440    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-h8h5g"]
Sep 10 15:27:29.420369 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:27:29.420342    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb9ea6621_b91a_44bd_a38f_666177f4d26b.slice/crio-e59d2f7bbfe5aef4948970a3a28129b0a845f83792b1bbe43af518ddccd48026 WatchSource:0}: Error finding container e59d2f7bbfe5aef4948970a3a28129b0a845f83792b1bbe43af518ddccd48026: Status 404 returned error can't find the container with id e59d2f7bbfe5aef4948970a3a28129b0a845f83792b1bbe43af518ddccd48026
Sep 10 15:27:29.481406 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:27:29.481366    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5ef1f4eb-6292-4a77-8515-48e1671d8d13-registry-tls\") pod \"image-registry-659bc4b685-mpbnt\" (UID: \"5ef1f4eb-6292-4a77-8515-48e1671d8d13\") " pod="openshift-image-registry/image-registry-659bc4b685-mpbnt"
Sep 10 15:27:29.481566 ip-10-0-128-241 kubenswrapper[2567]: E0910 15:27:29.481524    2567 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 10 15:27:29.481566 ip-10-0-128-241 kubenswrapper[2567]: E0910 15:27:29.481538    2567 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-659bc4b685-mpbnt: secret "image-registry-tls" not found
Sep 10 15:27:29.481650 ip-10-0-128-241 kubenswrapper[2567]: E0910 15:27:29.481598    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5ef1f4eb-6292-4a77-8515-48e1671d8d13-registry-tls podName:5ef1f4eb-6292-4a77-8515-48e1671d8d13 nodeName:}" failed. No retries permitted until 2026-09-10 15:27:30.481584212 +0000 UTC m=+36.049815943 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/5ef1f4eb-6292-4a77-8515-48e1671d8d13-registry-tls") pod "image-registry-659bc4b685-mpbnt" (UID: "5ef1f4eb-6292-4a77-8515-48e1671d8d13") : secret "image-registry-tls" not found
Sep 10 15:27:29.582795 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:27:29.582755    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9818ae98-700a-4e5c-a4a5-24c0fc624542-metrics-tls\") pod \"dns-default-4k8rd\" (UID: \"9818ae98-700a-4e5c-a4a5-24c0fc624542\") " pod="openshift-dns/dns-default-4k8rd"
Sep 10 15:27:29.583427 ip-10-0-128-241 kubenswrapper[2567]: E0910 15:27:29.582930    2567 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 10 15:27:29.583427 ip-10-0-128-241 kubenswrapper[2567]: E0910 15:27:29.583008    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9818ae98-700a-4e5c-a4a5-24c0fc624542-metrics-tls podName:9818ae98-700a-4e5c-a4a5-24c0fc624542 nodeName:}" failed. No retries permitted until 2026-09-10 15:27:30.582987517 +0000 UTC m=+36.151219239 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/9818ae98-700a-4e5c-a4a5-24c0fc624542-metrics-tls") pod "dns-default-4k8rd" (UID: "9818ae98-700a-4e5c-a4a5-24c0fc624542") : secret "dns-default-metrics-tls" not found
Sep 10 15:27:29.897896 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:27:29.897865    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2f7vt"
Sep 10 15:27:29.898096 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:27:29.898011    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fgx9b"
Sep 10 15:27:29.898096 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:27:29.897880    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zxpwj"
Sep 10 15:27:29.900629 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:27:29.900609    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 10 15:27:29.900741 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:27:29.900655    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 10 15:27:29.903814 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:27:29.901794    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 10 15:27:29.903814 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:27:29.902091    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 10 15:27:29.903814 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:27:29.902391    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-5rkds\""
Sep 10 15:27:29.903814 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:27:29.902613    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-h8jcj\""
Sep 10 15:27:30.080701 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:27:30.080661    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:27:30.127124 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:27:30.127083    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-h8h5g" event={"ID":"b9ea6621-b91a-44bd-a38f-666177f4d26b","Type":"ContainerStarted","Data":"e59d2f7bbfe5aef4948970a3a28129b0a845f83792b1bbe43af518ddccd48026"}
Sep 10 15:27:30.490661 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:27:30.490619    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5ef1f4eb-6292-4a77-8515-48e1671d8d13-registry-tls\") pod \"image-registry-659bc4b685-mpbnt\" (UID: \"5ef1f4eb-6292-4a77-8515-48e1671d8d13\") " pod="openshift-image-registry/image-registry-659bc4b685-mpbnt"
Sep 10 15:27:30.490830 ip-10-0-128-241 kubenswrapper[2567]: E0910 15:27:30.490821    2567 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 10 15:27:30.490905 ip-10-0-128-241 kubenswrapper[2567]: E0910 15:27:30.490836    2567 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-659bc4b685-mpbnt: secret "image-registry-tls" not found
Sep 10 15:27:30.490905 ip-10-0-128-241 kubenswrapper[2567]: E0910 15:27:30.490902    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5ef1f4eb-6292-4a77-8515-48e1671d8d13-registry-tls podName:5ef1f4eb-6292-4a77-8515-48e1671d8d13 nodeName:}" failed. No retries permitted until 2026-09-10 15:27:32.490884983 +0000 UTC m=+38.059116712 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/5ef1f4eb-6292-4a77-8515-48e1671d8d13-registry-tls") pod "image-registry-659bc4b685-mpbnt" (UID: "5ef1f4eb-6292-4a77-8515-48e1671d8d13") : secret "image-registry-tls" not found
Sep 10 15:27:30.591865 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:27:30.591826    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9818ae98-700a-4e5c-a4a5-24c0fc624542-metrics-tls\") pod \"dns-default-4k8rd\" (UID: \"9818ae98-700a-4e5c-a4a5-24c0fc624542\") " pod="openshift-dns/dns-default-4k8rd"
Sep 10 15:27:30.592309 ip-10-0-128-241 kubenswrapper[2567]: E0910 15:27:30.592004    2567 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 10 15:27:30.592309 ip-10-0-128-241 kubenswrapper[2567]: E0910 15:27:30.592099    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9818ae98-700a-4e5c-a4a5-24c0fc624542-metrics-tls podName:9818ae98-700a-4e5c-a4a5-24c0fc624542 nodeName:}" failed. No retries permitted until 2026-09-10 15:27:32.592076279 +0000 UTC m=+38.160307990 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/9818ae98-700a-4e5c-a4a5-24c0fc624542-metrics-tls") pod "dns-default-4k8rd" (UID: "9818ae98-700a-4e5c-a4a5-24c0fc624542") : secret "dns-default-metrics-tls" not found
Sep 10 15:27:31.983404 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:27:31.983352    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:27:32.057772 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:27:32.057735    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:27:32.158727 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:27:32.158688    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:27:32.260007 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:27:32.259899    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:27:32.507458 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:27:32.507428    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5ef1f4eb-6292-4a77-8515-48e1671d8d13-registry-tls\") pod \"image-registry-659bc4b685-mpbnt\" (UID: \"5ef1f4eb-6292-4a77-8515-48e1671d8d13\") " pod="openshift-image-registry/image-registry-659bc4b685-mpbnt"
Sep 10 15:27:32.507626 ip-10-0-128-241 kubenswrapper[2567]: E0910 15:27:32.507553    2567 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 10 15:27:32.507626 ip-10-0-128-241 kubenswrapper[2567]: E0910 15:27:32.507570    2567 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-659bc4b685-mpbnt: secret "image-registry-tls" not found
Sep 10 15:27:32.507722 ip-10-0-128-241 kubenswrapper[2567]: E0910 15:27:32.507648    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5ef1f4eb-6292-4a77-8515-48e1671d8d13-registry-tls podName:5ef1f4eb-6292-4a77-8515-48e1671d8d13 nodeName:}" failed. No retries permitted until 2026-09-10 15:27:36.507631468 +0000 UTC m=+42.075863195 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/5ef1f4eb-6292-4a77-8515-48e1671d8d13-registry-tls") pod "image-registry-659bc4b685-mpbnt" (UID: "5ef1f4eb-6292-4a77-8515-48e1671d8d13") : secret "image-registry-tls" not found
Sep 10 15:27:32.608462 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:27:32.608376    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9818ae98-700a-4e5c-a4a5-24c0fc624542-metrics-tls\") pod \"dns-default-4k8rd\" (UID: \"9818ae98-700a-4e5c-a4a5-24c0fc624542\") " pod="openshift-dns/dns-default-4k8rd"
Sep 10 15:27:32.608622 ip-10-0-128-241 kubenswrapper[2567]: E0910 15:27:32.608570    2567 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 10 15:27:32.608671 ip-10-0-128-241 kubenswrapper[2567]: E0910 15:27:32.608649    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9818ae98-700a-4e5c-a4a5-24c0fc624542-metrics-tls podName:9818ae98-700a-4e5c-a4a5-24c0fc624542 nodeName:}" failed. No retries permitted until 2026-09-10 15:27:36.608628076 +0000 UTC m=+42.176859798 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/9818ae98-700a-4e5c-a4a5-24c0fc624542-metrics-tls") pod "dns-default-4k8rd" (UID: "9818ae98-700a-4e5c-a4a5-24c0fc624542") : secret "dns-default-metrics-tls" not found
Sep 10 15:27:33.002688 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:27:33.002631    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:27:33.088291 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:27:33.088254    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:27:33.141743 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:27:33.141711    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:27:33.858029 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:27:33.857992    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:27:33.938806 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:27:33.938775    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:27:34.139848 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:27:34.139773    2567 generic.go:356] "Generic (PLEG): container finished" podID="b44e8918-d279-456a-b0cc-63526a3947c2" containerID="fcf4f0a76a6407148cdae1f1df626346e14893dc7b13fee23896eae1bf74c2a5" exitCode=0
Sep 10 15:27:34.140380 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:27:34.139857    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-82drl" event={"ID":"b44e8918-d279-456a-b0cc-63526a3947c2","Type":"ContainerDied","Data":"fcf4f0a76a6407148cdae1f1df626346e14893dc7b13fee23896eae1bf74c2a5"}
Sep 10 15:27:34.141415 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:27:34.141396    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-h8h5g" event={"ID":"b9ea6621-b91a-44bd-a38f-666177f4d26b","Type":"ContainerStarted","Data":"c37d275f2c6872aec8304410a9dddcef6ee6aa41f039f1153f9b0416baef07c4"}
Sep 10 15:27:34.186308 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:27:34.186258    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-h8h5g" podStartSLOduration=2.326517658 podStartE2EDuration="6.186223396s" totalImagesPullingTime="3.859705738s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 15:27:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 15:27:34.184564716 +0000 UTC m=+39.752796449" watchObservedRunningTime="2026-09-10 15:27:34.186223396 +0000 UTC m=+39.754455126"
Sep 10 15:27:35.147124 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:27:35.147090    2567 generic.go:356] "Generic (PLEG): container finished" podID="b44e8918-d279-456a-b0cc-63526a3947c2" containerID="cecc157e01fac1b774da465ff8a6db54ade71b63d92b03469171eced5e9a621b" exitCode=0
Sep 10 15:27:35.147492 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:27:35.147179    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-82drl" event={"ID":"b44e8918-d279-456a-b0cc-63526a3947c2","Type":"ContainerDied","Data":"cecc157e01fac1b774da465ff8a6db54ade71b63d92b03469171eced5e9a621b"}
Sep 10 15:27:36.124140 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:27:36.124105    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:27:36.153316 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:27:36.153284    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-82drl" event={"ID":"b44e8918-d279-456a-b0cc-63526a3947c2","Type":"ContainerStarted","Data":"c5b89a0677606b4a7a135b542506c78776bfb1c94e9594bf24839925c3aa3fe6"}
Sep 10 15:27:36.176473 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:27:36.176427    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-82drl" podStartSLOduration=9.311884234 podStartE2EDuration="41.176411845s" totalImagesPullingTime="31.864527611s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 15:26:55 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 15:27:36.17614832 +0000 UTC m=+41.744380051" watchObservedRunningTime="2026-09-10 15:27:36.176411845 +0000 UTC m=+41.744643574"
Sep 10 15:27:36.187379 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:27:36.187353    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:27:36.328400 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:27:36.328365    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:27:36.431244 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:27:36.431181    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:27:36.539835 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:27:36.539798    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5ef1f4eb-6292-4a77-8515-48e1671d8d13-registry-tls\") pod \"image-registry-659bc4b685-mpbnt\" (UID: \"5ef1f4eb-6292-4a77-8515-48e1671d8d13\") " pod="openshift-image-registry/image-registry-659bc4b685-mpbnt"
Sep 10 15:27:36.539993 ip-10-0-128-241 kubenswrapper[2567]: E0910 15:27:36.539949    2567 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 10 15:27:36.539993 ip-10-0-128-241 kubenswrapper[2567]: E0910 15:27:36.539968    2567 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-659bc4b685-mpbnt: secret "image-registry-tls" not found
Sep 10 15:27:36.540060 ip-10-0-128-241 kubenswrapper[2567]: E0910 15:27:36.540030    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5ef1f4eb-6292-4a77-8515-48e1671d8d13-registry-tls podName:5ef1f4eb-6292-4a77-8515-48e1671d8d13 nodeName:}" failed. No retries permitted until 2026-09-10 15:27:44.54001502 +0000 UTC m=+50.108246727 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/5ef1f4eb-6292-4a77-8515-48e1671d8d13-registry-tls") pod "image-registry-659bc4b685-mpbnt" (UID: "5ef1f4eb-6292-4a77-8515-48e1671d8d13") : secret "image-registry-tls" not found
Sep 10 15:27:36.640606 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:27:36.640573    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9818ae98-700a-4e5c-a4a5-24c0fc624542-metrics-tls\") pod \"dns-default-4k8rd\" (UID: \"9818ae98-700a-4e5c-a4a5-24c0fc624542\") " pod="openshift-dns/dns-default-4k8rd"
Sep 10 15:27:36.640730 ip-10-0-128-241 kubenswrapper[2567]: E0910 15:27:36.640716    2567 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 10 15:27:36.640788 ip-10-0-128-241 kubenswrapper[2567]: E0910 15:27:36.640777    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9818ae98-700a-4e5c-a4a5-24c0fc624542-metrics-tls podName:9818ae98-700a-4e5c-a4a5-24c0fc624542 nodeName:}" failed. No retries permitted until 2026-09-10 15:27:44.640758341 +0000 UTC m=+50.208990066 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/9818ae98-700a-4e5c-a4a5-24c0fc624542-metrics-tls") pod "dns-default-4k8rd" (UID: "9818ae98-700a-4e5c-a4a5-24c0fc624542") : secret "dns-default-metrics-tls" not found
Sep 10 15:27:39.267283 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:27:39.267226    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:27:39.332065 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:27:39.332034    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:27:43.211196 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:27:43.211166    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:27:43.314226 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:27:43.314191    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:27:43.412456 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:27:43.412419    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:27:44.597664 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:27:44.597631    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5ef1f4eb-6292-4a77-8515-48e1671d8d13-registry-tls\") pod \"image-registry-659bc4b685-mpbnt\" (UID: \"5ef1f4eb-6292-4a77-8515-48e1671d8d13\") " pod="openshift-image-registry/image-registry-659bc4b685-mpbnt"
Sep 10 15:27:44.598025 ip-10-0-128-241 kubenswrapper[2567]: E0910 15:27:44.597780    2567 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 10 15:27:44.598025 ip-10-0-128-241 kubenswrapper[2567]: E0910 15:27:44.597797    2567 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-659bc4b685-mpbnt: secret "image-registry-tls" not found
Sep 10 15:27:44.598025 ip-10-0-128-241 kubenswrapper[2567]: E0910 15:27:44.597850    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5ef1f4eb-6292-4a77-8515-48e1671d8d13-registry-tls podName:5ef1f4eb-6292-4a77-8515-48e1671d8d13 nodeName:}" failed. No retries permitted until 2026-09-10 15:28:00.597833687 +0000 UTC m=+66.166065399 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/5ef1f4eb-6292-4a77-8515-48e1671d8d13-registry-tls") pod "image-registry-659bc4b685-mpbnt" (UID: "5ef1f4eb-6292-4a77-8515-48e1671d8d13") : secret "image-registry-tls" not found
Sep 10 15:27:44.698730 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:27:44.698698    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9818ae98-700a-4e5c-a4a5-24c0fc624542-metrics-tls\") pod \"dns-default-4k8rd\" (UID: \"9818ae98-700a-4e5c-a4a5-24c0fc624542\") " pod="openshift-dns/dns-default-4k8rd"
Sep 10 15:27:44.698875 ip-10-0-128-241 kubenswrapper[2567]: E0910 15:27:44.698850    2567 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 10 15:27:44.698930 ip-10-0-128-241 kubenswrapper[2567]: E0910 15:27:44.698921    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9818ae98-700a-4e5c-a4a5-24c0fc624542-metrics-tls podName:9818ae98-700a-4e5c-a4a5-24c0fc624542 nodeName:}" failed. No retries permitted until 2026-09-10 15:28:00.698903814 +0000 UTC m=+66.267135522 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/9818ae98-700a-4e5c-a4a5-24c0fc624542-metrics-tls") pod "dns-default-4k8rd" (UID: "9818ae98-700a-4e5c-a4a5-24c0fc624542") : secret "dns-default-metrics-tls" not found
Sep 10 15:27:45.210582 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:27:45.210540    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:27:45.306430 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:27:45.306387    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:27:46.997110 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:27:46.997076    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-85b454874b-t96pm"]
Sep 10 15:27:47.012241 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:27:47.012213    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-85b454874b-t96pm"
Sep 10 15:27:47.014380 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:27:47.014353    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-85b454874b-t96pm"]
Sep 10 15:27:47.014646 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:27:47.014620    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\""
Sep 10 15:27:47.014776 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:27:47.014722    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\""
Sep 10 15:27:47.014776 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:27:47.014739    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-8nvbn\""
Sep 10 15:27:47.015804 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:27:47.015786    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\""
Sep 10 15:27:47.015905 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:27:47.015825    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\""
Sep 10 15:27:47.020354 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:27:47.020332    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\""
Sep 10 15:27:47.118753 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:27:47.118710    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/0b9d2cda-73e3-4e7c-9e17-1a3d903d95cf-serving-cert\") pod \"console-operator-85b454874b-t96pm\" (UID: \"0b9d2cda-73e3-4e7c-9e17-1a3d903d95cf\") " pod="openshift-console-operator/console-operator-85b454874b-t96pm"
Sep 10 15:27:47.118753 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:27:47.118750    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/0b9d2cda-73e3-4e7c-9e17-1a3d903d95cf-trusted-ca\") pod \"console-operator-85b454874b-t96pm\" (UID: \"0b9d2cda-73e3-4e7c-9e17-1a3d903d95cf\") " pod="openshift-console-operator/console-operator-85b454874b-t96pm"
Sep 10 15:27:47.118966 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:27:47.118770    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/0b9d2cda-73e3-4e7c-9e17-1a3d903d95cf-config\") pod \"console-operator-85b454874b-t96pm\" (UID: \"0b9d2cda-73e3-4e7c-9e17-1a3d903d95cf\") " pod="openshift-console-operator/console-operator-85b454874b-t96pm"
Sep 10 15:27:47.118966 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:27:47.118856    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ptrm4\" (UniqueName: \"kubernetes.io/projected/0b9d2cda-73e3-4e7c-9e17-1a3d903d95cf-kube-api-access-ptrm4\") pod \"console-operator-85b454874b-t96pm\" (UID: \"0b9d2cda-73e3-4e7c-9e17-1a3d903d95cf\") " pod="openshift-console-operator/console-operator-85b454874b-t96pm"
Sep 10 15:27:47.219699 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:27:47.219663    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/0b9d2cda-73e3-4e7c-9e17-1a3d903d95cf-config\") pod \"console-operator-85b454874b-t96pm\" (UID: \"0b9d2cda-73e3-4e7c-9e17-1a3d903d95cf\") " pod="openshift-console-operator/console-operator-85b454874b-t96pm"
Sep 10 15:27:47.219872 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:27:47.219735    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ptrm4\" (UniqueName: \"kubernetes.io/projected/0b9d2cda-73e3-4e7c-9e17-1a3d903d95cf-kube-api-access-ptrm4\") pod \"console-operator-85b454874b-t96pm\" (UID: \"0b9d2cda-73e3-4e7c-9e17-1a3d903d95cf\") " pod="openshift-console-operator/console-operator-85b454874b-t96pm"
Sep 10 15:27:47.219872 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:27:47.219783    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/0b9d2cda-73e3-4e7c-9e17-1a3d903d95cf-serving-cert\") pod \"console-operator-85b454874b-t96pm\" (UID: \"0b9d2cda-73e3-4e7c-9e17-1a3d903d95cf\") " pod="openshift-console-operator/console-operator-85b454874b-t96pm"
Sep 10 15:27:47.219872 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:27:47.219801    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/0b9d2cda-73e3-4e7c-9e17-1a3d903d95cf-trusted-ca\") pod \"console-operator-85b454874b-t96pm\" (UID: \"0b9d2cda-73e3-4e7c-9e17-1a3d903d95cf\") " pod="openshift-console-operator/console-operator-85b454874b-t96pm"
Sep 10 15:27:47.220537 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:27:47.220518    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/0b9d2cda-73e3-4e7c-9e17-1a3d903d95cf-trusted-ca\") pod \"console-operator-85b454874b-t96pm\" (UID: \"0b9d2cda-73e3-4e7c-9e17-1a3d903d95cf\") " pod="openshift-console-operator/console-operator-85b454874b-t96pm"
Sep 10 15:27:47.220589 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:27:47.220550    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/0b9d2cda-73e3-4e7c-9e17-1a3d903d95cf-config\") pod \"console-operator-85b454874b-t96pm\" (UID: \"0b9d2cda-73e3-4e7c-9e17-1a3d903d95cf\") " pod="openshift-console-operator/console-operator-85b454874b-t96pm"
Sep 10 15:27:47.222679 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:27:47.222661    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/0b9d2cda-73e3-4e7c-9e17-1a3d903d95cf-serving-cert\") pod \"console-operator-85b454874b-t96pm\" (UID: \"0b9d2cda-73e3-4e7c-9e17-1a3d903d95cf\") " pod="openshift-console-operator/console-operator-85b454874b-t96pm"
Sep 10 15:27:47.236025 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:27:47.235998    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ptrm4\" (UniqueName: \"kubernetes.io/projected/0b9d2cda-73e3-4e7c-9e17-1a3d903d95cf-kube-api-access-ptrm4\") pod \"console-operator-85b454874b-t96pm\" (UID: \"0b9d2cda-73e3-4e7c-9e17-1a3d903d95cf\") " pod="openshift-console-operator/console-operator-85b454874b-t96pm"
Sep 10 15:27:47.287427 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:27:47.287343    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:27:47.322968 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:27:47.322930    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-85b454874b-t96pm"
Sep 10 15:27:47.388444 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:27:47.388403    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:27:47.616504 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:27:47.616477    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-85b454874b-t96pm"]
Sep 10 15:27:47.617494 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:27:47.617461    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0b9d2cda_73e3_4e7c_9e17_1a3d903d95cf.slice/crio-acd575828c154398125d7b90714cff2c93992c079f8eecb6eb4096d174eb7cb7 WatchSource:0}: Error finding container acd575828c154398125d7b90714cff2c93992c079f8eecb6eb4096d174eb7cb7: Status 404 returned error can't find the container with id acd575828c154398125d7b90714cff2c93992c079f8eecb6eb4096d174eb7cb7
Sep 10 15:27:48.181927 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:27:48.181889    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-85b454874b-t96pm" event={"ID":"0b9d2cda-73e3-4e7c-9e17-1a3d903d95cf","Type":"ContainerStarted","Data":"acd575828c154398125d7b90714cff2c93992c079f8eecb6eb4096d174eb7cb7"}
Sep 10 15:27:49.144152 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:27:49.144110    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:27:49.242050 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:27:49.242014    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:27:50.188799 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:27:50.188770    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-t96pm_0b9d2cda-73e3-4e7c-9e17-1a3d903d95cf/console-operator/0.log"
Sep 10 15:27:50.188968 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:27:50.188808    2567 generic.go:356] "Generic (PLEG): container finished" podID="0b9d2cda-73e3-4e7c-9e17-1a3d903d95cf" containerID="1c6717cf053cf073f2a17b8f4e4fc906be898281ab31a1156524f98d0d40f1a1" exitCode=255
Sep 10 15:27:50.188968 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:27:50.188847    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-85b454874b-t96pm" event={"ID":"0b9d2cda-73e3-4e7c-9e17-1a3d903d95cf","Type":"ContainerDied","Data":"1c6717cf053cf073f2a17b8f4e4fc906be898281ab31a1156524f98d0d40f1a1"}
Sep 10 15:27:50.189088 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:27:50.189073    2567 scope.go:117] "RemoveContainer" containerID="1c6717cf053cf073f2a17b8f4e4fc906be898281ab31a1156524f98d0d40f1a1"
Sep 10 15:27:50.212304 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:27:50.212276    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:27:50.311824 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:27:50.311790    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:27:51.192828 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:27:51.192807    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-t96pm_0b9d2cda-73e3-4e7c-9e17-1a3d903d95cf/console-operator/1.log"
Sep 10 15:27:51.193244 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:27:51.193215    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-t96pm_0b9d2cda-73e3-4e7c-9e17-1a3d903d95cf/console-operator/0.log"
Sep 10 15:27:51.193316 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:27:51.193268    2567 generic.go:356] "Generic (PLEG): container finished" podID="0b9d2cda-73e3-4e7c-9e17-1a3d903d95cf" containerID="8639f4fbdd150a963310dbb5b8cd98b7c798e8f9daf57eb7e0c3c0ee03e8ff44" exitCode=255
Sep 10 15:27:51.193354 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:27:51.193328    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-85b454874b-t96pm" event={"ID":"0b9d2cda-73e3-4e7c-9e17-1a3d903d95cf","Type":"ContainerDied","Data":"8639f4fbdd150a963310dbb5b8cd98b7c798e8f9daf57eb7e0c3c0ee03e8ff44"}
Sep 10 15:27:51.193395 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:27:51.193382    2567 scope.go:117] "RemoveContainer" containerID="1c6717cf053cf073f2a17b8f4e4fc906be898281ab31a1156524f98d0d40f1a1"
Sep 10 15:27:51.193589 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:27:51.193571    2567 scope.go:117] "RemoveContainer" containerID="8639f4fbdd150a963310dbb5b8cd98b7c798e8f9daf57eb7e0c3c0ee03e8ff44"
Sep 10 15:27:51.193768 ip-10-0-128-241 kubenswrapper[2567]: E0910 15:27:51.193746    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-85b454874b-t96pm_openshift-console-operator(0b9d2cda-73e3-4e7c-9e17-1a3d903d95cf)\"" pod="openshift-console-operator/console-operator-85b454874b-t96pm" podUID="0b9d2cda-73e3-4e7c-9e17-1a3d903d95cf"
Sep 10 15:27:51.449125 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:27:51.449050    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-5vhfm"]
Sep 10 15:27:51.454837 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:27:51.454813    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-5vhfm"
Sep 10 15:27:51.461799 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:27:51.461776    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Sep 10 15:27:51.461799 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:27:51.461792    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Sep 10 15:27:51.461952 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:27:51.461777    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Sep 10 15:27:51.462165 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:27:51.462151    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-xrkll\""
Sep 10 15:27:51.462208 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:27:51.462155    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Sep 10 15:27:51.467698 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:27:51.467677    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-5vhfm"]
Sep 10 15:27:51.477383 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:27:51.477355    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:27:51.553422 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:27:51.553390    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/66247a82-0118-4ced-87d2-5f65c49557dc-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-5vhfm\" (UID: \"66247a82-0118-4ced-87d2-5f65c49557dc\") " pod="openshift-insights/insights-runtime-extractor-5vhfm"
Sep 10 15:27:51.553619 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:27:51.553433    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/66247a82-0118-4ced-87d2-5f65c49557dc-data-volume\") pod \"insights-runtime-extractor-5vhfm\" (UID: \"66247a82-0118-4ced-87d2-5f65c49557dc\") " pod="openshift-insights/insights-runtime-extractor-5vhfm"
Sep 10 15:27:51.553619 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:27:51.553459    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nrkff\" (UniqueName: \"kubernetes.io/projected/66247a82-0118-4ced-87d2-5f65c49557dc-kube-api-access-nrkff\") pod \"insights-runtime-extractor-5vhfm\" (UID: \"66247a82-0118-4ced-87d2-5f65c49557dc\") " pod="openshift-insights/insights-runtime-extractor-5vhfm"
Sep 10 15:27:51.553619 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:27:51.553559    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/66247a82-0118-4ced-87d2-5f65c49557dc-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-5vhfm\" (UID: \"66247a82-0118-4ced-87d2-5f65c49557dc\") " pod="openshift-insights/insights-runtime-extractor-5vhfm"
Sep 10 15:27:51.553619 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:27:51.553591    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/66247a82-0118-4ced-87d2-5f65c49557dc-crio-socket\") pod \"insights-runtime-extractor-5vhfm\" (UID: \"66247a82-0118-4ced-87d2-5f65c49557dc\") " pod="openshift-insights/insights-runtime-extractor-5vhfm"
Sep 10 15:27:51.554251 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:27:51.554209    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:27:51.654633 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:27:51.654603    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/66247a82-0118-4ced-87d2-5f65c49557dc-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-5vhfm\" (UID: \"66247a82-0118-4ced-87d2-5f65c49557dc\") " pod="openshift-insights/insights-runtime-extractor-5vhfm"
Sep 10 15:27:51.654744 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:27:51.654638    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/66247a82-0118-4ced-87d2-5f65c49557dc-data-volume\") pod \"insights-runtime-extractor-5vhfm\" (UID: \"66247a82-0118-4ced-87d2-5f65c49557dc\") " pod="openshift-insights/insights-runtime-extractor-5vhfm"
Sep 10 15:27:51.654744 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:27:51.654658    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nrkff\" (UniqueName: \"kubernetes.io/projected/66247a82-0118-4ced-87d2-5f65c49557dc-kube-api-access-nrkff\") pod \"insights-runtime-extractor-5vhfm\" (UID: \"66247a82-0118-4ced-87d2-5f65c49557dc\") " pod="openshift-insights/insights-runtime-extractor-5vhfm"
Sep 10 15:27:51.654849 ip-10-0-128-241 kubenswrapper[2567]: E0910 15:27:51.654774    2567 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 10 15:27:51.654898 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:27:51.654844    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/66247a82-0118-4ced-87d2-5f65c49557dc-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-5vhfm\" (UID: \"66247a82-0118-4ced-87d2-5f65c49557dc\") " pod="openshift-insights/insights-runtime-extractor-5vhfm"
Sep 10 15:27:51.654898 ip-10-0-128-241 kubenswrapper[2567]: E0910 15:27:51.654854    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/66247a82-0118-4ced-87d2-5f65c49557dc-insights-runtime-extractor-tls podName:66247a82-0118-4ced-87d2-5f65c49557dc nodeName:}" failed. No retries permitted until 2026-09-10 15:27:52.154829952 +0000 UTC m=+57.723061676 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/66247a82-0118-4ced-87d2-5f65c49557dc-insights-runtime-extractor-tls") pod "insights-runtime-extractor-5vhfm" (UID: "66247a82-0118-4ced-87d2-5f65c49557dc") : secret "insights-runtime-extractor-tls" not found
Sep 10 15:27:51.655003 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:27:51.654898    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/66247a82-0118-4ced-87d2-5f65c49557dc-crio-socket\") pod \"insights-runtime-extractor-5vhfm\" (UID: \"66247a82-0118-4ced-87d2-5f65c49557dc\") " pod="openshift-insights/insights-runtime-extractor-5vhfm"
Sep 10 15:27:51.655003 ip-10-0-128-241 kubenswrapper[2567]: E0910 15:27:51.654946    2567 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 10 15:27:51.655093 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:27:51.654995    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/66247a82-0118-4ced-87d2-5f65c49557dc-data-volume\") pod \"insights-runtime-extractor-5vhfm\" (UID: \"66247a82-0118-4ced-87d2-5f65c49557dc\") " pod="openshift-insights/insights-runtime-extractor-5vhfm"
Sep 10 15:27:51.655093 ip-10-0-128-241 kubenswrapper[2567]: E0910 15:27:51.655017    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/66247a82-0118-4ced-87d2-5f65c49557dc-kube-rbac-proxy-cm podName:66247a82-0118-4ced-87d2-5f65c49557dc nodeName:}" failed. No retries permitted until 2026-09-10 15:27:52.155000317 +0000 UTC m=+57.723232025 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/66247a82-0118-4ced-87d2-5f65c49557dc-kube-rbac-proxy-cm") pod "insights-runtime-extractor-5vhfm" (UID: "66247a82-0118-4ced-87d2-5f65c49557dc") : configmap "kube-rbac-proxy" not found
Sep 10 15:27:51.655093 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:27:51.655035    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/66247a82-0118-4ced-87d2-5f65c49557dc-crio-socket\") pod \"insights-runtime-extractor-5vhfm\" (UID: \"66247a82-0118-4ced-87d2-5f65c49557dc\") " pod="openshift-insights/insights-runtime-extractor-5vhfm"
Sep 10 15:27:51.666945 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:27:51.666915    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nrkff\" (UniqueName: \"kubernetes.io/projected/66247a82-0118-4ced-87d2-5f65c49557dc-kube-api-access-nrkff\") pod \"insights-runtime-extractor-5vhfm\" (UID: \"66247a82-0118-4ced-87d2-5f65c49557dc\") " pod="openshift-insights/insights-runtime-extractor-5vhfm"
Sep 10 15:27:52.159500 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:27:52.159456    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/66247a82-0118-4ced-87d2-5f65c49557dc-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-5vhfm\" (UID: \"66247a82-0118-4ced-87d2-5f65c49557dc\") " pod="openshift-insights/insights-runtime-extractor-5vhfm"
Sep 10 15:27:52.159693 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:27:52.159538    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/66247a82-0118-4ced-87d2-5f65c49557dc-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-5vhfm\" (UID: \"66247a82-0118-4ced-87d2-5f65c49557dc\") " pod="openshift-insights/insights-runtime-extractor-5vhfm"
Sep 10 15:27:52.159693 ip-10-0-128-241 kubenswrapper[2567]: E0910 15:27:52.159612    2567 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 10 15:27:52.159693 ip-10-0-128-241 kubenswrapper[2567]: E0910 15:27:52.159651    2567 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 10 15:27:52.159693 ip-10-0-128-241 kubenswrapper[2567]: E0910 15:27:52.159684    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/66247a82-0118-4ced-87d2-5f65c49557dc-insights-runtime-extractor-tls podName:66247a82-0118-4ced-87d2-5f65c49557dc nodeName:}" failed. No retries permitted until 2026-09-10 15:27:53.159667618 +0000 UTC m=+58.727899325 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/66247a82-0118-4ced-87d2-5f65c49557dc-insights-runtime-extractor-tls") pod "insights-runtime-extractor-5vhfm" (UID: "66247a82-0118-4ced-87d2-5f65c49557dc") : secret "insights-runtime-extractor-tls" not found
Sep 10 15:27:52.159852 ip-10-0-128-241 kubenswrapper[2567]: E0910 15:27:52.159715    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/66247a82-0118-4ced-87d2-5f65c49557dc-kube-rbac-proxy-cm podName:66247a82-0118-4ced-87d2-5f65c49557dc nodeName:}" failed. No retries permitted until 2026-09-10 15:27:53.159703097 +0000 UTC m=+58.727934805 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/66247a82-0118-4ced-87d2-5f65c49557dc-kube-rbac-proxy-cm") pod "insights-runtime-extractor-5vhfm" (UID: "66247a82-0118-4ced-87d2-5f65c49557dc") : configmap "kube-rbac-proxy" not found
Sep 10 15:27:52.197556 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:27:52.197531    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-t96pm_0b9d2cda-73e3-4e7c-9e17-1a3d903d95cf/console-operator/1.log"
Sep 10 15:27:52.197878 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:27:52.197852    2567 scope.go:117] "RemoveContainer" containerID="8639f4fbdd150a963310dbb5b8cd98b7c798e8f9daf57eb7e0c3c0ee03e8ff44"
Sep 10 15:27:52.198036 ip-10-0-128-241 kubenswrapper[2567]: E0910 15:27:52.198018    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-85b454874b-t96pm_openshift-console-operator(0b9d2cda-73e3-4e7c-9e17-1a3d903d95cf)\"" pod="openshift-console-operator/console-operator-85b454874b-t96pm" podUID="0b9d2cda-73e3-4e7c-9e17-1a3d903d95cf"
Sep 10 15:27:53.167681 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:27:53.167639    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/66247a82-0118-4ced-87d2-5f65c49557dc-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-5vhfm\" (UID: \"66247a82-0118-4ced-87d2-5f65c49557dc\") " pod="openshift-insights/insights-runtime-extractor-5vhfm"
Sep 10 15:27:53.168087 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:27:53.167757    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/66247a82-0118-4ced-87d2-5f65c49557dc-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-5vhfm\" (UID: \"66247a82-0118-4ced-87d2-5f65c49557dc\") " pod="openshift-insights/insights-runtime-extractor-5vhfm"
Sep 10 15:27:53.168087 ip-10-0-128-241 kubenswrapper[2567]: E0910 15:27:53.167769    2567 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 10 15:27:53.168087 ip-10-0-128-241 kubenswrapper[2567]: E0910 15:27:53.167838    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/66247a82-0118-4ced-87d2-5f65c49557dc-kube-rbac-proxy-cm podName:66247a82-0118-4ced-87d2-5f65c49557dc nodeName:}" failed. No retries permitted until 2026-09-10 15:27:55.167817058 +0000 UTC m=+60.736048772 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/66247a82-0118-4ced-87d2-5f65c49557dc-kube-rbac-proxy-cm") pod "insights-runtime-extractor-5vhfm" (UID: "66247a82-0118-4ced-87d2-5f65c49557dc") : configmap "kube-rbac-proxy" not found
Sep 10 15:27:53.168087 ip-10-0-128-241 kubenswrapper[2567]: E0910 15:27:53.167867    2567 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 10 15:27:53.168087 ip-10-0-128-241 kubenswrapper[2567]: E0910 15:27:53.167902    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/66247a82-0118-4ced-87d2-5f65c49557dc-insights-runtime-extractor-tls podName:66247a82-0118-4ced-87d2-5f65c49557dc nodeName:}" failed. No retries permitted until 2026-09-10 15:27:55.167891777 +0000 UTC m=+60.736123500 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/66247a82-0118-4ced-87d2-5f65c49557dc-insights-runtime-extractor-tls") pod "insights-runtime-extractor-5vhfm" (UID: "66247a82-0118-4ced-87d2-5f65c49557dc") : secret "insights-runtime-extractor-tls" not found
Sep 10 15:27:53.596582 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:27:53.596554    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-rctzd_4dd1b47c-e497-45d2-a404-4ea3a2d8ec94/dns-node-resolver/0.log"
Sep 10 15:27:53.996615 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:27:53.996588    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-g8rkv_e9b09c08-aec9-41db-8386-e62731150ccc/node-ca/0.log"
Sep 10 15:27:54.996567 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:27:54.996528    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:27:55.002597 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:27:55.002570    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-fdbb8fdff-zf25j"]
Sep 10 15:27:55.024730 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:27:55.024707    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-fdbb8fdff-zf25j"]
Sep 10 15:27:55.024861 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:27:55.024801    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-zf25j"
Sep 10 15:27:55.027707 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:27:55.027687    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\""
Sep 10 15:27:55.027859 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:27:55.027780    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-gdf49\""
Sep 10 15:27:55.028726 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:27:55.028713    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\""
Sep 10 15:27:55.083166 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:27:55.083131    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k9b4c\" (UniqueName: \"kubernetes.io/projected/81a9b4ad-8e9d-4813-a613-f3628103d69f-kube-api-access-k9b4c\") pod \"migrator-fdbb8fdff-zf25j\" (UID: \"81a9b4ad-8e9d-4813-a613-f3628103d69f\") " pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-zf25j"
Sep 10 15:27:55.100878 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:27:55.100847    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:27:55.127482 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:27:55.127456    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-bctsx"
Sep 10 15:27:55.183599 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:27:55.183564    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k9b4c\" (UniqueName: \"kubernetes.io/projected/81a9b4ad-8e9d-4813-a613-f3628103d69f-kube-api-access-k9b4c\") pod \"migrator-fdbb8fdff-zf25j\" (UID: \"81a9b4ad-8e9d-4813-a613-f3628103d69f\") " pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-zf25j"
Sep 10 15:27:55.183771 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:27:55.183633    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/66247a82-0118-4ced-87d2-5f65c49557dc-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-5vhfm\" (UID: \"66247a82-0118-4ced-87d2-5f65c49557dc\") " pod="openshift-insights/insights-runtime-extractor-5vhfm"
Sep 10 15:27:55.183771 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:27:55.183738    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/66247a82-0118-4ced-87d2-5f65c49557dc-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-5vhfm\" (UID: \"66247a82-0118-4ced-87d2-5f65c49557dc\") " pod="openshift-insights/insights-runtime-extractor-5vhfm"
Sep 10 15:27:55.183883 ip-10-0-128-241 kubenswrapper[2567]: E0910 15:27:55.183862    2567 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 10 15:27:55.183940 ip-10-0-128-241 kubenswrapper[2567]: E0910 15:27:55.183885    2567 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 10 15:27:55.183940 ip-10-0-128-241 kubenswrapper[2567]: E0910 15:27:55.183935    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/66247a82-0118-4ced-87d2-5f65c49557dc-insights-runtime-extractor-tls podName:66247a82-0118-4ced-87d2-5f65c49557dc nodeName:}" failed. No retries permitted until 2026-09-10 15:27:59.183913478 +0000 UTC m=+64.752145189 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/66247a82-0118-4ced-87d2-5f65c49557dc-insights-runtime-extractor-tls") pod "insights-runtime-extractor-5vhfm" (UID: "66247a82-0118-4ced-87d2-5f65c49557dc") : secret "insights-runtime-extractor-tls" not found
Sep 10 15:27:55.184039 ip-10-0-128-241 kubenswrapper[2567]: E0910 15:27:55.183969    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/66247a82-0118-4ced-87d2-5f65c49557dc-kube-rbac-proxy-cm podName:66247a82-0118-4ced-87d2-5f65c49557dc nodeName:}" failed. No retries permitted until 2026-09-10 15:27:59.183947937 +0000 UTC m=+64.752179670 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/66247a82-0118-4ced-87d2-5f65c49557dc-kube-rbac-proxy-cm") pod "insights-runtime-extractor-5vhfm" (UID: "66247a82-0118-4ced-87d2-5f65c49557dc") : configmap "kube-rbac-proxy" not found
Sep 10 15:27:55.194063 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:27:55.194043    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k9b4c\" (UniqueName: \"kubernetes.io/projected/81a9b4ad-8e9d-4813-a613-f3628103d69f-kube-api-access-k9b4c\") pod \"migrator-fdbb8fdff-zf25j\" (UID: \"81a9b4ad-8e9d-4813-a613-f3628103d69f\") " pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-zf25j"
Sep 10 15:27:55.196097 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:27:55.196068    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-h8h5g_b9ea6621-b91a-44bd-a38f-666177f4d26b/serve-healthcheck-canary/0.log"
Sep 10 15:27:55.198796 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:27:55.198773    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:27:55.299630 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:27:55.299533    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:27:55.333397 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:27:55.333360    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-zf25j"
Sep 10 15:27:55.370994 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:27:55.370949    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:27:55.602211 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:27:55.602005    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-fdbb8fdff-zf25j"]
Sep 10 15:27:55.604714 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:27:55.604692    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod81a9b4ad_8e9d_4813_a613_f3628103d69f.slice/crio-90c1d65e15ddaf0b9d4a53fe735039dd7e3437a0614c46af726855d5f463b253 WatchSource:0}: Error finding container 90c1d65e15ddaf0b9d4a53fe735039dd7e3437a0614c46af726855d5f463b253: Status 404 returned error can't find the container with id 90c1d65e15ddaf0b9d4a53fe735039dd7e3437a0614c46af726855d5f463b253
Sep 10 15:27:56.207372 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:27:56.207338    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-zf25j" event={"ID":"81a9b4ad-8e9d-4813-a613-f3628103d69f","Type":"ContainerStarted","Data":"90c1d65e15ddaf0b9d4a53fe735039dd7e3437a0614c46af726855d5f463b253"}
Sep 10 15:27:57.211707 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:27:57.211674    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-zf25j" event={"ID":"81a9b4ad-8e9d-4813-a613-f3628103d69f","Type":"ContainerStarted","Data":"a6222e6b80528be0d4b2f82a7ee052ea377d4ee13f0782d057b591786802e236"}
Sep 10 15:27:57.212088 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:27:57.211754    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-zf25j" event={"ID":"81a9b4ad-8e9d-4813-a613-f3628103d69f","Type":"ContainerStarted","Data":"8f6fe4ab90e89ff248d615d029d4a610c7271fcf3c9b3fd34176f91d12795d43"}
Sep 10 15:27:57.230943 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:27:57.230894    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-zf25j" podStartSLOduration=2.044880198 podStartE2EDuration="3.230878668s" totalImagesPullingTime="1.18599847s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 15:27:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 15:27:57.230456295 +0000 UTC m=+62.798688026" watchObservedRunningTime="2026-09-10 15:27:57.230878668 +0000 UTC m=+62.799110398"
Sep 10 15:27:57.321559 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:27:57.321531    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:27:57.323401 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:27:57.323382    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-85b454874b-t96pm"
Sep 10 15:27:57.323452 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:27:57.323429    2567 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-85b454874b-t96pm"
Sep 10 15:27:57.323765 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:27:57.323753    2567 scope.go:117] "RemoveContainer" containerID="8639f4fbdd150a963310dbb5b8cd98b7c798e8f9daf57eb7e0c3c0ee03e8ff44"
Sep 10 15:27:57.323921 ip-10-0-128-241 kubenswrapper[2567]: E0910 15:27:57.323905    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-85b454874b-t96pm_openshift-console-operator(0b9d2cda-73e3-4e7c-9e17-1a3d903d95cf)\"" pod="openshift-console-operator/console-operator-85b454874b-t96pm" podUID="0b9d2cda-73e3-4e7c-9e17-1a3d903d95cf"
Sep 10 15:27:57.420898 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:27:57.420865    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:27:59.218525 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:27:59.218490    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/66247a82-0118-4ced-87d2-5f65c49557dc-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-5vhfm\" (UID: \"66247a82-0118-4ced-87d2-5f65c49557dc\") " pod="openshift-insights/insights-runtime-extractor-5vhfm"
Sep 10 15:27:59.218887 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:27:59.218554    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/66247a82-0118-4ced-87d2-5f65c49557dc-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-5vhfm\" (UID: \"66247a82-0118-4ced-87d2-5f65c49557dc\") " pod="openshift-insights/insights-runtime-extractor-5vhfm"
Sep 10 15:27:59.218887 ip-10-0-128-241 kubenswrapper[2567]: E0910 15:27:59.218640    2567 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 10 15:27:59.218887 ip-10-0-128-241 kubenswrapper[2567]: E0910 15:27:59.218659    2567 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 10 15:27:59.218887 ip-10-0-128-241 kubenswrapper[2567]: E0910 15:27:59.218715    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/66247a82-0118-4ced-87d2-5f65c49557dc-insights-runtime-extractor-tls podName:66247a82-0118-4ced-87d2-5f65c49557dc nodeName:}" failed. No retries permitted until 2026-09-10 15:28:07.218695553 +0000 UTC m=+72.786927274 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/66247a82-0118-4ced-87d2-5f65c49557dc-insights-runtime-extractor-tls") pod "insights-runtime-extractor-5vhfm" (UID: "66247a82-0118-4ced-87d2-5f65c49557dc") : secret "insights-runtime-extractor-tls" not found
Sep 10 15:27:59.218887 ip-10-0-128-241 kubenswrapper[2567]: E0910 15:27:59.218730    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/66247a82-0118-4ced-87d2-5f65c49557dc-kube-rbac-proxy-cm podName:66247a82-0118-4ced-87d2-5f65c49557dc nodeName:}" failed. No retries permitted until 2026-09-10 15:28:07.21872356 +0000 UTC m=+72.786955268 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/66247a82-0118-4ced-87d2-5f65c49557dc-kube-rbac-proxy-cm") pod "insights-runtime-extractor-5vhfm" (UID: "66247a82-0118-4ced-87d2-5f65c49557dc") : configmap "kube-rbac-proxy" not found
Sep 10 15:27:59.248691 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:27:59.248649    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:27:59.346499 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:27:59.346470    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:27:59.675090 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:27:59.675062    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:27:59.733595 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:27:59.733567    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:27:59.947131 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:27:59.947033    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:28:00.048434 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:00.048403    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:28:00.628682 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:00.628636    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f91d1ea8-2355-4b51-b736-448a89aba131-metrics-certs\") pod \"network-metrics-daemon-zxpwj\" (UID: \"f91d1ea8-2355-4b51-b736-448a89aba131\") " pod="openshift-multus/network-metrics-daemon-zxpwj"
Sep 10 15:28:00.628682 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:00.628688    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5ef1f4eb-6292-4a77-8515-48e1671d8d13-registry-tls\") pod \"image-registry-659bc4b685-mpbnt\" (UID: \"5ef1f4eb-6292-4a77-8515-48e1671d8d13\") " pod="openshift-image-registry/image-registry-659bc4b685-mpbnt"
Sep 10 15:28:00.629110 ip-10-0-128-241 kubenswrapper[2567]: E0910 15:28:00.628793    2567 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 10 15:28:00.629110 ip-10-0-128-241 kubenswrapper[2567]: E0910 15:28:00.628809    2567 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-659bc4b685-mpbnt: secret "image-registry-tls" not found
Sep 10 15:28:00.629110 ip-10-0-128-241 kubenswrapper[2567]: E0910 15:28:00.628864    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5ef1f4eb-6292-4a77-8515-48e1671d8d13-registry-tls podName:5ef1f4eb-6292-4a77-8515-48e1671d8d13 nodeName:}" failed. No retries permitted until 2026-09-10 15:28:32.628847531 +0000 UTC m=+98.197079257 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/5ef1f4eb-6292-4a77-8515-48e1671d8d13-registry-tls") pod "image-registry-659bc4b685-mpbnt" (UID: "5ef1f4eb-6292-4a77-8515-48e1671d8d13") : secret "image-registry-tls" not found
Sep 10 15:28:00.631409 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:00.631393    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 10 15:28:00.639635 ip-10-0-128-241 kubenswrapper[2567]: E0910 15:28:00.639618    2567 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Sep 10 15:28:00.639677 ip-10-0-128-241 kubenswrapper[2567]: E0910 15:28:00.639666    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f91d1ea8-2355-4b51-b736-448a89aba131-metrics-certs podName:f91d1ea8-2355-4b51-b736-448a89aba131 nodeName:}" failed. No retries permitted until 2026-09-10 15:29:04.639653049 +0000 UTC m=+130.207884770 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f91d1ea8-2355-4b51-b736-448a89aba131-metrics-certs") pod "network-metrics-daemon-zxpwj" (UID: "f91d1ea8-2355-4b51-b736-448a89aba131") : secret "metrics-daemon-secret" not found
Sep 10 15:28:00.729863 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:00.729810    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/49bbf894-04df-427c-b71a-c108fbf25054-original-pull-secret\") pod \"global-pull-secret-syncer-fgx9b\" (UID: \"49bbf894-04df-427c-b71a-c108fbf25054\") " pod="kube-system/global-pull-secret-syncer-fgx9b"
Sep 10 15:28:00.730024 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:00.729886    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9818ae98-700a-4e5c-a4a5-24c0fc624542-metrics-tls\") pod \"dns-default-4k8rd\" (UID: \"9818ae98-700a-4e5c-a4a5-24c0fc624542\") " pod="openshift-dns/dns-default-4k8rd"
Sep 10 15:28:00.730024 ip-10-0-128-241 kubenswrapper[2567]: E0910 15:28:00.729986    2567 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 10 15:28:00.730087 ip-10-0-128-241 kubenswrapper[2567]: E0910 15:28:00.730049    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9818ae98-700a-4e5c-a4a5-24c0fc624542-metrics-tls podName:9818ae98-700a-4e5c-a4a5-24c0fc624542 nodeName:}" failed. No retries permitted until 2026-09-10 15:28:32.730034119 +0000 UTC m=+98.298265831 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/9818ae98-700a-4e5c-a4a5-24c0fc624542-metrics-tls") pod "dns-default-4k8rd" (UID: "9818ae98-700a-4e5c-a4a5-24c0fc624542") : secret "dns-default-metrics-tls" not found
Sep 10 15:28:00.732764 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:00.732745    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 10 15:28:00.742738 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:00.742718    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/49bbf894-04df-427c-b71a-c108fbf25054-original-pull-secret\") pod \"global-pull-secret-syncer-fgx9b\" (UID: \"49bbf894-04df-427c-b71a-c108fbf25054\") " pod="kube-system/global-pull-secret-syncer-fgx9b"
Sep 10 15:28:00.816365 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:00.816307    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fgx9b"
Sep 10 15:28:00.830709 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:00.830682    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4wmx2\" (UniqueName: \"kubernetes.io/projected/e105e176-14c2-4093-946d-492879428ef7-kube-api-access-4wmx2\") pod \"network-check-target-2f7vt\" (UID: \"e105e176-14c2-4093-946d-492879428ef7\") " pod="openshift-network-diagnostics/network-check-target-2f7vt"
Sep 10 15:28:00.833270 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:00.833251    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 10 15:28:00.843465 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:00.843443    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 10 15:28:00.853603 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:00.853573    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4wmx2\" (UniqueName: \"kubernetes.io/projected/e105e176-14c2-4093-946d-492879428ef7-kube-api-access-4wmx2\") pod \"network-check-target-2f7vt\" (UID: \"e105e176-14c2-4093-946d-492879428ef7\") " pod="openshift-network-diagnostics/network-check-target-2f7vt"
Sep 10 15:28:01.088701 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:01.088666    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-fgx9b"]
Sep 10 15:28:01.091933 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:28:01.091905    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod49bbf894_04df_427c_b71a_c108fbf25054.slice/crio-027d815ba8995f4070451801d68538de17b6d11d9ee4fab02f66d2122413684e WatchSource:0}: Error finding container 027d815ba8995f4070451801d68538de17b6d11d9ee4fab02f66d2122413684e: Status 404 returned error can't find the container with id 027d815ba8995f4070451801d68538de17b6d11d9ee4fab02f66d2122413684e
Sep 10 15:28:01.113663 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:01.113636    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-5rkds\""
Sep 10 15:28:01.121634 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:01.121612    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2f7vt"
Sep 10 15:28:01.223047 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:01.223015    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-fgx9b" event={"ID":"49bbf894-04df-427c-b71a-c108fbf25054","Type":"ContainerStarted","Data":"027d815ba8995f4070451801d68538de17b6d11d9ee4fab02f66d2122413684e"}
Sep 10 15:28:01.389530 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:01.389437    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-2f7vt"]
Sep 10 15:28:01.393250 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:28:01.393204    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode105e176_14c2_4093_946d_492879428ef7.slice/crio-f514743c7827854213ca2ca71dec0b16d37c7e55d67ab04330d2b18cec5ac3f5 WatchSource:0}: Error finding container f514743c7827854213ca2ca71dec0b16d37c7e55d67ab04330d2b18cec5ac3f5: Status 404 returned error can't find the container with id f514743c7827854213ca2ca71dec0b16d37c7e55d67ab04330d2b18cec5ac3f5
Sep 10 15:28:01.612934 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:01.612898    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-75d99c484-k29gq"]
Sep 10 15:28:01.615427 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:01.615405    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-75d99c484-k29gq"
Sep 10 15:28:01.618050 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:01.618026    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-frtpv\""
Sep 10 15:28:01.618328 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:01.618313    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\""
Sep 10 15:28:01.619125 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:01.619107    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\""
Sep 10 15:28:01.619182 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:01.619124    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\""
Sep 10 15:28:01.619449 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:01.619424    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\""
Sep 10 15:28:01.623827 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:01.623807    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-75d99c484-k29gq"]
Sep 10 15:28:01.635709 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:01.635672    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:28:01.705167 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:01.705127    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:28:01.740464 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:01.740432    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/24d8e2c4-8b2c-460e-9eb3-e6b06ecf82ff-signing-key\") pod \"service-ca-75d99c484-k29gq\" (UID: \"24d8e2c4-8b2c-460e-9eb3-e6b06ecf82ff\") " pod="openshift-service-ca/service-ca-75d99c484-k29gq"
Sep 10 15:28:01.740598 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:01.740466    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/24d8e2c4-8b2c-460e-9eb3-e6b06ecf82ff-signing-cabundle\") pod \"service-ca-75d99c484-k29gq\" (UID: \"24d8e2c4-8b2c-460e-9eb3-e6b06ecf82ff\") " pod="openshift-service-ca/service-ca-75d99c484-k29gq"
Sep 10 15:28:01.740598 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:01.740534    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l668w\" (UniqueName: \"kubernetes.io/projected/24d8e2c4-8b2c-460e-9eb3-e6b06ecf82ff-kube-api-access-l668w\") pod \"service-ca-75d99c484-k29gq\" (UID: \"24d8e2c4-8b2c-460e-9eb3-e6b06ecf82ff\") " pod="openshift-service-ca/service-ca-75d99c484-k29gq"
Sep 10 15:28:01.841039 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:01.840997    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l668w\" (UniqueName: \"kubernetes.io/projected/24d8e2c4-8b2c-460e-9eb3-e6b06ecf82ff-kube-api-access-l668w\") pod \"service-ca-75d99c484-k29gq\" (UID: \"24d8e2c4-8b2c-460e-9eb3-e6b06ecf82ff\") " pod="openshift-service-ca/service-ca-75d99c484-k29gq"
Sep 10 15:28:01.841226 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:01.841077    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/24d8e2c4-8b2c-460e-9eb3-e6b06ecf82ff-signing-key\") pod \"service-ca-75d99c484-k29gq\" (UID: \"24d8e2c4-8b2c-460e-9eb3-e6b06ecf82ff\") " pod="openshift-service-ca/service-ca-75d99c484-k29gq"
Sep 10 15:28:01.841298 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:01.841253    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/24d8e2c4-8b2c-460e-9eb3-e6b06ecf82ff-signing-cabundle\") pod \"service-ca-75d99c484-k29gq\" (UID: \"24d8e2c4-8b2c-460e-9eb3-e6b06ecf82ff\") " pod="openshift-service-ca/service-ca-75d99c484-k29gq"
Sep 10 15:28:01.841822 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:01.841804    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/24d8e2c4-8b2c-460e-9eb3-e6b06ecf82ff-signing-cabundle\") pod \"service-ca-75d99c484-k29gq\" (UID: \"24d8e2c4-8b2c-460e-9eb3-e6b06ecf82ff\") " pod="openshift-service-ca/service-ca-75d99c484-k29gq"
Sep 10 15:28:01.843415 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:01.843396    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/24d8e2c4-8b2c-460e-9eb3-e6b06ecf82ff-signing-key\") pod \"service-ca-75d99c484-k29gq\" (UID: \"24d8e2c4-8b2c-460e-9eb3-e6b06ecf82ff\") " pod="openshift-service-ca/service-ca-75d99c484-k29gq"
Sep 10 15:28:01.851325 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:01.851300    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l668w\" (UniqueName: \"kubernetes.io/projected/24d8e2c4-8b2c-460e-9eb3-e6b06ecf82ff-kube-api-access-l668w\") pod \"service-ca-75d99c484-k29gq\" (UID: \"24d8e2c4-8b2c-460e-9eb3-e6b06ecf82ff\") " pod="openshift-service-ca/service-ca-75d99c484-k29gq"
Sep 10 15:28:01.924900 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:01.924860    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-75d99c484-k29gq"
Sep 10 15:28:02.207694 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:02.207661    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-75d99c484-k29gq"]
Sep 10 15:28:02.211515 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:28:02.211483    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod24d8e2c4_8b2c_460e_9eb3_e6b06ecf82ff.slice/crio-4a99ab76e0c42aab5c492aa4c3d690574ff0cc1679ba99f5f1c0655cceba6a7b WatchSource:0}: Error finding container 4a99ab76e0c42aab5c492aa4c3d690574ff0cc1679ba99f5f1c0655cceba6a7b: Status 404 returned error can't find the container with id 4a99ab76e0c42aab5c492aa4c3d690574ff0cc1679ba99f5f1c0655cceba6a7b
Sep 10 15:28:02.227130 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:02.227040    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-75d99c484-k29gq" event={"ID":"24d8e2c4-8b2c-460e-9eb3-e6b06ecf82ff","Type":"ContainerStarted","Data":"4a99ab76e0c42aab5c492aa4c3d690574ff0cc1679ba99f5f1c0655cceba6a7b"}
Sep 10 15:28:02.228630 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:02.228602    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-2f7vt" event={"ID":"e105e176-14c2-4093-946d-492879428ef7","Type":"ContainerStarted","Data":"f514743c7827854213ca2ca71dec0b16d37c7e55d67ab04330d2b18cec5ac3f5"}
Sep 10 15:28:02.936107 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:02.936071    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:28:04.600880 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:04.600837    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:28:05.884061 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:05.884023    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:28:06.125218 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:06.125179    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:28:06.190929 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:06.190890    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:28:07.286824 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:07.286785    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/66247a82-0118-4ced-87d2-5f65c49557dc-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-5vhfm\" (UID: \"66247a82-0118-4ced-87d2-5f65c49557dc\") " pod="openshift-insights/insights-runtime-extractor-5vhfm"
Sep 10 15:28:07.287287 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:07.286850    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/66247a82-0118-4ced-87d2-5f65c49557dc-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-5vhfm\" (UID: \"66247a82-0118-4ced-87d2-5f65c49557dc\") " pod="openshift-insights/insights-runtime-extractor-5vhfm"
Sep 10 15:28:07.287287 ip-10-0-128-241 kubenswrapper[2567]: E0910 15:28:07.286953    2567 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 10 15:28:07.287287 ip-10-0-128-241 kubenswrapper[2567]: E0910 15:28:07.287018    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/66247a82-0118-4ced-87d2-5f65c49557dc-insights-runtime-extractor-tls podName:66247a82-0118-4ced-87d2-5f65c49557dc nodeName:}" failed. No retries permitted until 2026-09-10 15:28:23.287000027 +0000 UTC m=+88.855231735 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/66247a82-0118-4ced-87d2-5f65c49557dc-insights-runtime-extractor-tls") pod "insights-runtime-extractor-5vhfm" (UID: "66247a82-0118-4ced-87d2-5f65c49557dc") : secret "insights-runtime-extractor-tls" not found
Sep 10 15:28:07.287801 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:07.287732    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/66247a82-0118-4ced-87d2-5f65c49557dc-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-5vhfm\" (UID: \"66247a82-0118-4ced-87d2-5f65c49557dc\") " pod="openshift-insights/insights-runtime-extractor-5vhfm"
Sep 10 15:28:08.248006 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:08.247972    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-fgx9b" event={"ID":"49bbf894-04df-427c-b71a-c108fbf25054","Type":"ContainerStarted","Data":"b543de6677d41c7648c93f86c082b65e22c567be93feb42de779426adde96d7c"}
Sep 10 15:28:08.249491 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:08.249464    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-75d99c484-k29gq" event={"ID":"24d8e2c4-8b2c-460e-9eb3-e6b06ecf82ff","Type":"ContainerStarted","Data":"39ecf4e448cacadf3988443643279fd94369cd4b03d4521b22e41327bb2318a0"}
Sep 10 15:28:08.250942 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:08.250924    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-2f7vt" event={"ID":"e105e176-14c2-4093-946d-492879428ef7","Type":"ContainerStarted","Data":"76e681a24655dee6cbd67d34936a908881f2f0e10324d089e6b9eb9dc4c71a4f"}
Sep 10 15:28:08.251063 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:08.251050    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-2f7vt"
Sep 10 15:28:08.264527 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:08.264481    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-fgx9b" podStartSLOduration=66.220021863 podStartE2EDuration="1m12.26446449s" totalImagesPullingTime="6.044442627s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 15:26:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 15:28:08.264293188 +0000 UTC m=+73.832524931" watchObservedRunningTime="2026-09-10 15:28:08.26446449 +0000 UTC m=+73.832696215"
Sep 10 15:28:08.281522 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:08.281473    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-75d99c484-k29gq" podStartSLOduration=2.369651427 podStartE2EDuration="7.281459067s" totalImagesPullingTime="4.91180764s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 15:28:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 15:28:08.281126204 +0000 UTC m=+73.849357938" watchObservedRunningTime="2026-09-10 15:28:08.281459067 +0000 UTC m=+73.849690797"
Sep 10 15:28:08.303363 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:08.303314    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-2f7vt" podStartSLOduration=67.573319907 podStartE2EDuration="1m13.303295368s" totalImagesPullingTime="5.729975461s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 15:26:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 15:28:08.30218916 +0000 UTC m=+73.870420892" watchObservedRunningTime="2026-09-10 15:28:08.303295368 +0000 UTC m=+73.871527124"
Sep 10 15:28:09.124484 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:09.124446    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:28:09.345675 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:09.345644    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:28:09.496740 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:09.496704    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:28:09.711100 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:09.711064    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:28:09.810245 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:09.810143    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:28:09.911585 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:09.911547    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:28:10.424737 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:10.424702    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:28:10.818583 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:10.818508    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:28:11.421541 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:11.421503    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:28:11.619802 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:11.619746    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:28:11.821957 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:11.821872    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:28:12.761250 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:12.761199    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:28:12.862462 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:12.862429    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:28:12.898688 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:12.898660    2567 scope.go:117] "RemoveContainer" containerID="8639f4fbdd150a963310dbb5b8cd98b7c798e8f9daf57eb7e0c3c0ee03e8ff44"
Sep 10 15:28:13.218407 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:13.218368    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:28:13.271900 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:13.271872    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-t96pm_0b9d2cda-73e3-4e7c-9e17-1a3d903d95cf/console-operator/2.log"
Sep 10 15:28:13.272367 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:13.272349    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-t96pm_0b9d2cda-73e3-4e7c-9e17-1a3d903d95cf/console-operator/1.log"
Sep 10 15:28:13.272447 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:13.272392    2567 generic.go:356] "Generic (PLEG): container finished" podID="0b9d2cda-73e3-4e7c-9e17-1a3d903d95cf" containerID="967ccc3b2063b8bf7949bddfce0f7e483fd3bb5dce81442e082c3f83d7e8fa9e" exitCode=255
Sep 10 15:28:13.272447 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:13.272436    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-85b454874b-t96pm" event={"ID":"0b9d2cda-73e3-4e7c-9e17-1a3d903d95cf","Type":"ContainerDied","Data":"967ccc3b2063b8bf7949bddfce0f7e483fd3bb5dce81442e082c3f83d7e8fa9e"}
Sep 10 15:28:13.272539 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:13.272525    2567 scope.go:117] "RemoveContainer" containerID="8639f4fbdd150a963310dbb5b8cd98b7c798e8f9daf57eb7e0c3c0ee03e8ff44"
Sep 10 15:28:13.272866 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:13.272846    2567 scope.go:117] "RemoveContainer" containerID="967ccc3b2063b8bf7949bddfce0f7e483fd3bb5dce81442e082c3f83d7e8fa9e"
Sep 10 15:28:13.273070 ip-10-0-128-241 kubenswrapper[2567]: E0910 15:28:13.273034    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-85b454874b-t96pm_openshift-console-operator(0b9d2cda-73e3-4e7c-9e17-1a3d903d95cf)\"" pod="openshift-console-operator/console-operator-85b454874b-t96pm" podUID="0b9d2cda-73e3-4e7c-9e17-1a3d903d95cf"
Sep 10 15:28:13.427527 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:13.427476    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:28:13.627754 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:13.627662    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:28:13.688938 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:13.688900    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:28:13.833218 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:13.833185    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:28:14.277113 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:14.277086    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-t96pm_0b9d2cda-73e3-4e7c-9e17-1a3d903d95cf/console-operator/2.log"
Sep 10 15:28:14.416517 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:14.416487    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:28:15.029099 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:15.029068    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:28:15.220435 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:15.220400    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:28:15.387303 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:15.387211    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:28:15.488756 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:15.488706    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:28:15.622365 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:15.622330    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:28:15.818312 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:15.818262    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:28:16.825955 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:16.825923    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:28:17.041736 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:17.041334    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:28:17.323402 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:17.323361    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-85b454874b-t96pm"
Sep 10 15:28:17.323571 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:17.323454    2567 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-85b454874b-t96pm"
Sep 10 15:28:17.323803 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:17.323786    2567 scope.go:117] "RemoveContainer" containerID="967ccc3b2063b8bf7949bddfce0f7e483fd3bb5dce81442e082c3f83d7e8fa9e"
Sep 10 15:28:17.323983 ip-10-0-128-241 kubenswrapper[2567]: E0910 15:28:17.323963    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-85b454874b-t96pm_openshift-console-operator(0b9d2cda-73e3-4e7c-9e17-1a3d903d95cf)\"" pod="openshift-console-operator/console-operator-85b454874b-t96pm" podUID="0b9d2cda-73e3-4e7c-9e17-1a3d903d95cf"
Sep 10 15:28:17.417113 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:17.417070    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:28:17.620497 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:17.620414    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:28:18.219479 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:18.219423    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:28:18.623014 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:18.622929    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:28:18.820628 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:18.820582    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:28:19.427216 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:19.427178    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:28:19.631469 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:19.631427    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:28:19.820197 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:19.820111    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:28:21.148538 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:21.148503    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:28:21.248423 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:21.248392    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:28:22.168137 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:22.168099    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:28:22.237715 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:22.237677    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:28:22.339579 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:22.339537    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:28:22.515528 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:22.515435    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:28:23.075943 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:23.075909    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-54b68bc5b9-z4thq"]
Sep 10 15:28:23.080031 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:23.080015    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-54b68bc5b9-z4thq"
Sep 10 15:28:23.084005 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:23.083977    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\""
Sep 10 15:28:23.084005 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:23.083986    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Sep 10 15:28:23.084207 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:23.084019    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Sep 10 15:28:23.084207 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:23.084093    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Sep 10 15:28:23.091144 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:23.091123    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-54b68bc5b9-z4thq"]
Sep 10 15:28:23.215164 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:23.215125    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/733f521d-33df-4e2f-8684-048d4cbaa0b1-tmp\") pod \"klusterlet-addon-workmgr-54b68bc5b9-z4thq\" (UID: \"733f521d-33df-4e2f-8684-048d4cbaa0b1\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-54b68bc5b9-z4thq"
Sep 10 15:28:23.215164 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:23.215163    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/733f521d-33df-4e2f-8684-048d4cbaa0b1-klusterlet-config\") pod \"klusterlet-addon-workmgr-54b68bc5b9-z4thq\" (UID: \"733f521d-33df-4e2f-8684-048d4cbaa0b1\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-54b68bc5b9-z4thq"
Sep 10 15:28:23.215612 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:23.215196    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xhpj6\" (UniqueName: \"kubernetes.io/projected/733f521d-33df-4e2f-8684-048d4cbaa0b1-kube-api-access-xhpj6\") pod \"klusterlet-addon-workmgr-54b68bc5b9-z4thq\" (UID: \"733f521d-33df-4e2f-8684-048d4cbaa0b1\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-54b68bc5b9-z4thq"
Sep 10 15:28:23.241318 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:23.241261    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:28:23.316598 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:23.316569    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/733f521d-33df-4e2f-8684-048d4cbaa0b1-tmp\") pod \"klusterlet-addon-workmgr-54b68bc5b9-z4thq\" (UID: \"733f521d-33df-4e2f-8684-048d4cbaa0b1\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-54b68bc5b9-z4thq"
Sep 10 15:28:23.316598 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:23.316601    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/733f521d-33df-4e2f-8684-048d4cbaa0b1-klusterlet-config\") pod \"klusterlet-addon-workmgr-54b68bc5b9-z4thq\" (UID: \"733f521d-33df-4e2f-8684-048d4cbaa0b1\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-54b68bc5b9-z4thq"
Sep 10 15:28:23.316860 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:23.316634    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xhpj6\" (UniqueName: \"kubernetes.io/projected/733f521d-33df-4e2f-8684-048d4cbaa0b1-kube-api-access-xhpj6\") pod \"klusterlet-addon-workmgr-54b68bc5b9-z4thq\" (UID: \"733f521d-33df-4e2f-8684-048d4cbaa0b1\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-54b68bc5b9-z4thq"
Sep 10 15:28:23.316860 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:23.316652    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/66247a82-0118-4ced-87d2-5f65c49557dc-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-5vhfm\" (UID: \"66247a82-0118-4ced-87d2-5f65c49557dc\") " pod="openshift-insights/insights-runtime-extractor-5vhfm"
Sep 10 15:28:23.317052 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:23.317023    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/733f521d-33df-4e2f-8684-048d4cbaa0b1-tmp\") pod \"klusterlet-addon-workmgr-54b68bc5b9-z4thq\" (UID: \"733f521d-33df-4e2f-8684-048d4cbaa0b1\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-54b68bc5b9-z4thq"
Sep 10 15:28:23.318918 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:23.318887    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/66247a82-0118-4ced-87d2-5f65c49557dc-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-5vhfm\" (UID: \"66247a82-0118-4ced-87d2-5f65c49557dc\") " pod="openshift-insights/insights-runtime-extractor-5vhfm"
Sep 10 15:28:23.319558 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:23.319537    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/733f521d-33df-4e2f-8684-048d4cbaa0b1-klusterlet-config\") pod \"klusterlet-addon-workmgr-54b68bc5b9-z4thq\" (UID: \"733f521d-33df-4e2f-8684-048d4cbaa0b1\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-54b68bc5b9-z4thq"
Sep 10 15:28:23.326319 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:23.326263    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xhpj6\" (UniqueName: \"kubernetes.io/projected/733f521d-33df-4e2f-8684-048d4cbaa0b1-kube-api-access-xhpj6\") pod \"klusterlet-addon-workmgr-54b68bc5b9-z4thq\" (UID: \"733f521d-33df-4e2f-8684-048d4cbaa0b1\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-54b68bc5b9-z4thq"
Sep 10 15:28:23.339041 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:23.339014    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:28:23.390208 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:23.390176    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-54b68bc5b9-z4thq"
Sep 10 15:28:23.565668 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:23.565633    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-xrkll\""
Sep 10 15:28:23.573621 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:23.573604    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-5vhfm"
Sep 10 15:28:23.661994 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:23.661955    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-54b68bc5b9-z4thq"]
Sep 10 15:28:23.665458 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:28:23.665435    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod733f521d_33df_4e2f_8684_048d4cbaa0b1.slice/crio-23d155c175eb9ab68072dc756c8c2d1c23cb6678e6dde7c6c69fc1b5d1e15506 WatchSource:0}: Error finding container 23d155c175eb9ab68072dc756c8c2d1c23cb6678e6dde7c6c69fc1b5d1e15506: Status 404 returned error can't find the container with id 23d155c175eb9ab68072dc756c8c2d1c23cb6678e6dde7c6c69fc1b5d1e15506
Sep 10 15:28:23.843750 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:23.843671    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-5vhfm"]
Sep 10 15:28:23.847139 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:28:23.847113    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod66247a82_0118_4ced_87d2_5f65c49557dc.slice/crio-3dd124c3e294e6fee6732507b67bca57d7fe0801c38bbed783749714711c7c2e WatchSource:0}: Error finding container 3dd124c3e294e6fee6732507b67bca57d7fe0801c38bbed783749714711c7c2e: Status 404 returned error can't find the container with id 3dd124c3e294e6fee6732507b67bca57d7fe0801c38bbed783749714711c7c2e
Sep 10 15:28:24.309200 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:24.309168    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-5vhfm" event={"ID":"66247a82-0118-4ced-87d2-5f65c49557dc","Type":"ContainerStarted","Data":"4fd02a03dc2fd343dc3d639fb4f0c072996dc29acfcac04fee22afadedab87cd"}
Sep 10 15:28:24.309648 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:24.309220    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-5vhfm" event={"ID":"66247a82-0118-4ced-87d2-5f65c49557dc","Type":"ContainerStarted","Data":"3dd124c3e294e6fee6732507b67bca57d7fe0801c38bbed783749714711c7c2e"}
Sep 10 15:28:24.310284 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:24.310264    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-54b68bc5b9-z4thq" event={"ID":"733f521d-33df-4e2f-8684-048d4cbaa0b1","Type":"ContainerStarted","Data":"23d155c175eb9ab68072dc756c8c2d1c23cb6678e6dde7c6c69fc1b5d1e15506"}
Sep 10 15:28:25.317574 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:25.317486    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-5vhfm" event={"ID":"66247a82-0118-4ced-87d2-5f65c49557dc","Type":"ContainerStarted","Data":"3b6e13909cc4a92f350224ed44b084c67e991b0ba9474fa9dbdaa5ef0b8287bc"}
Sep 10 15:28:26.244772 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:26.244726    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:28:26.307525 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:26.307482    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:28:27.384831 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:27.384773    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:28:27.461603 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:27.461566    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:28:27.630817 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:27.630783    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:28:28.329623 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:28.329588    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-54b68bc5b9-z4thq" event={"ID":"733f521d-33df-4e2f-8684-048d4cbaa0b1","Type":"ContainerStarted","Data":"9d67c2097922b33f7e6fafeb6d1438f7766075862fe8b5751168fe1482e7768f"}
Sep 10 15:28:28.329842 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:28.329642    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-54b68bc5b9-z4thq"
Sep 10 15:28:28.331595 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:28.331567    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-54b68bc5b9-z4thq"
Sep 10 15:28:28.331891 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:28.331870    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-5vhfm" event={"ID":"66247a82-0118-4ced-87d2-5f65c49557dc","Type":"ContainerStarted","Data":"7a455bdbdb67be560d98c52a4ded1f03b0545e944ec4ff8bc2f9b04b4fb33228"}
Sep 10 15:28:28.351220 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:28.351164    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-54b68bc5b9-z4thq" podStartSLOduration=1.14288401 podStartE2EDuration="5.351148664s" totalImagesPullingTime="4.208264654s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 15:28:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 15:28:28.350679521 +0000 UTC m=+93.918911253" watchObservedRunningTime="2026-09-10 15:28:28.351148664 +0000 UTC m=+93.919380436"
Sep 10 15:28:28.369199 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:28.369148    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-5vhfm" podStartSLOduration=33.489883601 podStartE2EDuration="37.369133914s" totalImagesPullingTime="3.879250313s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 15:27:51 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 15:28:28.36850937 +0000 UTC m=+93.936741112" watchObservedRunningTime="2026-09-10 15:28:28.369133914 +0000 UTC m=+93.937365643"
Sep 10 15:28:30.313196 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:30.313163    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:28:32.554626 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:32.554590    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:28:32.666936 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:32.666902    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:28:32.689949 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:32.689911    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5ef1f4eb-6292-4a77-8515-48e1671d8d13-registry-tls\") pod \"image-registry-659bc4b685-mpbnt\" (UID: \"5ef1f4eb-6292-4a77-8515-48e1671d8d13\") " pod="openshift-image-registry/image-registry-659bc4b685-mpbnt"
Sep 10 15:28:32.692291 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:32.692270    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5ef1f4eb-6292-4a77-8515-48e1671d8d13-registry-tls\") pod \"image-registry-659bc4b685-mpbnt\" (UID: \"5ef1f4eb-6292-4a77-8515-48e1671d8d13\") " pod="openshift-image-registry/image-registry-659bc4b685-mpbnt"
Sep 10 15:28:32.705092 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:32.705066    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-f2kws\""
Sep 10 15:28:32.711707 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:32.711685    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-659bc4b685-mpbnt"
Sep 10 15:28:32.791071 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:32.791037    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9818ae98-700a-4e5c-a4a5-24c0fc624542-metrics-tls\") pod \"dns-default-4k8rd\" (UID: \"9818ae98-700a-4e5c-a4a5-24c0fc624542\") " pod="openshift-dns/dns-default-4k8rd"
Sep 10 15:28:32.793560 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:32.793512    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9818ae98-700a-4e5c-a4a5-24c0fc624542-metrics-tls\") pod \"dns-default-4k8rd\" (UID: \"9818ae98-700a-4e5c-a4a5-24c0fc624542\") " pod="openshift-dns/dns-default-4k8rd"
Sep 10 15:28:32.817936 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:32.817859    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:28:32.872854 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:32.872820    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-74cb57df7-pncp7"]
Sep 10 15:28:32.876924 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:32.876900    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-tz22l"]
Sep 10 15:28:32.877075 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:32.877055    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-74cb57df7-pncp7"
Sep 10 15:28:32.879207 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:32.879185    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\""
Sep 10 15:28:32.879459 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:32.879443    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-k8mdm\""
Sep 10 15:28:32.879590 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:32.879555    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\""
Sep 10 15:28:32.879917 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:32.879901    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Sep 10 15:28:32.880177 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:32.880159    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-tz22l"
Sep 10 15:28:32.880366 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:32.880350    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Sep 10 15:28:32.880640 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:32.880622    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Sep 10 15:28:32.880809 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:32.880789    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\""
Sep 10 15:28:32.882560 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:32.882538    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Sep 10 15:28:32.882664 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:32.882537    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Sep 10 15:28:32.882728 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:32.882674    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Sep 10 15:28:32.882728 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:32.882544    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-nl9w4\""
Sep 10 15:28:32.889134 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:32.889026    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-74cb57df7-pncp7"]
Sep 10 15:28:32.898157 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:32.898138    2567 scope.go:117] "RemoveContainer" containerID="967ccc3b2063b8bf7949bddfce0f7e483fd3bb5dce81442e082c3f83d7e8fa9e"
Sep 10 15:28:32.898349 ip-10-0-128-241 kubenswrapper[2567]: E0910 15:28:32.898328    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-85b454874b-t96pm_openshift-console-operator(0b9d2cda-73e3-4e7c-9e17-1a3d903d95cf)\"" pod="openshift-console-operator/console-operator-85b454874b-t96pm" podUID="0b9d2cda-73e3-4e7c-9e17-1a3d903d95cf"
Sep 10 15:28:32.907616 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:32.907468    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:28:32.992878 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:32.992844    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-659bc4b685-mpbnt"]
Sep 10 15:28:32.993066 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:32.992913    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/64105509-967c-440a-948f-8ff2b09eec87-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-74cb57df7-pncp7\" (UID: \"64105509-967c-440a-948f-8ff2b09eec87\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-pncp7"
Sep 10 15:28:32.993066 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:32.992954    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6cabb203-0adb-478d-92b6-9dd54c84e8e2-sys\") pod \"node-exporter-tz22l\" (UID: \"6cabb203-0adb-478d-92b6-9dd54c84e8e2\") " pod="openshift-monitoring/node-exporter-tz22l"
Sep 10 15:28:32.993066 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:32.992973    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xfw6b\" (UniqueName: \"kubernetes.io/projected/64105509-967c-440a-948f-8ff2b09eec87-kube-api-access-xfw6b\") pod \"kube-state-metrics-74cb57df7-pncp7\" (UID: \"64105509-967c-440a-948f-8ff2b09eec87\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-pncp7"
Sep 10 15:28:32.993066 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:32.993004    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/6cabb203-0adb-478d-92b6-9dd54c84e8e2-node-exporter-accelerators-collector-config\") pod \"node-exporter-tz22l\" (UID: \"6cabb203-0adb-478d-92b6-9dd54c84e8e2\") " pod="openshift-monitoring/node-exporter-tz22l"
Sep 10 15:28:32.993311 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:32.993031    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/64105509-967c-440a-948f-8ff2b09eec87-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-74cb57df7-pncp7\" (UID: \"64105509-967c-440a-948f-8ff2b09eec87\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-pncp7"
Sep 10 15:28:32.993311 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:32.993103    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/6cabb203-0adb-478d-92b6-9dd54c84e8e2-node-exporter-wtmp\") pod \"node-exporter-tz22l\" (UID: \"6cabb203-0adb-478d-92b6-9dd54c84e8e2\") " pod="openshift-monitoring/node-exporter-tz22l"
Sep 10 15:28:32.993311 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:32.993208    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fd9w4\" (UniqueName: \"kubernetes.io/projected/6cabb203-0adb-478d-92b6-9dd54c84e8e2-kube-api-access-fd9w4\") pod \"node-exporter-tz22l\" (UID: \"6cabb203-0adb-478d-92b6-9dd54c84e8e2\") " pod="openshift-monitoring/node-exporter-tz22l"
Sep 10 15:28:32.993311 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:32.993277    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/64105509-967c-440a-948f-8ff2b09eec87-kube-state-metrics-tls\") pod \"kube-state-metrics-74cb57df7-pncp7\" (UID: \"64105509-967c-440a-948f-8ff2b09eec87\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-pncp7"
Sep 10 15:28:32.993592 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:32.993381    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/64105509-967c-440a-948f-8ff2b09eec87-volume-directive-shadow\") pod \"kube-state-metrics-74cb57df7-pncp7\" (UID: \"64105509-967c-440a-948f-8ff2b09eec87\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-pncp7"
Sep 10 15:28:32.993592 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:32.993437    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/64105509-967c-440a-948f-8ff2b09eec87-metrics-client-ca\") pod \"kube-state-metrics-74cb57df7-pncp7\" (UID: \"64105509-967c-440a-948f-8ff2b09eec87\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-pncp7"
Sep 10 15:28:32.993592 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:32.993519    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/6cabb203-0adb-478d-92b6-9dd54c84e8e2-node-exporter-textfile\") pod \"node-exporter-tz22l\" (UID: \"6cabb203-0adb-478d-92b6-9dd54c84e8e2\") " pod="openshift-monitoring/node-exporter-tz22l"
Sep 10 15:28:32.993592 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:32.993563    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6cabb203-0adb-478d-92b6-9dd54c84e8e2-metrics-client-ca\") pod \"node-exporter-tz22l\" (UID: \"6cabb203-0adb-478d-92b6-9dd54c84e8e2\") " pod="openshift-monitoring/node-exporter-tz22l"
Sep 10 15:28:32.993797 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:32.993593    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/6cabb203-0adb-478d-92b6-9dd54c84e8e2-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-tz22l\" (UID: \"6cabb203-0adb-478d-92b6-9dd54c84e8e2\") " pod="openshift-monitoring/node-exporter-tz22l"
Sep 10 15:28:32.993797 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:32.993619    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/6cabb203-0adb-478d-92b6-9dd54c84e8e2-node-exporter-tls\") pod \"node-exporter-tz22l\" (UID: \"6cabb203-0adb-478d-92b6-9dd54c84e8e2\") " pod="openshift-monitoring/node-exporter-tz22l"
Sep 10 15:28:32.993797 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:32.993650    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/6cabb203-0adb-478d-92b6-9dd54c84e8e2-root\") pod \"node-exporter-tz22l\" (UID: \"6cabb203-0adb-478d-92b6-9dd54c84e8e2\") " pod="openshift-monitoring/node-exporter-tz22l"
Sep 10 15:28:32.996078 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:28:32.996047    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5ef1f4eb_6292_4a77_8515_48e1671d8d13.slice/crio-1e21a5e5d1ff7b122944dc4b49c9e07848ccca9a798f25b4bb55fd30d0018f62 WatchSource:0}: Error finding container 1e21a5e5d1ff7b122944dc4b49c9e07848ccca9a798f25b4bb55fd30d0018f62: Status 404 returned error can't find the container with id 1e21a5e5d1ff7b122944dc4b49c9e07848ccca9a798f25b4bb55fd30d0018f62
Sep 10 15:28:33.021137 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:33.021113    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-w5mt4\""
Sep 10 15:28:33.028905 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:33.028886    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-4k8rd"
Sep 10 15:28:33.096518 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:33.096481    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/64105509-967c-440a-948f-8ff2b09eec87-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-74cb57df7-pncp7\" (UID: \"64105509-967c-440a-948f-8ff2b09eec87\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-pncp7"
Sep 10 15:28:33.096655 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:33.096528    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6cabb203-0adb-478d-92b6-9dd54c84e8e2-sys\") pod \"node-exporter-tz22l\" (UID: \"6cabb203-0adb-478d-92b6-9dd54c84e8e2\") " pod="openshift-monitoring/node-exporter-tz22l"
Sep 10 15:28:33.096655 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:33.096553    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xfw6b\" (UniqueName: \"kubernetes.io/projected/64105509-967c-440a-948f-8ff2b09eec87-kube-api-access-xfw6b\") pod \"kube-state-metrics-74cb57df7-pncp7\" (UID: \"64105509-967c-440a-948f-8ff2b09eec87\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-pncp7"
Sep 10 15:28:33.096655 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:33.096592    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/6cabb203-0adb-478d-92b6-9dd54c84e8e2-node-exporter-accelerators-collector-config\") pod \"node-exporter-tz22l\" (UID: \"6cabb203-0adb-478d-92b6-9dd54c84e8e2\") " pod="openshift-monitoring/node-exporter-tz22l"
Sep 10 15:28:33.096915 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:33.096887    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6cabb203-0adb-478d-92b6-9dd54c84e8e2-sys\") pod \"node-exporter-tz22l\" (UID: \"6cabb203-0adb-478d-92b6-9dd54c84e8e2\") " pod="openshift-monitoring/node-exporter-tz22l"
Sep 10 15:28:33.097221 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:33.097156    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/6cabb203-0adb-478d-92b6-9dd54c84e8e2-node-exporter-accelerators-collector-config\") pod \"node-exporter-tz22l\" (UID: \"6cabb203-0adb-478d-92b6-9dd54c84e8e2\") " pod="openshift-monitoring/node-exporter-tz22l"
Sep 10 15:28:33.099039 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:33.097371    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/64105509-967c-440a-948f-8ff2b09eec87-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-74cb57df7-pncp7\" (UID: \"64105509-967c-440a-948f-8ff2b09eec87\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-pncp7"
Sep 10 15:28:33.099039 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:33.097420    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/6cabb203-0adb-478d-92b6-9dd54c84e8e2-node-exporter-wtmp\") pod \"node-exporter-tz22l\" (UID: \"6cabb203-0adb-478d-92b6-9dd54c84e8e2\") " pod="openshift-monitoring/node-exporter-tz22l"
Sep 10 15:28:33.099039 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:33.097467    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fd9w4\" (UniqueName: \"kubernetes.io/projected/6cabb203-0adb-478d-92b6-9dd54c84e8e2-kube-api-access-fd9w4\") pod \"node-exporter-tz22l\" (UID: \"6cabb203-0adb-478d-92b6-9dd54c84e8e2\") " pod="openshift-monitoring/node-exporter-tz22l"
Sep 10 15:28:33.099039 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:33.097491    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/64105509-967c-440a-948f-8ff2b09eec87-kube-state-metrics-tls\") pod \"kube-state-metrics-74cb57df7-pncp7\" (UID: \"64105509-967c-440a-948f-8ff2b09eec87\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-pncp7"
Sep 10 15:28:33.099039 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:33.097532    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/64105509-967c-440a-948f-8ff2b09eec87-volume-directive-shadow\") pod \"kube-state-metrics-74cb57df7-pncp7\" (UID: \"64105509-967c-440a-948f-8ff2b09eec87\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-pncp7"
Sep 10 15:28:33.099039 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:33.097567    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/64105509-967c-440a-948f-8ff2b09eec87-metrics-client-ca\") pod \"kube-state-metrics-74cb57df7-pncp7\" (UID: \"64105509-967c-440a-948f-8ff2b09eec87\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-pncp7"
Sep 10 15:28:33.099039 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:33.097622    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/6cabb203-0adb-478d-92b6-9dd54c84e8e2-node-exporter-textfile\") pod \"node-exporter-tz22l\" (UID: \"6cabb203-0adb-478d-92b6-9dd54c84e8e2\") " pod="openshift-monitoring/node-exporter-tz22l"
Sep 10 15:28:33.099039 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:33.097651    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6cabb203-0adb-478d-92b6-9dd54c84e8e2-metrics-client-ca\") pod \"node-exporter-tz22l\" (UID: \"6cabb203-0adb-478d-92b6-9dd54c84e8e2\") " pod="openshift-monitoring/node-exporter-tz22l"
Sep 10 15:28:33.099039 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:33.097679    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/6cabb203-0adb-478d-92b6-9dd54c84e8e2-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-tz22l\" (UID: \"6cabb203-0adb-478d-92b6-9dd54c84e8e2\") " pod="openshift-monitoring/node-exporter-tz22l"
Sep 10 15:28:33.099039 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:33.097707    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/6cabb203-0adb-478d-92b6-9dd54c84e8e2-node-exporter-tls\") pod \"node-exporter-tz22l\" (UID: \"6cabb203-0adb-478d-92b6-9dd54c84e8e2\") " pod="openshift-monitoring/node-exporter-tz22l"
Sep 10 15:28:33.099039 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:33.097735    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/6cabb203-0adb-478d-92b6-9dd54c84e8e2-root\") pod \"node-exporter-tz22l\" (UID: \"6cabb203-0adb-478d-92b6-9dd54c84e8e2\") " pod="openshift-monitoring/node-exporter-tz22l"
Sep 10 15:28:33.099039 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:33.097834    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/6cabb203-0adb-478d-92b6-9dd54c84e8e2-root\") pod \"node-exporter-tz22l\" (UID: \"6cabb203-0adb-478d-92b6-9dd54c84e8e2\") " pod="openshift-monitoring/node-exporter-tz22l"
Sep 10 15:28:33.099039 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:33.097963    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/6cabb203-0adb-478d-92b6-9dd54c84e8e2-node-exporter-wtmp\") pod \"node-exporter-tz22l\" (UID: \"6cabb203-0adb-478d-92b6-9dd54c84e8e2\") " pod="openshift-monitoring/node-exporter-tz22l"
Sep 10 15:28:33.099039 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:33.098046    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/64105509-967c-440a-948f-8ff2b09eec87-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-74cb57df7-pncp7\" (UID: \"64105509-967c-440a-948f-8ff2b09eec87\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-pncp7"
Sep 10 15:28:33.099039 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:33.098093    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/6cabb203-0adb-478d-92b6-9dd54c84e8e2-node-exporter-textfile\") pod \"node-exporter-tz22l\" (UID: \"6cabb203-0adb-478d-92b6-9dd54c84e8e2\") " pod="openshift-monitoring/node-exporter-tz22l"
Sep 10 15:28:33.099039 ip-10-0-128-241 kubenswrapper[2567]: E0910 15:28:33.098146    2567 secret.go:189] Couldn't get secret openshift-monitoring/kube-state-metrics-tls: secret "kube-state-metrics-tls" not found
Sep 10 15:28:33.099867 ip-10-0-128-241 kubenswrapper[2567]: E0910 15:28:33.098203    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/64105509-967c-440a-948f-8ff2b09eec87-kube-state-metrics-tls podName:64105509-967c-440a-948f-8ff2b09eec87 nodeName:}" failed. No retries permitted until 2026-09-10 15:28:33.598186192 +0000 UTC m=+99.166417915 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-state-metrics-tls" (UniqueName: "kubernetes.io/secret/64105509-967c-440a-948f-8ff2b09eec87-kube-state-metrics-tls") pod "kube-state-metrics-74cb57df7-pncp7" (UID: "64105509-967c-440a-948f-8ff2b09eec87") : secret "kube-state-metrics-tls" not found
Sep 10 15:28:33.099867 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:33.098567    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/64105509-967c-440a-948f-8ff2b09eec87-metrics-client-ca\") pod \"kube-state-metrics-74cb57df7-pncp7\" (UID: \"64105509-967c-440a-948f-8ff2b09eec87\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-pncp7"
Sep 10 15:28:33.099867 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:33.098838    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6cabb203-0adb-478d-92b6-9dd54c84e8e2-metrics-client-ca\") pod \"node-exporter-tz22l\" (UID: \"6cabb203-0adb-478d-92b6-9dd54c84e8e2\") " pod="openshift-monitoring/node-exporter-tz22l"
Sep 10 15:28:33.099867 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:33.098841    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/64105509-967c-440a-948f-8ff2b09eec87-volume-directive-shadow\") pod \"kube-state-metrics-74cb57df7-pncp7\" (UID: \"64105509-967c-440a-948f-8ff2b09eec87\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-pncp7"
Sep 10 15:28:33.099867 ip-10-0-128-241 kubenswrapper[2567]: E0910 15:28:33.098915    2567 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found
Sep 10 15:28:33.099867 ip-10-0-128-241 kubenswrapper[2567]: E0910 15:28:33.098969    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6cabb203-0adb-478d-92b6-9dd54c84e8e2-node-exporter-tls podName:6cabb203-0adb-478d-92b6-9dd54c84e8e2 nodeName:}" failed. No retries permitted until 2026-09-10 15:28:33.598954822 +0000 UTC m=+99.167186538 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/6cabb203-0adb-478d-92b6-9dd54c84e8e2-node-exporter-tls") pod "node-exporter-tz22l" (UID: "6cabb203-0adb-478d-92b6-9dd54c84e8e2") : secret "node-exporter-tls" not found
Sep 10 15:28:33.101631 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:33.101370    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/64105509-967c-440a-948f-8ff2b09eec87-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-74cb57df7-pncp7\" (UID: \"64105509-967c-440a-948f-8ff2b09eec87\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-pncp7"
Sep 10 15:28:33.102490 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:33.102469    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/6cabb203-0adb-478d-92b6-9dd54c84e8e2-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-tz22l\" (UID: \"6cabb203-0adb-478d-92b6-9dd54c84e8e2\") " pod="openshift-monitoring/node-exporter-tz22l"
Sep 10 15:28:33.114633 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:33.114564    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fd9w4\" (UniqueName: \"kubernetes.io/projected/6cabb203-0adb-478d-92b6-9dd54c84e8e2-kube-api-access-fd9w4\") pod \"node-exporter-tz22l\" (UID: \"6cabb203-0adb-478d-92b6-9dd54c84e8e2\") " pod="openshift-monitoring/node-exporter-tz22l"
Sep 10 15:28:33.115962 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:33.115904    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xfw6b\" (UniqueName: \"kubernetes.io/projected/64105509-967c-440a-948f-8ff2b09eec87-kube-api-access-xfw6b\") pod \"kube-state-metrics-74cb57df7-pncp7\" (UID: \"64105509-967c-440a-948f-8ff2b09eec87\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-pncp7"
Sep 10 15:28:33.171063 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:33.171028    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:28:33.275432 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:33.275400    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:28:33.314273 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:33.314214    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-4k8rd"]
Sep 10 15:28:33.317442 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:28:33.317412    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9818ae98_700a_4e5c_a4a5_24c0fc624542.slice/crio-d970ff0f2b8214520961c7af2815729a0eeeec8272f0d7bdc245408db9b13828 WatchSource:0}: Error finding container d970ff0f2b8214520961c7af2815729a0eeeec8272f0d7bdc245408db9b13828: Status 404 returned error can't find the container with id d970ff0f2b8214520961c7af2815729a0eeeec8272f0d7bdc245408db9b13828
Sep 10 15:28:33.348342 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:33.348264    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-4k8rd" event={"ID":"9818ae98-700a-4e5c-a4a5-24c0fc624542","Type":"ContainerStarted","Data":"d970ff0f2b8214520961c7af2815729a0eeeec8272f0d7bdc245408db9b13828"}
Sep 10 15:28:33.349987 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:33.349959    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-659bc4b685-mpbnt" event={"ID":"5ef1f4eb-6292-4a77-8515-48e1671d8d13","Type":"ContainerStarted","Data":"5ba2a924cc88b4bdc5c0bb676a0b1a7361ed80ecbc84c67def66226e9f5f6c7c"}
Sep 10 15:28:33.350108 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:33.350006    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-659bc4b685-mpbnt" event={"ID":"5ef1f4eb-6292-4a77-8515-48e1671d8d13","Type":"ContainerStarted","Data":"1e21a5e5d1ff7b122944dc4b49c9e07848ccca9a798f25b4bb55fd30d0018f62"}
Sep 10 15:28:33.350108 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:33.350051    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-659bc4b685-mpbnt"
Sep 10 15:28:33.372537 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:33.372490    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-659bc4b685-mpbnt" podStartSLOduration=97.372474563 podStartE2EDuration="1m37.372474563s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 15:26:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 15:28:33.371289227 +0000 UTC m=+98.939520951" watchObservedRunningTime="2026-09-10 15:28:33.372474563 +0000 UTC m=+98.940706293"
Sep 10 15:28:33.435609 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:33.435570    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:28:33.601912 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:33.601825    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/64105509-967c-440a-948f-8ff2b09eec87-kube-state-metrics-tls\") pod \"kube-state-metrics-74cb57df7-pncp7\" (UID: \"64105509-967c-440a-948f-8ff2b09eec87\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-pncp7"
Sep 10 15:28:33.601912 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:33.601881    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/6cabb203-0adb-478d-92b6-9dd54c84e8e2-node-exporter-tls\") pod \"node-exporter-tz22l\" (UID: \"6cabb203-0adb-478d-92b6-9dd54c84e8e2\") " pod="openshift-monitoring/node-exporter-tz22l"
Sep 10 15:28:33.604127 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:33.604100    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/6cabb203-0adb-478d-92b6-9dd54c84e8e2-node-exporter-tls\") pod \"node-exporter-tz22l\" (UID: \"6cabb203-0adb-478d-92b6-9dd54c84e8e2\") " pod="openshift-monitoring/node-exporter-tz22l"
Sep 10 15:28:33.604267 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:33.604247    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/64105509-967c-440a-948f-8ff2b09eec87-kube-state-metrics-tls\") pod \"kube-state-metrics-74cb57df7-pncp7\" (UID: \"64105509-967c-440a-948f-8ff2b09eec87\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-pncp7"
Sep 10 15:28:33.789385 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:33.789355    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-74cb57df7-pncp7"
Sep 10 15:28:33.789773 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:33.789690    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:28:33.796513 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:33.796491    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-tz22l"
Sep 10 15:28:33.815597 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:28:33.815567    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6cabb203_0adb_478d_92b6_9dd54c84e8e2.slice/crio-476e7b130953796d3f9348302277dc12a23a13a7699ed2ade6f4a847a59c6589 WatchSource:0}: Error finding container 476e7b130953796d3f9348302277dc12a23a13a7699ed2ade6f4a847a59c6589: Status 404 returned error can't find the container with id 476e7b130953796d3f9348302277dc12a23a13a7699ed2ade6f4a847a59c6589
Sep 10 15:28:33.893146 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:33.892987    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:28:34.072175 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:34.072143    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-74cb57df7-pncp7"]
Sep 10 15:28:34.075886 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:28:34.075855    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod64105509_967c_440a_948f_8ff2b09eec87.slice/crio-ca6e99725c380b5950f570693447955a707eb8fafec1d7373455d8ce96b223ee WatchSource:0}: Error finding container ca6e99725c380b5950f570693447955a707eb8fafec1d7373455d8ce96b223ee: Status 404 returned error can't find the container with id ca6e99725c380b5950f570693447955a707eb8fafec1d7373455d8ce96b223ee
Sep 10 15:28:34.275352 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:34.275309    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:28:34.354514 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:34.354479    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-74cb57df7-pncp7" event={"ID":"64105509-967c-440a-948f-8ff2b09eec87","Type":"ContainerStarted","Data":"ca6e99725c380b5950f570693447955a707eb8fafec1d7373455d8ce96b223ee"}
Sep 10 15:28:34.355687 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:34.355663    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-tz22l" event={"ID":"6cabb203-0adb-478d-92b6-9dd54c84e8e2","Type":"ContainerStarted","Data":"476e7b130953796d3f9348302277dc12a23a13a7699ed2ade6f4a847a59c6589"}
Sep 10 15:28:35.361395 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:35.361367    2567 generic.go:356] "Generic (PLEG): container finished" podID="6cabb203-0adb-478d-92b6-9dd54c84e8e2" containerID="52b3bbc1eb05b95c7dd440ee5c5c1bcde42bc297f0820db5c373da130cb8f9dc" exitCode=0
Sep 10 15:28:35.361776 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:35.361464    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-tz22l" event={"ID":"6cabb203-0adb-478d-92b6-9dd54c84e8e2","Type":"ContainerDied","Data":"52b3bbc1eb05b95c7dd440ee5c5c1bcde42bc297f0820db5c373da130cb8f9dc"}
Sep 10 15:28:35.363365 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:35.363332    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-4k8rd" event={"ID":"9818ae98-700a-4e5c-a4a5-24c0fc624542","Type":"ContainerStarted","Data":"89bd95e6abc7b84be4f5cca078b5c98a9df6ddaadee20c36ee7b404fddb2dafa"}
Sep 10 15:28:35.816414 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:35.816375    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:28:35.917591 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:35.916593    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:28:36.137563 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:36.137520    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:28:36.349507 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:36.349425    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:28:36.368976 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:36.368944    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-74cb57df7-pncp7" event={"ID":"64105509-967c-440a-948f-8ff2b09eec87","Type":"ContainerStarted","Data":"bbd5ec46c5aed1fd7dd5e3a5a3bdb1e2f0a77a17d88df67e0f84b3624a08309d"}
Sep 10 15:28:36.369406 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:36.369015    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-74cb57df7-pncp7" event={"ID":"64105509-967c-440a-948f-8ff2b09eec87","Type":"ContainerStarted","Data":"cc3a4a0bab1147a539361576d06b0a8fcd86149ae33c9645b1a1ef325989418a"}
Sep 10 15:28:36.369406 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:36.369054    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-74cb57df7-pncp7" event={"ID":"64105509-967c-440a-948f-8ff2b09eec87","Type":"ContainerStarted","Data":"b2d6a3953ea9506fd0f9a774341f1485bc309c4106ab92737b87240d1b24ddfd"}
Sep 10 15:28:36.371120 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:36.371099    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-tz22l" event={"ID":"6cabb203-0adb-478d-92b6-9dd54c84e8e2","Type":"ContainerStarted","Data":"3a2b07fd8c3a1ee2704b3504fe9754f7b9f241a906db94941d3e39226c78484a"}
Sep 10 15:28:36.371223 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:36.371138    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-tz22l" event={"ID":"6cabb203-0adb-478d-92b6-9dd54c84e8e2","Type":"ContainerStarted","Data":"b0c76b5177ea29f7c01e4f84ab635437a47341d9ac1aa324bdef1c6f324939fb"}
Sep 10 15:28:36.372836 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:36.372816    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-4k8rd" event={"ID":"9818ae98-700a-4e5c-a4a5-24c0fc624542","Type":"ContainerStarted","Data":"c3573f1333ca349580ff7256b6a6078ca39baa5975a0d49a87da8fdf9eb74db2"}
Sep 10 15:28:36.372961 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:36.372943    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-4k8rd"
Sep 10 15:28:36.396135 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:36.396067    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-74cb57df7-pncp7" podStartSLOduration=2.681902156 podStartE2EDuration="4.396053253s" totalImagesPullingTime="1.714151097s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 15:28:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 15:28:36.394494818 +0000 UTC m=+101.962726568" watchObservedRunningTime="2026-09-10 15:28:36.396053253 +0000 UTC m=+101.964284983"
Sep 10 15:28:36.419338 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:36.419277    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-tz22l" podStartSLOduration=2.982867882 podStartE2EDuration="4.419255625s" totalImagesPullingTime="1.349917604s" totalInitContainerRuntime="86.470139ms" isStatefulPod=true podCreationTimestamp="2026-09-10 15:28:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 15:28:36.418819429 +0000 UTC m=+101.987051164" watchObservedRunningTime="2026-09-10 15:28:36.419255625 +0000 UTC m=+101.987487348"
Sep 10 15:28:36.440132 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:36.440082    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-4k8rd" podStartSLOduration=66.589983011 podStartE2EDuration="1m8.440066545s" totalImagesPullingTime="1.850083534s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 15:27:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 15:28:36.439413741 +0000 UTC m=+102.007645472" watchObservedRunningTime="2026-09-10 15:28:36.440066545 +0000 UTC m=+102.008298275"
Sep 10 15:28:37.609727 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:37.609688    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:28:38.053343 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:38.053309    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:28:39.147537 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:39.147507    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 10 15:28:39.151787 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:39.151757    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:28:39.179608 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:39.179574    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 10 15:28:39.179736 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:39.179709    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 15:28:39.182485 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:39.182458    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Sep 10 15:28:39.182619 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:39.182487    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Sep 10 15:28:39.182619 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:39.182505    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Sep 10 15:28:39.182710 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:39.182628    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Sep 10 15:28:39.182750 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:39.182727    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Sep 10 15:28:39.182802 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:39.182785    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Sep 10 15:28:39.183156 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:39.183139    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Sep 10 15:28:39.183259 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:39.183153    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Sep 10 15:28:39.183259 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:39.183138    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-96s8s24pf0b0d\""
Sep 10 15:28:39.183482 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:39.183457    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Sep 10 15:28:39.183482 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:39.183470    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Sep 10 15:28:39.183603 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:39.183500    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Sep 10 15:28:39.183603 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:39.183459    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Sep 10 15:28:39.183603 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:39.183570    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-j4pck\""
Sep 10 15:28:39.183813 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:39.183798    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Sep 10 15:28:39.186849 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:39.186822    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Sep 10 15:28:39.191329 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:39.191309    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Sep 10 15:28:39.259026 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:39.258997    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-2f7vt"
Sep 10 15:28:39.350368 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:39.350337    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/bb2321e3-340d-4603-a676-e4eb34ee306c-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"bb2321e3-340d-4603-a676-e4eb34ee306c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 15:28:39.350368 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:39.350370    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/bb2321e3-340d-4603-a676-e4eb34ee306c-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"bb2321e3-340d-4603-a676-e4eb34ee306c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 15:28:39.350565 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:39.350387    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/bb2321e3-340d-4603-a676-e4eb34ee306c-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"bb2321e3-340d-4603-a676-e4eb34ee306c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 15:28:39.350565 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:39.350408    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/bb2321e3-340d-4603-a676-e4eb34ee306c-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"bb2321e3-340d-4603-a676-e4eb34ee306c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 15:28:39.350565 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:39.350467    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/bb2321e3-340d-4603-a676-e4eb34ee306c-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"bb2321e3-340d-4603-a676-e4eb34ee306c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 15:28:39.350565 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:39.350511    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/bb2321e3-340d-4603-a676-e4eb34ee306c-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"bb2321e3-340d-4603-a676-e4eb34ee306c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 15:28:39.350565 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:39.350531    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/bb2321e3-340d-4603-a676-e4eb34ee306c-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"bb2321e3-340d-4603-a676-e4eb34ee306c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 15:28:39.350565 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:39.350555    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/bb2321e3-340d-4603-a676-e4eb34ee306c-config\") pod \"prometheus-k8s-0\" (UID: \"bb2321e3-340d-4603-a676-e4eb34ee306c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 15:28:39.350760 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:39.350573    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bb2321e3-340d-4603-a676-e4eb34ee306c-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"bb2321e3-340d-4603-a676-e4eb34ee306c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 15:28:39.350760 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:39.350633    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bb2321e3-340d-4603-a676-e4eb34ee306c-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"bb2321e3-340d-4603-a676-e4eb34ee306c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 15:28:39.350760 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:39.350683    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/bb2321e3-340d-4603-a676-e4eb34ee306c-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"bb2321e3-340d-4603-a676-e4eb34ee306c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 15:28:39.350760 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:39.350703    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/bb2321e3-340d-4603-a676-e4eb34ee306c-config-out\") pod \"prometheus-k8s-0\" (UID: \"bb2321e3-340d-4603-a676-e4eb34ee306c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 15:28:39.350760 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:39.350722    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/bb2321e3-340d-4603-a676-e4eb34ee306c-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"bb2321e3-340d-4603-a676-e4eb34ee306c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 15:28:39.350902 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:39.350769    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/bb2321e3-340d-4603-a676-e4eb34ee306c-web-config\") pod \"prometheus-k8s-0\" (UID: \"bb2321e3-340d-4603-a676-e4eb34ee306c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 15:28:39.350902 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:39.350798    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/bb2321e3-340d-4603-a676-e4eb34ee306c-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"bb2321e3-340d-4603-a676-e4eb34ee306c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 15:28:39.350902 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:39.350819    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/bb2321e3-340d-4603-a676-e4eb34ee306c-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"bb2321e3-340d-4603-a676-e4eb34ee306c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 15:28:39.350902 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:39.350841    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/bb2321e3-340d-4603-a676-e4eb34ee306c-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"bb2321e3-340d-4603-a676-e4eb34ee306c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 15:28:39.350902 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:39.350855    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7gdl8\" (UniqueName: \"kubernetes.io/projected/bb2321e3-340d-4603-a676-e4eb34ee306c-kube-api-access-7gdl8\") pod \"prometheus-k8s-0\" (UID: \"bb2321e3-340d-4603-a676-e4eb34ee306c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 15:28:39.350902 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:39.350886    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bb2321e3-340d-4603-a676-e4eb34ee306c-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"bb2321e3-340d-4603-a676-e4eb34ee306c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 15:28:39.351071 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:39.350906    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/bb2321e3-340d-4603-a676-e4eb34ee306c-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"bb2321e3-340d-4603-a676-e4eb34ee306c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 15:28:39.451713 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:39.451679    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bb2321e3-340d-4603-a676-e4eb34ee306c-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"bb2321e3-340d-4603-a676-e4eb34ee306c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 15:28:39.451713 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:39.451718    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/bb2321e3-340d-4603-a676-e4eb34ee306c-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"bb2321e3-340d-4603-a676-e4eb34ee306c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 15:28:39.451949 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:39.451737    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/bb2321e3-340d-4603-a676-e4eb34ee306c-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"bb2321e3-340d-4603-a676-e4eb34ee306c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 15:28:39.451949 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:39.451754    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/bb2321e3-340d-4603-a676-e4eb34ee306c-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"bb2321e3-340d-4603-a676-e4eb34ee306c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 15:28:39.451949 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:39.451769    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/bb2321e3-340d-4603-a676-e4eb34ee306c-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"bb2321e3-340d-4603-a676-e4eb34ee306c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 15:28:39.452086 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:39.451981    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/bb2321e3-340d-4603-a676-e4eb34ee306c-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"bb2321e3-340d-4603-a676-e4eb34ee306c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 15:28:39.452086 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:39.452025    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/bb2321e3-340d-4603-a676-e4eb34ee306c-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"bb2321e3-340d-4603-a676-e4eb34ee306c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 15:28:39.452086 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:39.452058    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/bb2321e3-340d-4603-a676-e4eb34ee306c-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"bb2321e3-340d-4603-a676-e4eb34ee306c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 15:28:39.452086 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:39.452080    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/bb2321e3-340d-4603-a676-e4eb34ee306c-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"bb2321e3-340d-4603-a676-e4eb34ee306c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 15:28:39.452308 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:39.452117    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/bb2321e3-340d-4603-a676-e4eb34ee306c-config\") pod \"prometheus-k8s-0\" (UID: \"bb2321e3-340d-4603-a676-e4eb34ee306c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 15:28:39.452308 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:39.452143    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bb2321e3-340d-4603-a676-e4eb34ee306c-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"bb2321e3-340d-4603-a676-e4eb34ee306c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 15:28:39.452674 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:39.452642    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/bb2321e3-340d-4603-a676-e4eb34ee306c-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"bb2321e3-340d-4603-a676-e4eb34ee306c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 15:28:39.452779 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:39.452707    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bb2321e3-340d-4603-a676-e4eb34ee306c-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"bb2321e3-340d-4603-a676-e4eb34ee306c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 15:28:39.452915 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:39.452889    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bb2321e3-340d-4603-a676-e4eb34ee306c-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"bb2321e3-340d-4603-a676-e4eb34ee306c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 15:28:39.453124 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:39.453099    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/bb2321e3-340d-4603-a676-e4eb34ee306c-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"bb2321e3-340d-4603-a676-e4eb34ee306c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 15:28:39.453319 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:39.453297    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bb2321e3-340d-4603-a676-e4eb34ee306c-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"bb2321e3-340d-4603-a676-e4eb34ee306c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 15:28:39.453439 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:39.453423    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/bb2321e3-340d-4603-a676-e4eb34ee306c-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"bb2321e3-340d-4603-a676-e4eb34ee306c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 15:28:39.453553 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:39.453540    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/bb2321e3-340d-4603-a676-e4eb34ee306c-config-out\") pod \"prometheus-k8s-0\" (UID: \"bb2321e3-340d-4603-a676-e4eb34ee306c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 15:28:39.453647 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:39.453636    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/bb2321e3-340d-4603-a676-e4eb34ee306c-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"bb2321e3-340d-4603-a676-e4eb34ee306c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 15:28:39.453757 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:39.453746    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/bb2321e3-340d-4603-a676-e4eb34ee306c-web-config\") pod \"prometheus-k8s-0\" (UID: \"bb2321e3-340d-4603-a676-e4eb34ee306c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 15:28:39.453852 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:39.453840    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/bb2321e3-340d-4603-a676-e4eb34ee306c-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"bb2321e3-340d-4603-a676-e4eb34ee306c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 15:28:39.453934 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:39.453923    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/bb2321e3-340d-4603-a676-e4eb34ee306c-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"bb2321e3-340d-4603-a676-e4eb34ee306c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 15:28:39.454032 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:39.454020    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/bb2321e3-340d-4603-a676-e4eb34ee306c-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"bb2321e3-340d-4603-a676-e4eb34ee306c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 15:28:39.454120 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:39.454109    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7gdl8\" (UniqueName: \"kubernetes.io/projected/bb2321e3-340d-4603-a676-e4eb34ee306c-kube-api-access-7gdl8\") pod \"prometheus-k8s-0\" (UID: \"bb2321e3-340d-4603-a676-e4eb34ee306c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 15:28:39.455113 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:39.454994    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/bb2321e3-340d-4603-a676-e4eb34ee306c-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"bb2321e3-340d-4603-a676-e4eb34ee306c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 15:28:39.455526 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:39.455502    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/bb2321e3-340d-4603-a676-e4eb34ee306c-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"bb2321e3-340d-4603-a676-e4eb34ee306c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 15:28:39.455614 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:39.455568    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/bb2321e3-340d-4603-a676-e4eb34ee306c-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"bb2321e3-340d-4603-a676-e4eb34ee306c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 15:28:39.455614 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:39.455572    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/bb2321e3-340d-4603-a676-e4eb34ee306c-config\") pod \"prometheus-k8s-0\" (UID: \"bb2321e3-340d-4603-a676-e4eb34ee306c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 15:28:39.455722 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:39.455649    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/bb2321e3-340d-4603-a676-e4eb34ee306c-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"bb2321e3-340d-4603-a676-e4eb34ee306c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 15:28:39.456216 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:39.455926    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/bb2321e3-340d-4603-a676-e4eb34ee306c-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"bb2321e3-340d-4603-a676-e4eb34ee306c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 15:28:39.456216 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:39.456102    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/bb2321e3-340d-4603-a676-e4eb34ee306c-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"bb2321e3-340d-4603-a676-e4eb34ee306c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 15:28:39.456216 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:39.456199    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/bb2321e3-340d-4603-a676-e4eb34ee306c-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"bb2321e3-340d-4603-a676-e4eb34ee306c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 15:28:39.456433 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:39.456255    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bb2321e3-340d-4603-a676-e4eb34ee306c-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"bb2321e3-340d-4603-a676-e4eb34ee306c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 15:28:39.456853 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:39.456825    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/bb2321e3-340d-4603-a676-e4eb34ee306c-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"bb2321e3-340d-4603-a676-e4eb34ee306c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 15:28:39.457651 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:39.457626    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/bb2321e3-340d-4603-a676-e4eb34ee306c-web-config\") pod \"prometheus-k8s-0\" (UID: \"bb2321e3-340d-4603-a676-e4eb34ee306c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 15:28:39.457727 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:39.457709    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/bb2321e3-340d-4603-a676-e4eb34ee306c-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"bb2321e3-340d-4603-a676-e4eb34ee306c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 15:28:39.458342 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:39.458318    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/bb2321e3-340d-4603-a676-e4eb34ee306c-config-out\") pod \"prometheus-k8s-0\" (UID: \"bb2321e3-340d-4603-a676-e4eb34ee306c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 15:28:39.458517 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:39.458498    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/bb2321e3-340d-4603-a676-e4eb34ee306c-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"bb2321e3-340d-4603-a676-e4eb34ee306c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 15:28:39.458654 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:39.458639    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/bb2321e3-340d-4603-a676-e4eb34ee306c-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"bb2321e3-340d-4603-a676-e4eb34ee306c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 15:28:39.464032 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:39.464011    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7gdl8\" (UniqueName: \"kubernetes.io/projected/bb2321e3-340d-4603-a676-e4eb34ee306c-kube-api-access-7gdl8\") pod \"prometheus-k8s-0\" (UID: \"bb2321e3-340d-4603-a676-e4eb34ee306c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 15:28:39.490804 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:39.490776    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 15:28:39.812729 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:39.812707    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 10 15:28:39.814198 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:28:39.814173    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbb2321e3_340d_4603_a676_e4eb34ee306c.slice/crio-8e51f92ff93f8a12e603012a92008cdbfaf5741fb7ba70542ab7581dd2f54e51 WatchSource:0}: Error finding container 8e51f92ff93f8a12e603012a92008cdbfaf5741fb7ba70542ab7581dd2f54e51: Status 404 returned error can't find the container with id 8e51f92ff93f8a12e603012a92008cdbfaf5741fb7ba70542ab7581dd2f54e51
Sep 10 15:28:40.389451 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:40.389405    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"bb2321e3-340d-4603-a676-e4eb34ee306c","Type":"ContainerStarted","Data":"8e51f92ff93f8a12e603012a92008cdbfaf5741fb7ba70542ab7581dd2f54e51"}
Sep 10 15:28:41.247720 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:41.247671    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:28:41.309070 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:41.309013    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:28:41.394848 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:41.394816    2567 generic.go:356] "Generic (PLEG): container finished" podID="bb2321e3-340d-4603-a676-e4eb34ee306c" containerID="039420531bd90c94b98f3cef76bc0a91542bf3f9e313c51cd719d8c555857d37" exitCode=0
Sep 10 15:28:41.395175 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:41.394898    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"bb2321e3-340d-4603-a676-e4eb34ee306c","Type":"ContainerDied","Data":"039420531bd90c94b98f3cef76bc0a91542bf3f9e313c51cd719d8c555857d37"}
Sep 10 15:28:44.568558 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:44.568527    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:28:44.670495 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:44.670466    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:28:44.900058 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:44.899976    2567 scope.go:117] "RemoveContainer" containerID="967ccc3b2063b8bf7949bddfce0f7e483fd3bb5dce81442e082c3f83d7e8fa9e"
Sep 10 15:28:45.414326 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:45.414290    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"bb2321e3-340d-4603-a676-e4eb34ee306c","Type":"ContainerStarted","Data":"61e755068cae75291d3f1c4ce644ef211119a92d1e0cd3fa98720f5a6cf5fee5"}
Sep 10 15:28:45.414518 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:45.414379    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"bb2321e3-340d-4603-a676-e4eb34ee306c","Type":"ContainerStarted","Data":"7c87859613b274e7559c034c76df9519c7090dba77e703819f1929ac9a89231e"}
Sep 10 15:28:45.416424 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:45.416403    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-t96pm_0b9d2cda-73e3-4e7c-9e17-1a3d903d95cf/console-operator/2.log"
Sep 10 15:28:45.416552 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:45.416480    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-85b454874b-t96pm" event={"ID":"0b9d2cda-73e3-4e7c-9e17-1a3d903d95cf","Type":"ContainerStarted","Data":"fcada77b9cd211112fddeada5a2f9d1d8e7a717544b21b8fe4812529ed86ede9"}
Sep 10 15:28:45.416795 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:45.416764    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-85b454874b-t96pm"
Sep 10 15:28:45.434674 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:45.434623    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-85b454874b-t96pm" podStartSLOduration=57.505025275 podStartE2EDuration="59.434607154s" totalImagesPullingTime="1.929581879s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 15:27:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 15:28:45.433714245 +0000 UTC m=+111.001945970" watchObservedRunningTime="2026-09-10 15:28:45.434607154 +0000 UTC m=+111.002838883"
Sep 10 15:28:45.589475 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:45.589382    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-85b454874b-t96pm"
Sep 10 15:28:45.787896 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:45.787853    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:28:45.884189 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:45.884158    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:28:45.946184 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:45.946151    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:28:46.078414 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:46.078304    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:28:46.163334 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:46.163303    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-659bc4b685-mpbnt"]
Sep 10 15:28:46.263282 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:46.263216    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:28:46.359547 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:46.359418    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:28:46.380200 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:46.380170    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-4k8rd"
Sep 10 15:28:46.501671 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:46.501644    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:28:46.602142 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:46.602114    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:28:46.851175 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:46.851135    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:28:47.428753 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:47.428717    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"bb2321e3-340d-4603-a676-e4eb34ee306c","Type":"ContainerStarted","Data":"0796e0a4a424cb8d1173e2a207dffebb9cc5d640a439927a8f7a61a6c5c83b0a"}
Sep 10 15:28:47.428949 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:47.428799    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"bb2321e3-340d-4603-a676-e4eb34ee306c","Type":"ContainerStarted","Data":"45eea007f84d477da6cdd4eb8cc359e35bf4476de3409a085ba2c13614ba6f6f"}
Sep 10 15:28:47.428949 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:47.428838    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"bb2321e3-340d-4603-a676-e4eb34ee306c","Type":"ContainerStarted","Data":"004e953343506b711888b77f94553f7234ef2e8b29079372de1dd1ec445fce6a"}
Sep 10 15:28:47.428949 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:47.428871    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"bb2321e3-340d-4603-a676-e4eb34ee306c","Type":"ContainerStarted","Data":"6c18303d4bc54d83c4ddd087a1b21ccf24238d82d0fe9c0dae02ccc3a0d5d594"}
Sep 10 15:28:47.457900 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:47.457844    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.467935779 podStartE2EDuration="8.457825108s" totalImagesPullingTime="5.971881379s" totalInitContainerRuntime="18.00795ms" isStatefulPod=false podCreationTimestamp="2026-09-10 15:28:39 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 15:28:47.456026802 +0000 UTC m=+113.024258532" watchObservedRunningTime="2026-09-10 15:28:47.457825108 +0000 UTC m=+113.026056834"
Sep 10 15:28:47.676199 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:47.676163    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:28:47.869157 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:47.869064    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:28:48.132289 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:48.132192    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:28:48.685004 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:48.684969    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:28:48.786101 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:48.786063    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:28:49.491680 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:49.491643    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 15:28:53.688909 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:53.688870    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:28:53.751754 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:53.751716    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:28:54.147606 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:54.147515    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:28:54.211197 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:54.211162    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:28:55.441197 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:55.441165    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:28:55.507064 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:55.507024    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:28:56.168781 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:56.168753    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-659bc4b685-mpbnt"
Sep 10 15:28:56.794652 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:56.794619    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:28:56.864789 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:56.864757    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:28:58.662015 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:58.661979    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:28:58.763839 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:58.763800    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:28:58.864914 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:28:58.864873    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:29:00.886855 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:29:00.886823    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:29:00.952182 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:29:00.952143    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:29:01.014873 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:29:01.014836    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:29:04.535024 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:29:04.534987    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:29:04.595873 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:29:04.595831    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:29:04.667685 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:29:04.667646    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f91d1ea8-2355-4b51-b736-448a89aba131-metrics-certs\") pod \"network-metrics-daemon-zxpwj\" (UID: \"f91d1ea8-2355-4b51-b736-448a89aba131\") " pod="openshift-multus/network-metrics-daemon-zxpwj"
Sep 10 15:29:04.669950 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:29:04.669926    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f91d1ea8-2355-4b51-b736-448a89aba131-metrics-certs\") pod \"network-metrics-daemon-zxpwj\" (UID: \"f91d1ea8-2355-4b51-b736-448a89aba131\") " pod="openshift-multus/network-metrics-daemon-zxpwj"
Sep 10 15:29:04.706478 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:29:04.706442    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:29:04.724257 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:29:04.724217    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-h8jcj\""
Sep 10 15:29:04.732217 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:29:04.732198    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zxpwj"
Sep 10 15:29:04.767858 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:29:04.767822    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:29:05.005070 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:29:05.004934    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-zxpwj"]
Sep 10 15:29:05.007572 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:29:05.007540    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf91d1ea8_2355_4b51_b736_448a89aba131.slice/crio-8e437cd2efb280d6533eb66f901a9416e015ae1f7b49550b144cb392ecba8209 WatchSource:0}: Error finding container 8e437cd2efb280d6533eb66f901a9416e015ae1f7b49550b144cb392ecba8209: Status 404 returned error can't find the container with id 8e437cd2efb280d6533eb66f901a9416e015ae1f7b49550b144cb392ecba8209
Sep 10 15:29:05.493692 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:29:05.493661    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-zxpwj" event={"ID":"f91d1ea8-2355-4b51-b736-448a89aba131","Type":"ContainerStarted","Data":"8e437cd2efb280d6533eb66f901a9416e015ae1f7b49550b144cb392ecba8209"}
Sep 10 15:29:06.122323 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:29:06.122294    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:29:06.224802 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:29:06.224756    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:29:06.498661 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:29:06.498627    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-zxpwj" event={"ID":"f91d1ea8-2355-4b51-b736-448a89aba131","Type":"ContainerStarted","Data":"ecd608e75bc2e384a2f7406ef1a2bc0a500e783720bf368fc6f96bebbc27e738"}
Sep 10 15:29:06.498822 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:29:06.498679    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-zxpwj" event={"ID":"f91d1ea8-2355-4b51-b736-448a89aba131","Type":"ContainerStarted","Data":"5c5ace99e07774a787c03629ebaeb8b7abfb81badd4534ce7a9a8b20df3ac1e4"}
Sep 10 15:29:06.516731 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:29:06.516684    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-zxpwj" podStartSLOduration=130.476028045 podStartE2EDuration="2m11.51667089s" totalImagesPullingTime="1.040642845s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 15:26:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 15:29:06.514990522 +0000 UTC m=+132.083222253" watchObservedRunningTime="2026-09-10 15:29:06.51667089 +0000 UTC m=+132.084902620"
Sep 10 15:29:06.614974 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:29:06.614942    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:29:06.679193 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:29:06.679159    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:29:09.087313 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:29:09.087278    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:29:09.897333 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:29:09.897294    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:29:09.998052 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:29:09.998011    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:29:11.185550 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:29:11.185509    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-659bc4b685-mpbnt" podUID="5ef1f4eb-6292-4a77-8515-48e1671d8d13" containerName="registry" containerID="cri-o://5ba2a924cc88b4bdc5c0bb676a0b1a7361ed80ecbc84c67def66226e9f5f6c7c" gracePeriod=30
Sep 10 15:29:11.422813 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:29:11.422788    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-659bc4b685-mpbnt"
Sep 10 15:29:11.516918 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:29:11.516843    2567 generic.go:356] "Generic (PLEG): container finished" podID="5ef1f4eb-6292-4a77-8515-48e1671d8d13" containerID="5ba2a924cc88b4bdc5c0bb676a0b1a7361ed80ecbc84c67def66226e9f5f6c7c" exitCode=0
Sep 10 15:29:11.516918 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:29:11.516892    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-659bc4b685-mpbnt"
Sep 10 15:29:11.517103 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:29:11.516884    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-659bc4b685-mpbnt" event={"ID":"5ef1f4eb-6292-4a77-8515-48e1671d8d13","Type":"ContainerDied","Data":"5ba2a924cc88b4bdc5c0bb676a0b1a7361ed80ecbc84c67def66226e9f5f6c7c"}
Sep 10 15:29:11.517103 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:29:11.517016    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-659bc4b685-mpbnt" event={"ID":"5ef1f4eb-6292-4a77-8515-48e1671d8d13","Type":"ContainerDied","Data":"1e21a5e5d1ff7b122944dc4b49c9e07848ccca9a798f25b4bb55fd30d0018f62"}
Sep 10 15:29:11.517103 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:29:11.517044    2567 scope.go:117] "RemoveContainer" containerID="5ba2a924cc88b4bdc5c0bb676a0b1a7361ed80ecbc84c67def66226e9f5f6c7c"
Sep 10 15:29:11.524667 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:29:11.524643    2567 scope.go:117] "RemoveContainer" containerID="5ba2a924cc88b4bdc5c0bb676a0b1a7361ed80ecbc84c67def66226e9f5f6c7c"
Sep 10 15:29:11.524774 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:29:11.524698    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/5ef1f4eb-6292-4a77-8515-48e1671d8d13-trusted-ca\") pod \"5ef1f4eb-6292-4a77-8515-48e1671d8d13\" (UID: \"5ef1f4eb-6292-4a77-8515-48e1671d8d13\") "
Sep 10 15:29:11.524774 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:29:11.524753    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/5ef1f4eb-6292-4a77-8515-48e1671d8d13-bound-sa-token\") pod \"5ef1f4eb-6292-4a77-8515-48e1671d8d13\" (UID: \"5ef1f4eb-6292-4a77-8515-48e1671d8d13\") "
Sep 10 15:29:11.524851 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:29:11.524780    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/5ef1f4eb-6292-4a77-8515-48e1671d8d13-ca-trust-extracted\") pod \"5ef1f4eb-6292-4a77-8515-48e1671d8d13\" (UID: \"5ef1f4eb-6292-4a77-8515-48e1671d8d13\") "
Sep 10 15:29:11.524851 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:29:11.524802    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/5ef1f4eb-6292-4a77-8515-48e1671d8d13-registry-certificates\") pod \"5ef1f4eb-6292-4a77-8515-48e1671d8d13\" (UID: \"5ef1f4eb-6292-4a77-8515-48e1671d8d13\") "
Sep 10 15:29:11.524851 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:29:11.524838    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rzs4q\" (UniqueName: \"kubernetes.io/projected/5ef1f4eb-6292-4a77-8515-48e1671d8d13-kube-api-access-rzs4q\") pod \"5ef1f4eb-6292-4a77-8515-48e1671d8d13\" (UID: \"5ef1f4eb-6292-4a77-8515-48e1671d8d13\") "
Sep 10 15:29:11.525154 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:29:11.524868    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/5ef1f4eb-6292-4a77-8515-48e1671d8d13-image-registry-private-configuration\") pod \"5ef1f4eb-6292-4a77-8515-48e1671d8d13\" (UID: \"5ef1f4eb-6292-4a77-8515-48e1671d8d13\") "
Sep 10 15:29:11.525154 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:29:11.524897    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/5ef1f4eb-6292-4a77-8515-48e1671d8d13-installation-pull-secrets\") pod \"5ef1f4eb-6292-4a77-8515-48e1671d8d13\" (UID: \"5ef1f4eb-6292-4a77-8515-48e1671d8d13\") "
Sep 10 15:29:11.525154 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:29:11.524931    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5ef1f4eb-6292-4a77-8515-48e1671d8d13-registry-tls\") pod \"5ef1f4eb-6292-4a77-8515-48e1671d8d13\" (UID: \"5ef1f4eb-6292-4a77-8515-48e1671d8d13\") "
Sep 10 15:29:11.525154 ip-10-0-128-241 kubenswrapper[2567]: E0910 15:29:11.524955    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5ba2a924cc88b4bdc5c0bb676a0b1a7361ed80ecbc84c67def66226e9f5f6c7c\": container with ID starting with 5ba2a924cc88b4bdc5c0bb676a0b1a7361ed80ecbc84c67def66226e9f5f6c7c not found: ID does not exist" containerID="5ba2a924cc88b4bdc5c0bb676a0b1a7361ed80ecbc84c67def66226e9f5f6c7c"
Sep 10 15:29:11.525154 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:29:11.524994    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5ba2a924cc88b4bdc5c0bb676a0b1a7361ed80ecbc84c67def66226e9f5f6c7c"} err="failed to get container status \"5ba2a924cc88b4bdc5c0bb676a0b1a7361ed80ecbc84c67def66226e9f5f6c7c\": rpc error: code = NotFound desc = could not find container \"5ba2a924cc88b4bdc5c0bb676a0b1a7361ed80ecbc84c67def66226e9f5f6c7c\": container with ID starting with 5ba2a924cc88b4bdc5c0bb676a0b1a7361ed80ecbc84c67def66226e9f5f6c7c not found: ID does not exist"
Sep 10 15:29:11.525426 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:29:11.525155    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5ef1f4eb-6292-4a77-8515-48e1671d8d13-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "5ef1f4eb-6292-4a77-8515-48e1671d8d13" (UID: "5ef1f4eb-6292-4a77-8515-48e1671d8d13"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 15:29:11.525426 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:29:11.525380    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5ef1f4eb-6292-4a77-8515-48e1671d8d13-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "5ef1f4eb-6292-4a77-8515-48e1671d8d13" (UID: "5ef1f4eb-6292-4a77-8515-48e1671d8d13"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 15:29:11.527513 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:29:11.527486    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5ef1f4eb-6292-4a77-8515-48e1671d8d13-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "5ef1f4eb-6292-4a77-8515-48e1671d8d13" (UID: "5ef1f4eb-6292-4a77-8515-48e1671d8d13"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 15:29:11.527686 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:29:11.527656    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5ef1f4eb-6292-4a77-8515-48e1671d8d13-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "5ef1f4eb-6292-4a77-8515-48e1671d8d13" (UID: "5ef1f4eb-6292-4a77-8515-48e1671d8d13"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 15:29:11.527786 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:29:11.527678    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5ef1f4eb-6292-4a77-8515-48e1671d8d13-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "5ef1f4eb-6292-4a77-8515-48e1671d8d13" (UID: "5ef1f4eb-6292-4a77-8515-48e1671d8d13"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 15:29:11.527786 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:29:11.527769    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5ef1f4eb-6292-4a77-8515-48e1671d8d13-kube-api-access-rzs4q" (OuterVolumeSpecName: "kube-api-access-rzs4q") pod "5ef1f4eb-6292-4a77-8515-48e1671d8d13" (UID: "5ef1f4eb-6292-4a77-8515-48e1671d8d13"). InnerVolumeSpecName "kube-api-access-rzs4q". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 15:29:11.527869 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:29:11.527786    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5ef1f4eb-6292-4a77-8515-48e1671d8d13-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "5ef1f4eb-6292-4a77-8515-48e1671d8d13" (UID: "5ef1f4eb-6292-4a77-8515-48e1671d8d13"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 15:29:11.533682 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:29:11.533662    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5ef1f4eb-6292-4a77-8515-48e1671d8d13-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "5ef1f4eb-6292-4a77-8515-48e1671d8d13" (UID: "5ef1f4eb-6292-4a77-8515-48e1671d8d13"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 15:29:11.626293 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:29:11.626258    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rzs4q\" (UniqueName: \"kubernetes.io/projected/5ef1f4eb-6292-4a77-8515-48e1671d8d13-kube-api-access-rzs4q\") on node \"ip-10-0-128-241.ec2.internal\" DevicePath \"\""
Sep 10 15:29:11.626293 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:29:11.626288    2567 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/5ef1f4eb-6292-4a77-8515-48e1671d8d13-image-registry-private-configuration\") on node \"ip-10-0-128-241.ec2.internal\" DevicePath \"\""
Sep 10 15:29:11.626293 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:29:11.626298    2567 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/5ef1f4eb-6292-4a77-8515-48e1671d8d13-installation-pull-secrets\") on node \"ip-10-0-128-241.ec2.internal\" DevicePath \"\""
Sep 10 15:29:11.626694 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:29:11.626309    2567 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5ef1f4eb-6292-4a77-8515-48e1671d8d13-registry-tls\") on node \"ip-10-0-128-241.ec2.internal\" DevicePath \"\""
Sep 10 15:29:11.626694 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:29:11.626318    2567 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/5ef1f4eb-6292-4a77-8515-48e1671d8d13-trusted-ca\") on node \"ip-10-0-128-241.ec2.internal\" DevicePath \"\""
Sep 10 15:29:11.626694 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:29:11.626326    2567 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/5ef1f4eb-6292-4a77-8515-48e1671d8d13-bound-sa-token\") on node \"ip-10-0-128-241.ec2.internal\" DevicePath \"\""
Sep 10 15:29:11.626694 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:29:11.626333    2567 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/5ef1f4eb-6292-4a77-8515-48e1671d8d13-ca-trust-extracted\") on node \"ip-10-0-128-241.ec2.internal\" DevicePath \"\""
Sep 10 15:29:11.626694 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:29:11.626341    2567 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/5ef1f4eb-6292-4a77-8515-48e1671d8d13-registry-certificates\") on node \"ip-10-0-128-241.ec2.internal\" DevicePath \"\""
Sep 10 15:29:11.846289 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:29:11.846255    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-659bc4b685-mpbnt"]
Sep 10 15:29:11.848885 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:29:11.848852    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-659bc4b685-mpbnt"]
Sep 10 15:29:12.903091 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:29:12.903054    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5ef1f4eb-6292-4a77-8515-48e1671d8d13" path="/var/lib/kubelet/pods/5ef1f4eb-6292-4a77-8515-48e1671d8d13/volumes"
Sep 10 15:29:13.396590 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:29:13.396564    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:29:13.497020 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:29:13.496985    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:29:16.582901 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:29:16.582848    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:29:16.681411 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:29:16.681378    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:29:16.782016 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:29:16.781977    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:29:16.935583 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:29:16.935546    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:29:17.047007 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:29:17.046961    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:29:18.150828 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:29:18.150798    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-74cb57df7-pncp7_64105509-967c-440a-948f-8ff2b09eec87/kube-state-metrics/0.log"
Sep 10 15:29:18.349306 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:29:18.349275    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-74cb57df7-pncp7_64105509-967c-440a-948f-8ff2b09eec87/kube-rbac-proxy-main/0.log"
Sep 10 15:29:18.549575 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:29:18.549545    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-74cb57df7-pncp7_64105509-967c-440a-948f-8ff2b09eec87/kube-rbac-proxy-self/0.log"
Sep 10 15:29:20.349734 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:29:20.349710    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-tz22l_6cabb203-0adb-478d-92b6-9dd54c84e8e2/init-textfile/0.log"
Sep 10 15:29:20.551506 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:29:20.551472    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-tz22l_6cabb203-0adb-478d-92b6-9dd54c84e8e2/node-exporter/0.log"
Sep 10 15:29:20.749933 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:29:20.749904    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-tz22l_6cabb203-0adb-478d-92b6-9dd54c84e8e2/kube-rbac-proxy/0.log"
Sep 10 15:29:21.123702 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:29:21.123620    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:29:21.223545 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:29:21.223510    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:29:21.549858 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:29:21.549833    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_bb2321e3-340d-4603-a676-e4eb34ee306c/init-config-reloader/0.log"
Sep 10 15:29:21.752815 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:29:21.752787    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_bb2321e3-340d-4603-a676-e4eb34ee306c/prometheus/0.log"
Sep 10 15:29:21.950290 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:29:21.950267    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_bb2321e3-340d-4603-a676-e4eb34ee306c/config-reloader/0.log"
Sep 10 15:29:22.151182 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:29:22.151152    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_bb2321e3-340d-4603-a676-e4eb34ee306c/thanos-sidecar/0.log"
Sep 10 15:29:22.349825 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:29:22.349746    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_bb2321e3-340d-4603-a676-e4eb34ee306c/kube-rbac-proxy-web/0.log"
Sep 10 15:29:22.550678 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:29:22.550649    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_bb2321e3-340d-4603-a676-e4eb34ee306c/kube-rbac-proxy/0.log"
Sep 10 15:29:22.752094 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:29:22.752062    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_bb2321e3-340d-4603-a676-e4eb34ee306c/kube-rbac-proxy-thanos/0.log"
Sep 10 15:29:23.540632 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:29:23.540598    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:29:23.641593 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:29:23.641543    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:29:23.700981 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:29:23.700951    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:29:24.949303 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:29:24.949268    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-t96pm_0b9d2cda-73e3-4e7c-9e17-1a3d903d95cf/console-operator/2.log"
Sep 10 15:29:25.152486 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:29:25.152457    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-t96pm_0b9d2cda-73e3-4e7c-9e17-1a3d903d95cf/console-operator/3.log"
Sep 10 15:29:31.883610 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:29:31.883569    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:29:36.125515 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:29:36.125474    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:29:36.186552 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:29:36.186519    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:29:39.491129 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:29:39.491085    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 15:29:39.510026 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:29:39.509999    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 15:29:39.637384 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:29:39.637357    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 15:29:52.138393 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:29:52.138305    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:29:52.553159 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:29:52.553121    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:29:57.544872 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:29:57.544844    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 10 15:29:57.545307 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:29:57.545175    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="bb2321e3-340d-4603-a676-e4eb34ee306c" containerName="prometheus" containerID="cri-o://7c87859613b274e7559c034c76df9519c7090dba77e703819f1929ac9a89231e" gracePeriod=600
Sep 10 15:29:57.545307 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:29:57.545180    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="bb2321e3-340d-4603-a676-e4eb34ee306c" containerName="kube-rbac-proxy" containerID="cri-o://45eea007f84d477da6cdd4eb8cc359e35bf4476de3409a085ba2c13614ba6f6f" gracePeriod=600
Sep 10 15:29:57.545307 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:29:57.545256    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="bb2321e3-340d-4603-a676-e4eb34ee306c" containerName="kube-rbac-proxy-web" containerID="cri-o://004e953343506b711888b77f94553f7234ef2e8b29079372de1dd1ec445fce6a" gracePeriod=600
Sep 10 15:29:57.545475 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:29:57.545243    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="bb2321e3-340d-4603-a676-e4eb34ee306c" containerName="config-reloader" containerID="cri-o://61e755068cae75291d3f1c4ce644ef211119a92d1e0cd3fa98720f5a6cf5fee5" gracePeriod=600
Sep 10 15:29:57.545475 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:29:57.545224    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="bb2321e3-340d-4603-a676-e4eb34ee306c" containerName="thanos-sidecar" containerID="cri-o://6c18303d4bc54d83c4ddd087a1b21ccf24238d82d0fe9c0dae02ccc3a0d5d594" gracePeriod=600
Sep 10 15:29:57.545475 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:29:57.545225    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="bb2321e3-340d-4603-a676-e4eb34ee306c" containerName="kube-rbac-proxy-thanos" containerID="cri-o://0796e0a4a424cb8d1173e2a207dffebb9cc5d640a439927a8f7a61a6c5c83b0a" gracePeriod=600
Sep 10 15:29:57.690059 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:29:57.690011    2567 generic.go:356] "Generic (PLEG): container finished" podID="bb2321e3-340d-4603-a676-e4eb34ee306c" containerID="0796e0a4a424cb8d1173e2a207dffebb9cc5d640a439927a8f7a61a6c5c83b0a" exitCode=0
Sep 10 15:29:57.690059 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:29:57.690050    2567 generic.go:356] "Generic (PLEG): container finished" podID="bb2321e3-340d-4603-a676-e4eb34ee306c" containerID="45eea007f84d477da6cdd4eb8cc359e35bf4476de3409a085ba2c13614ba6f6f" exitCode=0
Sep 10 15:29:57.690059 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:29:57.690063    2567 generic.go:356] "Generic (PLEG): container finished" podID="bb2321e3-340d-4603-a676-e4eb34ee306c" containerID="004e953343506b711888b77f94553f7234ef2e8b29079372de1dd1ec445fce6a" exitCode=0
Sep 10 15:29:57.690294 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:29:57.690076    2567 generic.go:356] "Generic (PLEG): container finished" podID="bb2321e3-340d-4603-a676-e4eb34ee306c" containerID="6c18303d4bc54d83c4ddd087a1b21ccf24238d82d0fe9c0dae02ccc3a0d5d594" exitCode=0
Sep 10 15:29:57.690294 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:29:57.690088    2567 generic.go:356] "Generic (PLEG): container finished" podID="bb2321e3-340d-4603-a676-e4eb34ee306c" containerID="61e755068cae75291d3f1c4ce644ef211119a92d1e0cd3fa98720f5a6cf5fee5" exitCode=0
Sep 10 15:29:57.690294 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:29:57.690107    2567 generic.go:356] "Generic (PLEG): container finished" podID="bb2321e3-340d-4603-a676-e4eb34ee306c" containerID="7c87859613b274e7559c034c76df9519c7090dba77e703819f1929ac9a89231e" exitCode=0
Sep 10 15:29:57.690294 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:29:57.690081    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"bb2321e3-340d-4603-a676-e4eb34ee306c","Type":"ContainerDied","Data":"0796e0a4a424cb8d1173e2a207dffebb9cc5d640a439927a8f7a61a6c5c83b0a"}
Sep 10 15:29:57.690294 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:29:57.690246    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"bb2321e3-340d-4603-a676-e4eb34ee306c","Type":"ContainerDied","Data":"45eea007f84d477da6cdd4eb8cc359e35bf4476de3409a085ba2c13614ba6f6f"}
Sep 10 15:29:57.690294 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:29:57.690277    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"bb2321e3-340d-4603-a676-e4eb34ee306c","Type":"ContainerDied","Data":"004e953343506b711888b77f94553f7234ef2e8b29079372de1dd1ec445fce6a"}
Sep 10 15:29:57.690465 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:29:57.690307    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"bb2321e3-340d-4603-a676-e4eb34ee306c","Type":"ContainerDied","Data":"6c18303d4bc54d83c4ddd087a1b21ccf24238d82d0fe9c0dae02ccc3a0d5d594"}
Sep 10 15:29:57.690465 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:29:57.690332    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"bb2321e3-340d-4603-a676-e4eb34ee306c","Type":"ContainerDied","Data":"61e755068cae75291d3f1c4ce644ef211119a92d1e0cd3fa98720f5a6cf5fee5"}
Sep 10 15:29:57.690465 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:29:57.690356    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"bb2321e3-340d-4603-a676-e4eb34ee306c","Type":"ContainerDied","Data":"7c87859613b274e7559c034c76df9519c7090dba77e703819f1929ac9a89231e"}
Sep 10 15:29:57.777274 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:29:57.777244    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 15:29:57.878287 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:29:57.878192    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/bb2321e3-340d-4603-a676-e4eb34ee306c-configmap-metrics-client-ca\") pod \"bb2321e3-340d-4603-a676-e4eb34ee306c\" (UID: \"bb2321e3-340d-4603-a676-e4eb34ee306c\") "
Sep 10 15:29:57.878287 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:29:57.878260    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7gdl8\" (UniqueName: \"kubernetes.io/projected/bb2321e3-340d-4603-a676-e4eb34ee306c-kube-api-access-7gdl8\") pod \"bb2321e3-340d-4603-a676-e4eb34ee306c\" (UID: \"bb2321e3-340d-4603-a676-e4eb34ee306c\") "
Sep 10 15:29:57.878287 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:29:57.878284    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/bb2321e3-340d-4603-a676-e4eb34ee306c-prometheus-k8s-db\") pod \"bb2321e3-340d-4603-a676-e4eb34ee306c\" (UID: \"bb2321e3-340d-4603-a676-e4eb34ee306c\") "
Sep 10 15:29:57.878547 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:29:57.878387    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/bb2321e3-340d-4603-a676-e4eb34ee306c-prometheus-k8s-rulefiles-2\") pod \"bb2321e3-340d-4603-a676-e4eb34ee306c\" (UID: \"bb2321e3-340d-4603-a676-e4eb34ee306c\") "
Sep 10 15:29:57.878547 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:29:57.878418    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/bb2321e3-340d-4603-a676-e4eb34ee306c-secret-prometheus-k8s-tls\") pod \"bb2321e3-340d-4603-a676-e4eb34ee306c\" (UID: \"bb2321e3-340d-4603-a676-e4eb34ee306c\") "
Sep 10 15:29:57.878547 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:29:57.878444    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/bb2321e3-340d-4603-a676-e4eb34ee306c-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"bb2321e3-340d-4603-a676-e4eb34ee306c\" (UID: \"bb2321e3-340d-4603-a676-e4eb34ee306c\") "
Sep 10 15:29:57.878547 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:29:57.878481    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/bb2321e3-340d-4603-a676-e4eb34ee306c-config\") pod \"bb2321e3-340d-4603-a676-e4eb34ee306c\" (UID: \"bb2321e3-340d-4603-a676-e4eb34ee306c\") "
Sep 10 15:29:57.878547 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:29:57.878512    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/bb2321e3-340d-4603-a676-e4eb34ee306c-secret-grpc-tls\") pod \"bb2321e3-340d-4603-a676-e4eb34ee306c\" (UID: \"bb2321e3-340d-4603-a676-e4eb34ee306c\") "
Sep 10 15:29:57.878547 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:29:57.878545    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/bb2321e3-340d-4603-a676-e4eb34ee306c-secret-kube-rbac-proxy\") pod \"bb2321e3-340d-4603-a676-e4eb34ee306c\" (UID: \"bb2321e3-340d-4603-a676-e4eb34ee306c\") "
Sep 10 15:29:57.878829 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:29:57.878569    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/bb2321e3-340d-4603-a676-e4eb34ee306c-thanos-prometheus-http-client-file\") pod \"bb2321e3-340d-4603-a676-e4eb34ee306c\" (UID: \"bb2321e3-340d-4603-a676-e4eb34ee306c\") "
Sep 10 15:29:57.878829 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:29:57.878598    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bb2321e3-340d-4603-a676-e4eb34ee306c-configmap-kubelet-serving-ca-bundle\") pod \"bb2321e3-340d-4603-a676-e4eb34ee306c\" (UID: \"bb2321e3-340d-4603-a676-e4eb34ee306c\") "
Sep 10 15:29:57.878829 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:29:57.878653    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bb2321e3-340d-4603-a676-e4eb34ee306c-configmap-serving-certs-ca-bundle\") pod \"bb2321e3-340d-4603-a676-e4eb34ee306c\" (UID: \"bb2321e3-340d-4603-a676-e4eb34ee306c\") "
Sep 10 15:29:57.878829 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:29:57.878686    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/bb2321e3-340d-4603-a676-e4eb34ee306c-prometheus-k8s-rulefiles-1\") pod \"bb2321e3-340d-4603-a676-e4eb34ee306c\" (UID: \"bb2321e3-340d-4603-a676-e4eb34ee306c\") "
Sep 10 15:29:57.878829 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:29:57.878707    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/bb2321e3-340d-4603-a676-e4eb34ee306c-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"bb2321e3-340d-4603-a676-e4eb34ee306c\" (UID: \"bb2321e3-340d-4603-a676-e4eb34ee306c\") "
Sep 10 15:29:57.878829 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:29:57.878736    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/bb2321e3-340d-4603-a676-e4eb34ee306c-secret-metrics-client-certs\") pod \"bb2321e3-340d-4603-a676-e4eb34ee306c\" (UID: \"bb2321e3-340d-4603-a676-e4eb34ee306c\") "
Sep 10 15:29:57.878829 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:29:57.878763    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/bb2321e3-340d-4603-a676-e4eb34ee306c-web-config\") pod \"bb2321e3-340d-4603-a676-e4eb34ee306c\" (UID: \"bb2321e3-340d-4603-a676-e4eb34ee306c\") "
Sep 10 15:29:57.878829 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:29:57.878796    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bb2321e3-340d-4603-a676-e4eb34ee306c-prometheus-trusted-ca-bundle\") pod \"bb2321e3-340d-4603-a676-e4eb34ee306c\" (UID: \"bb2321e3-340d-4603-a676-e4eb34ee306c\") "
Sep 10 15:29:57.878829 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:29:57.878830    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/bb2321e3-340d-4603-a676-e4eb34ee306c-config-out\") pod \"bb2321e3-340d-4603-a676-e4eb34ee306c\" (UID: \"bb2321e3-340d-4603-a676-e4eb34ee306c\") "
Sep 10 15:29:57.879302 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:29:57.878862    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/bb2321e3-340d-4603-a676-e4eb34ee306c-prometheus-k8s-rulefiles-0\") pod \"bb2321e3-340d-4603-a676-e4eb34ee306c\" (UID: \"bb2321e3-340d-4603-a676-e4eb34ee306c\") "
Sep 10 15:29:57.879302 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:29:57.878888    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/bb2321e3-340d-4603-a676-e4eb34ee306c-tls-assets\") pod \"bb2321e3-340d-4603-a676-e4eb34ee306c\" (UID: \"bb2321e3-340d-4603-a676-e4eb34ee306c\") "
Sep 10 15:29:57.881551 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:29:57.878731    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bb2321e3-340d-4603-a676-e4eb34ee306c-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "bb2321e3-340d-4603-a676-e4eb34ee306c" (UID: "bb2321e3-340d-4603-a676-e4eb34ee306c"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 15:29:57.881551 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:29:57.878813    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bb2321e3-340d-4603-a676-e4eb34ee306c-prometheus-k8s-rulefiles-2" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-2") pod "bb2321e3-340d-4603-a676-e4eb34ee306c" (UID: "bb2321e3-340d-4603-a676-e4eb34ee306c"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 15:29:57.881551 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:29:57.879055    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bb2321e3-340d-4603-a676-e4eb34ee306c-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "bb2321e3-340d-4603-a676-e4eb34ee306c" (UID: "bb2321e3-340d-4603-a676-e4eb34ee306c"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 15:29:57.881551 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:29:57.879437    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bb2321e3-340d-4603-a676-e4eb34ee306c-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "bb2321e3-340d-4603-a676-e4eb34ee306c" (UID: "bb2321e3-340d-4603-a676-e4eb34ee306c"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 15:29:57.881551 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:29:57.881375    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bb2321e3-340d-4603-a676-e4eb34ee306c-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "bb2321e3-340d-4603-a676-e4eb34ee306c" (UID: "bb2321e3-340d-4603-a676-e4eb34ee306c"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 15:29:57.881933 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:29:57.881408    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bb2321e3-340d-4603-a676-e4eb34ee306c-kube-api-access-7gdl8" (OuterVolumeSpecName: "kube-api-access-7gdl8") pod "bb2321e3-340d-4603-a676-e4eb34ee306c" (UID: "bb2321e3-340d-4603-a676-e4eb34ee306c"). InnerVolumeSpecName "kube-api-access-7gdl8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 15:29:57.881933 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:29:57.881420    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bb2321e3-340d-4603-a676-e4eb34ee306c-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "bb2321e3-340d-4603-a676-e4eb34ee306c" (UID: "bb2321e3-340d-4603-a676-e4eb34ee306c"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 15:29:57.881933 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:29:57.881464    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bb2321e3-340d-4603-a676-e4eb34ee306c-prometheus-k8s-rulefiles-1" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-1") pod "bb2321e3-340d-4603-a676-e4eb34ee306c" (UID: "bb2321e3-340d-4603-a676-e4eb34ee306c"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 15:29:57.881933 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:29:57.881658    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bb2321e3-340d-4603-a676-e4eb34ee306c-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "bb2321e3-340d-4603-a676-e4eb34ee306c" (UID: "bb2321e3-340d-4603-a676-e4eb34ee306c"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 15:29:57.882157 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:29:57.881926    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bb2321e3-340d-4603-a676-e4eb34ee306c-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "bb2321e3-340d-4603-a676-e4eb34ee306c" (UID: "bb2321e3-340d-4603-a676-e4eb34ee306c"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 15:29:57.882453 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:29:57.882289    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bb2321e3-340d-4603-a676-e4eb34ee306c-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "bb2321e3-340d-4603-a676-e4eb34ee306c" (UID: "bb2321e3-340d-4603-a676-e4eb34ee306c"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 15:29:57.882453 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:29:57.882325    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bb2321e3-340d-4603-a676-e4eb34ee306c-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "bb2321e3-340d-4603-a676-e4eb34ee306c" (UID: "bb2321e3-340d-4603-a676-e4eb34ee306c"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 15:29:57.882453 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:29:57.882382    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bb2321e3-340d-4603-a676-e4eb34ee306c-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "bb2321e3-340d-4603-a676-e4eb34ee306c" (UID: "bb2321e3-340d-4603-a676-e4eb34ee306c"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 15:29:57.882453 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:29:57.882422    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bb2321e3-340d-4603-a676-e4eb34ee306c-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "bb2321e3-340d-4603-a676-e4eb34ee306c" (UID: "bb2321e3-340d-4603-a676-e4eb34ee306c"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 15:29:57.882695 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:29:57.882668    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bb2321e3-340d-4603-a676-e4eb34ee306c-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "bb2321e3-340d-4603-a676-e4eb34ee306c" (UID: "bb2321e3-340d-4603-a676-e4eb34ee306c"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 15:29:57.882878 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:29:57.882854    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bb2321e3-340d-4603-a676-e4eb34ee306c-config" (OuterVolumeSpecName: "config") pod "bb2321e3-340d-4603-a676-e4eb34ee306c" (UID: "bb2321e3-340d-4603-a676-e4eb34ee306c"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 15:29:57.883305 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:29:57.883287    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bb2321e3-340d-4603-a676-e4eb34ee306c-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "bb2321e3-340d-4603-a676-e4eb34ee306c" (UID: "bb2321e3-340d-4603-a676-e4eb34ee306c"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 15:29:57.883446 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:29:57.883423    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bb2321e3-340d-4603-a676-e4eb34ee306c-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "bb2321e3-340d-4603-a676-e4eb34ee306c" (UID: "bb2321e3-340d-4603-a676-e4eb34ee306c"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 15:29:57.884597 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:29:57.884579    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bb2321e3-340d-4603-a676-e4eb34ee306c-config-out" (OuterVolumeSpecName: "config-out") pod "bb2321e3-340d-4603-a676-e4eb34ee306c" (UID: "bb2321e3-340d-4603-a676-e4eb34ee306c"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 15:29:57.892924 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:29:57.892906    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bb2321e3-340d-4603-a676-e4eb34ee306c-web-config" (OuterVolumeSpecName: "web-config") pod "bb2321e3-340d-4603-a676-e4eb34ee306c" (UID: "bb2321e3-340d-4603-a676-e4eb34ee306c"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 15:29:57.979961 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:29:57.979927    2567 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bb2321e3-340d-4603-a676-e4eb34ee306c-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-128-241.ec2.internal\" DevicePath \"\""
Sep 10 15:29:57.979961 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:29:57.979957    2567 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/bb2321e3-340d-4603-a676-e4eb34ee306c-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-128-241.ec2.internal\" DevicePath \"\""
Sep 10 15:29:57.979961 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:29:57.979968    2567 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/bb2321e3-340d-4603-a676-e4eb34ee306c-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-128-241.ec2.internal\" DevicePath \"\""
Sep 10 15:29:57.980164 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:29:57.979978    2567 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/bb2321e3-340d-4603-a676-e4eb34ee306c-secret-metrics-client-certs\") on node \"ip-10-0-128-241.ec2.internal\" DevicePath \"\""
Sep 10 15:29:57.980164 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:29:57.979988    2567 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/bb2321e3-340d-4603-a676-e4eb34ee306c-web-config\") on node \"ip-10-0-128-241.ec2.internal\" DevicePath \"\""
Sep 10 15:29:57.980164 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:29:57.979996    2567 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bb2321e3-340d-4603-a676-e4eb34ee306c-prometheus-trusted-ca-bundle\") on node \"ip-10-0-128-241.ec2.internal\" DevicePath \"\""
Sep 10 15:29:57.980164 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:29:57.980004    2567 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/bb2321e3-340d-4603-a676-e4eb34ee306c-config-out\") on node \"ip-10-0-128-241.ec2.internal\" DevicePath \"\""
Sep 10 15:29:57.980164 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:29:57.980012    2567 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/bb2321e3-340d-4603-a676-e4eb34ee306c-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-128-241.ec2.internal\" DevicePath \"\""
Sep 10 15:29:57.980164 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:29:57.980020    2567 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/bb2321e3-340d-4603-a676-e4eb34ee306c-tls-assets\") on node \"ip-10-0-128-241.ec2.internal\" DevicePath \"\""
Sep 10 15:29:57.980164 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:29:57.980029    2567 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/bb2321e3-340d-4603-a676-e4eb34ee306c-configmap-metrics-client-ca\") on node \"ip-10-0-128-241.ec2.internal\" DevicePath \"\""
Sep 10 15:29:57.980164 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:29:57.980039    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7gdl8\" (UniqueName: \"kubernetes.io/projected/bb2321e3-340d-4603-a676-e4eb34ee306c-kube-api-access-7gdl8\") on node \"ip-10-0-128-241.ec2.internal\" DevicePath \"\""
Sep 10 15:29:57.980164 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:29:57.980046    2567 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/bb2321e3-340d-4603-a676-e4eb34ee306c-prometheus-k8s-db\") on node \"ip-10-0-128-241.ec2.internal\" DevicePath \"\""
Sep 10 15:29:57.980164 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:29:57.980055    2567 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/bb2321e3-340d-4603-a676-e4eb34ee306c-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-128-241.ec2.internal\" DevicePath \"\""
Sep 10 15:29:57.980164 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:29:57.980063    2567 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/bb2321e3-340d-4603-a676-e4eb34ee306c-secret-prometheus-k8s-tls\") on node \"ip-10-0-128-241.ec2.internal\" DevicePath \"\""
Sep 10 15:29:57.980164 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:29:57.980071    2567 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/bb2321e3-340d-4603-a676-e4eb34ee306c-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-128-241.ec2.internal\" DevicePath \"\""
Sep 10 15:29:57.980164 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:29:57.980080    2567 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/bb2321e3-340d-4603-a676-e4eb34ee306c-config\") on node \"ip-10-0-128-241.ec2.internal\" DevicePath \"\""
Sep 10 15:29:57.980164 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:29:57.980088    2567 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/bb2321e3-340d-4603-a676-e4eb34ee306c-secret-grpc-tls\") on node \"ip-10-0-128-241.ec2.internal\" DevicePath \"\""
Sep 10 15:29:57.980164 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:29:57.980096    2567 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/bb2321e3-340d-4603-a676-e4eb34ee306c-secret-kube-rbac-proxy\") on node \"ip-10-0-128-241.ec2.internal\" DevicePath \"\""
Sep 10 15:29:57.980164 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:29:57.980104    2567 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/bb2321e3-340d-4603-a676-e4eb34ee306c-thanos-prometheus-http-client-file\") on node \"ip-10-0-128-241.ec2.internal\" DevicePath \"\""
Sep 10 15:29:57.980164 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:29:57.980112    2567 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bb2321e3-340d-4603-a676-e4eb34ee306c-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-128-241.ec2.internal\" DevicePath \"\""
Sep 10 15:29:58.696737 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:29:58.696706    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"bb2321e3-340d-4603-a676-e4eb34ee306c","Type":"ContainerDied","Data":"8e51f92ff93f8a12e603012a92008cdbfaf5741fb7ba70542ab7581dd2f54e51"}
Sep 10 15:29:58.697132 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:29:58.696771    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 15:29:58.697132 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:29:58.696782    2567 scope.go:117] "RemoveContainer" containerID="0796e0a4a424cb8d1173e2a207dffebb9cc5d640a439927a8f7a61a6c5c83b0a"
Sep 10 15:29:58.705363 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:29:58.705084    2567 scope.go:117] "RemoveContainer" containerID="45eea007f84d477da6cdd4eb8cc359e35bf4476de3409a085ba2c13614ba6f6f"
Sep 10 15:29:58.711746 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:29:58.711722    2567 scope.go:117] "RemoveContainer" containerID="004e953343506b711888b77f94553f7234ef2e8b29079372de1dd1ec445fce6a"
Sep 10 15:29:58.717980 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:29:58.717961    2567 scope.go:117] "RemoveContainer" containerID="6c18303d4bc54d83c4ddd087a1b21ccf24238d82d0fe9c0dae02ccc3a0d5d594"
Sep 10 15:29:58.722649 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:29:58.722630    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 10 15:29:58.724967 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:29:58.724948    2567 scope.go:117] "RemoveContainer" containerID="61e755068cae75291d3f1c4ce644ef211119a92d1e0cd3fa98720f5a6cf5fee5"
Sep 10 15:29:58.727639 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:29:58.727618    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 10 15:29:58.732035 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:29:58.732018    2567 scope.go:117] "RemoveContainer" containerID="7c87859613b274e7559c034c76df9519c7090dba77e703819f1929ac9a89231e"
Sep 10 15:29:58.738712 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:29:58.738695    2567 scope.go:117] "RemoveContainer" containerID="039420531bd90c94b98f3cef76bc0a91542bf3f9e313c51cd719d8c555857d37"
Sep 10 15:29:58.755268 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:29:58.752916    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 10 15:29:58.755268 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:29:58.753497    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bb2321e3-340d-4603-a676-e4eb34ee306c" containerName="init-config-reloader"
Sep 10 15:29:58.755268 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:29:58.753517    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="bb2321e3-340d-4603-a676-e4eb34ee306c" containerName="init-config-reloader"
Sep 10 15:29:58.755268 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:29:58.753535    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bb2321e3-340d-4603-a676-e4eb34ee306c" containerName="thanos-sidecar"
Sep 10 15:29:58.755268 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:29:58.753546    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="bb2321e3-340d-4603-a676-e4eb34ee306c" containerName="thanos-sidecar"
Sep 10 15:29:58.755268 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:29:58.753557    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bb2321e3-340d-4603-a676-e4eb34ee306c" containerName="kube-rbac-proxy"
Sep 10 15:29:58.755268 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:29:58.753566    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="bb2321e3-340d-4603-a676-e4eb34ee306c" containerName="kube-rbac-proxy"
Sep 10 15:29:58.755268 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:29:58.753578    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bb2321e3-340d-4603-a676-e4eb34ee306c" containerName="prometheus"
Sep 10 15:29:58.755268 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:29:58.753587    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="bb2321e3-340d-4603-a676-e4eb34ee306c" containerName="prometheus"
Sep 10 15:29:58.755268 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:29:58.753613    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5ef1f4eb-6292-4a77-8515-48e1671d8d13" containerName="registry"
Sep 10 15:29:58.755268 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:29:58.753621    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="5ef1f4eb-6292-4a77-8515-48e1671d8d13" containerName="registry"
Sep 10 15:29:58.755268 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:29:58.753640    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bb2321e3-340d-4603-a676-e4eb34ee306c" containerName="kube-rbac-proxy-web"
Sep 10 15:29:58.755268 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:29:58.753650    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="bb2321e3-340d-4603-a676-e4eb34ee306c" containerName="kube-rbac-proxy-web"
Sep 10 15:29:58.755268 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:29:58.753660    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bb2321e3-340d-4603-a676-e4eb34ee306c" containerName="kube-rbac-proxy-thanos"
Sep 10 15:29:58.755268 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:29:58.753668    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="bb2321e3-340d-4603-a676-e4eb34ee306c" containerName="kube-rbac-proxy-thanos"
Sep 10 15:29:58.755268 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:29:58.753694    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bb2321e3-340d-4603-a676-e4eb34ee306c" containerName="config-reloader"
Sep 10 15:29:58.755268 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:29:58.753703    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="bb2321e3-340d-4603-a676-e4eb34ee306c" containerName="config-reloader"
Sep 10 15:29:58.755268 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:29:58.753803    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="bb2321e3-340d-4603-a676-e4eb34ee306c" containerName="kube-rbac-proxy-web"
Sep 10 15:29:58.755268 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:29:58.753814    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="5ef1f4eb-6292-4a77-8515-48e1671d8d13" containerName="registry"
Sep 10 15:29:58.755268 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:29:58.753823    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="bb2321e3-340d-4603-a676-e4eb34ee306c" containerName="thanos-sidecar"
Sep 10 15:29:58.755268 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:29:58.753838    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="bb2321e3-340d-4603-a676-e4eb34ee306c" containerName="prometheus"
Sep 10 15:29:58.755268 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:29:58.753845    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="bb2321e3-340d-4603-a676-e4eb34ee306c" containerName="kube-rbac-proxy"
Sep 10 15:29:58.755268 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:29:58.753854    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="bb2321e3-340d-4603-a676-e4eb34ee306c" containerName="kube-rbac-proxy-thanos"
Sep 10 15:29:58.755268 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:29:58.753863    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="bb2321e3-340d-4603-a676-e4eb34ee306c" containerName="config-reloader"
Sep 10 15:29:58.760689 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:29:58.760670    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 15:29:58.763042 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:29:58.763021    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Sep 10 15:29:58.763139 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:29:58.763055    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Sep 10 15:29:58.763139 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:29:58.763055    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Sep 10 15:29:58.763320 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:29:58.763175    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Sep 10 15:29:58.763436 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:29:58.763422    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Sep 10 15:29:58.763507 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:29:58.763422    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Sep 10 15:29:58.763655 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:29:58.763638    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Sep 10 15:29:58.763754 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:29:58.763740    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Sep 10 15:29:58.763829 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:29:58.763756    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Sep 10 15:29:58.763884 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:29:58.763864    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-96s8s24pf0b0d\""
Sep 10 15:29:58.764025 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:29:58.764009    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Sep 10 15:29:58.764096 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:29:58.764032    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Sep 10 15:29:58.764202 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:29:58.764186    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-j4pck\""
Sep 10 15:29:58.764260 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:29:58.764205    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Sep 10 15:29:58.764497 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:29:58.764481    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Sep 10 15:29:58.766128 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:29:58.766111    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Sep 10 15:29:58.768573 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:29:58.768552    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Sep 10 15:29:58.770986 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:29:58.770964    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 10 15:29:58.886118 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:29:58.886026    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/65aca3d0-4b3b-4f36-b3b3-456394ada2a3-config-out\") pod \"prometheus-k8s-0\" (UID: \"65aca3d0-4b3b-4f36-b3b3-456394ada2a3\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 15:29:58.886118 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:29:58.886066    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/65aca3d0-4b3b-4f36-b3b3-456394ada2a3-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"65aca3d0-4b3b-4f36-b3b3-456394ada2a3\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 15:29:58.886118 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:29:58.886095    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/65aca3d0-4b3b-4f36-b3b3-456394ada2a3-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"65aca3d0-4b3b-4f36-b3b3-456394ada2a3\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 15:29:58.886118 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:29:58.886118    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/65aca3d0-4b3b-4f36-b3b3-456394ada2a3-web-config\") pod \"prometheus-k8s-0\" (UID: \"65aca3d0-4b3b-4f36-b3b3-456394ada2a3\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 15:29:58.886401 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:29:58.886138    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/65aca3d0-4b3b-4f36-b3b3-456394ada2a3-config\") pod \"prometheus-k8s-0\" (UID: \"65aca3d0-4b3b-4f36-b3b3-456394ada2a3\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 15:29:58.886401 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:29:58.886152    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/65aca3d0-4b3b-4f36-b3b3-456394ada2a3-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"65aca3d0-4b3b-4f36-b3b3-456394ada2a3\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 15:29:58.886401 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:29:58.886174    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/65aca3d0-4b3b-4f36-b3b3-456394ada2a3-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"65aca3d0-4b3b-4f36-b3b3-456394ada2a3\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 15:29:58.886401 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:29:58.886197    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/65aca3d0-4b3b-4f36-b3b3-456394ada2a3-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"65aca3d0-4b3b-4f36-b3b3-456394ada2a3\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 15:29:58.886401 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:29:58.886217    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/65aca3d0-4b3b-4f36-b3b3-456394ada2a3-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"65aca3d0-4b3b-4f36-b3b3-456394ada2a3\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 15:29:58.886401 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:29:58.886256    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/65aca3d0-4b3b-4f36-b3b3-456394ada2a3-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"65aca3d0-4b3b-4f36-b3b3-456394ada2a3\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 15:29:58.886401 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:29:58.886279    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/65aca3d0-4b3b-4f36-b3b3-456394ada2a3-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"65aca3d0-4b3b-4f36-b3b3-456394ada2a3\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 15:29:58.886401 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:29:58.886294    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/65aca3d0-4b3b-4f36-b3b3-456394ada2a3-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"65aca3d0-4b3b-4f36-b3b3-456394ada2a3\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 15:29:58.886401 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:29:58.886319    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/65aca3d0-4b3b-4f36-b3b3-456394ada2a3-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"65aca3d0-4b3b-4f36-b3b3-456394ada2a3\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 15:29:58.886401 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:29:58.886343    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/65aca3d0-4b3b-4f36-b3b3-456394ada2a3-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"65aca3d0-4b3b-4f36-b3b3-456394ada2a3\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 15:29:58.886401 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:29:58.886362    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/65aca3d0-4b3b-4f36-b3b3-456394ada2a3-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"65aca3d0-4b3b-4f36-b3b3-456394ada2a3\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 15:29:58.886401 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:29:58.886377    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/65aca3d0-4b3b-4f36-b3b3-456394ada2a3-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"65aca3d0-4b3b-4f36-b3b3-456394ada2a3\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 15:29:58.886401 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:29:58.886395    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/65aca3d0-4b3b-4f36-b3b3-456394ada2a3-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"65aca3d0-4b3b-4f36-b3b3-456394ada2a3\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 15:29:58.886825 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:29:58.886410    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/65aca3d0-4b3b-4f36-b3b3-456394ada2a3-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"65aca3d0-4b3b-4f36-b3b3-456394ada2a3\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 15:29:58.886825 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:29:58.886432    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/65aca3d0-4b3b-4f36-b3b3-456394ada2a3-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"65aca3d0-4b3b-4f36-b3b3-456394ada2a3\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 15:29:58.886825 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:29:58.886501    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z6jw4\" (UniqueName: \"kubernetes.io/projected/65aca3d0-4b3b-4f36-b3b3-456394ada2a3-kube-api-access-z6jw4\") pod \"prometheus-k8s-0\" (UID: \"65aca3d0-4b3b-4f36-b3b3-456394ada2a3\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 15:29:58.902957 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:29:58.902926    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bb2321e3-340d-4603-a676-e4eb34ee306c" path="/var/lib/kubelet/pods/bb2321e3-340d-4603-a676-e4eb34ee306c/volumes"
Sep 10 15:29:58.987049 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:29:58.987008    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/65aca3d0-4b3b-4f36-b3b3-456394ada2a3-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"65aca3d0-4b3b-4f36-b3b3-456394ada2a3\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 15:29:58.987288 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:29:58.987067    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/65aca3d0-4b3b-4f36-b3b3-456394ada2a3-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"65aca3d0-4b3b-4f36-b3b3-456394ada2a3\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 15:29:58.987288 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:29:58.987083    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/65aca3d0-4b3b-4f36-b3b3-456394ada2a3-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"65aca3d0-4b3b-4f36-b3b3-456394ada2a3\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 15:29:58.987288 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:29:58.987101    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/65aca3d0-4b3b-4f36-b3b3-456394ada2a3-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"65aca3d0-4b3b-4f36-b3b3-456394ada2a3\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 15:29:58.987288 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:29:58.987129    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z6jw4\" (UniqueName: \"kubernetes.io/projected/65aca3d0-4b3b-4f36-b3b3-456394ada2a3-kube-api-access-z6jw4\") pod \"prometheus-k8s-0\" (UID: \"65aca3d0-4b3b-4f36-b3b3-456394ada2a3\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 15:29:58.987288 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:29:58.987157    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/65aca3d0-4b3b-4f36-b3b3-456394ada2a3-config-out\") pod \"prometheus-k8s-0\" (UID: \"65aca3d0-4b3b-4f36-b3b3-456394ada2a3\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 15:29:58.987288 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:29:58.987176    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/65aca3d0-4b3b-4f36-b3b3-456394ada2a3-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"65aca3d0-4b3b-4f36-b3b3-456394ada2a3\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 15:29:58.987288 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:29:58.987203    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/65aca3d0-4b3b-4f36-b3b3-456394ada2a3-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"65aca3d0-4b3b-4f36-b3b3-456394ada2a3\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 15:29:58.987658 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:29:58.987477    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/65aca3d0-4b3b-4f36-b3b3-456394ada2a3-web-config\") pod \"prometheus-k8s-0\" (UID: \"65aca3d0-4b3b-4f36-b3b3-456394ada2a3\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 15:29:58.987658 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:29:58.987531    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/65aca3d0-4b3b-4f36-b3b3-456394ada2a3-config\") pod \"prometheus-k8s-0\" (UID: \"65aca3d0-4b3b-4f36-b3b3-456394ada2a3\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 15:29:58.987658 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:29:58.987561    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/65aca3d0-4b3b-4f36-b3b3-456394ada2a3-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"65aca3d0-4b3b-4f36-b3b3-456394ada2a3\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 15:29:58.987658 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:29:58.987590    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/65aca3d0-4b3b-4f36-b3b3-456394ada2a3-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"65aca3d0-4b3b-4f36-b3b3-456394ada2a3\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 15:29:58.987658 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:29:58.987625    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/65aca3d0-4b3b-4f36-b3b3-456394ada2a3-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"65aca3d0-4b3b-4f36-b3b3-456394ada2a3\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 15:29:58.987888 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:29:58.987655    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/65aca3d0-4b3b-4f36-b3b3-456394ada2a3-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"65aca3d0-4b3b-4f36-b3b3-456394ada2a3\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 15:29:58.987888 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:29:58.987682    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/65aca3d0-4b3b-4f36-b3b3-456394ada2a3-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"65aca3d0-4b3b-4f36-b3b3-456394ada2a3\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 15:29:58.987888 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:29:58.987711    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/65aca3d0-4b3b-4f36-b3b3-456394ada2a3-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"65aca3d0-4b3b-4f36-b3b3-456394ada2a3\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 15:29:58.987888 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:29:58.987737    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/65aca3d0-4b3b-4f36-b3b3-456394ada2a3-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"65aca3d0-4b3b-4f36-b3b3-456394ada2a3\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 15:29:58.987888 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:29:58.987787    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/65aca3d0-4b3b-4f36-b3b3-456394ada2a3-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"65aca3d0-4b3b-4f36-b3b3-456394ada2a3\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 15:29:58.987888 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:29:58.987811    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/65aca3d0-4b3b-4f36-b3b3-456394ada2a3-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"65aca3d0-4b3b-4f36-b3b3-456394ada2a3\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 15:29:58.987888 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:29:58.987843    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/65aca3d0-4b3b-4f36-b3b3-456394ada2a3-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"65aca3d0-4b3b-4f36-b3b3-456394ada2a3\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 15:29:58.987888 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:29:58.987844    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/65aca3d0-4b3b-4f36-b3b3-456394ada2a3-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"65aca3d0-4b3b-4f36-b3b3-456394ada2a3\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 15:29:58.988277 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:29:58.987737    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/65aca3d0-4b3b-4f36-b3b3-456394ada2a3-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"65aca3d0-4b3b-4f36-b3b3-456394ada2a3\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 15:29:58.988643 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:29:58.988433    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/65aca3d0-4b3b-4f36-b3b3-456394ada2a3-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"65aca3d0-4b3b-4f36-b3b3-456394ada2a3\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 15:29:58.988643 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:29:58.988529    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/65aca3d0-4b3b-4f36-b3b3-456394ada2a3-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"65aca3d0-4b3b-4f36-b3b3-456394ada2a3\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 15:29:58.990216 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:29:58.990191    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/65aca3d0-4b3b-4f36-b3b3-456394ada2a3-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"65aca3d0-4b3b-4f36-b3b3-456394ada2a3\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 15:29:58.990500 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:29:58.990466    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/65aca3d0-4b3b-4f36-b3b3-456394ada2a3-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"65aca3d0-4b3b-4f36-b3b3-456394ada2a3\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 15:29:58.990907 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:29:58.990887    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/65aca3d0-4b3b-4f36-b3b3-456394ada2a3-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"65aca3d0-4b3b-4f36-b3b3-456394ada2a3\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 15:29:58.990985 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:29:58.990886    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/65aca3d0-4b3b-4f36-b3b3-456394ada2a3-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"65aca3d0-4b3b-4f36-b3b3-456394ada2a3\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 15:29:58.992311 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:29:58.991849    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/65aca3d0-4b3b-4f36-b3b3-456394ada2a3-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"65aca3d0-4b3b-4f36-b3b3-456394ada2a3\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 15:29:58.992311 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:29:58.991901    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/65aca3d0-4b3b-4f36-b3b3-456394ada2a3-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"65aca3d0-4b3b-4f36-b3b3-456394ada2a3\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 15:29:58.992311 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:29:58.992271    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/65aca3d0-4b3b-4f36-b3b3-456394ada2a3-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"65aca3d0-4b3b-4f36-b3b3-456394ada2a3\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 15:29:58.992519 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:29:58.992377    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/65aca3d0-4b3b-4f36-b3b3-456394ada2a3-config-out\") pod \"prometheus-k8s-0\" (UID: \"65aca3d0-4b3b-4f36-b3b3-456394ada2a3\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 15:29:58.992519 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:29:58.992443    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/65aca3d0-4b3b-4f36-b3b3-456394ada2a3-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"65aca3d0-4b3b-4f36-b3b3-456394ada2a3\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 15:29:58.992656 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:29:58.992630    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/65aca3d0-4b3b-4f36-b3b3-456394ada2a3-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"65aca3d0-4b3b-4f36-b3b3-456394ada2a3\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 15:29:58.992716 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:29:58.992653    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/65aca3d0-4b3b-4f36-b3b3-456394ada2a3-web-config\") pod \"prometheus-k8s-0\" (UID: \"65aca3d0-4b3b-4f36-b3b3-456394ada2a3\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 15:29:58.992979 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:29:58.992951    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/65aca3d0-4b3b-4f36-b3b3-456394ada2a3-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"65aca3d0-4b3b-4f36-b3b3-456394ada2a3\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 15:29:58.992979 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:29:58.992962    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/65aca3d0-4b3b-4f36-b3b3-456394ada2a3-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"65aca3d0-4b3b-4f36-b3b3-456394ada2a3\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 15:29:58.993273 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:29:58.993252    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/65aca3d0-4b3b-4f36-b3b3-456394ada2a3-config\") pod \"prometheus-k8s-0\" (UID: \"65aca3d0-4b3b-4f36-b3b3-456394ada2a3\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 15:29:58.993931 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:29:58.993916    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/65aca3d0-4b3b-4f36-b3b3-456394ada2a3-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"65aca3d0-4b3b-4f36-b3b3-456394ada2a3\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 15:29:58.997961 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:29:58.997935    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z6jw4\" (UniqueName: \"kubernetes.io/projected/65aca3d0-4b3b-4f36-b3b3-456394ada2a3-kube-api-access-z6jw4\") pod \"prometheus-k8s-0\" (UID: \"65aca3d0-4b3b-4f36-b3b3-456394ada2a3\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 15:29:59.071472 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:29:59.071431    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 15:29:59.352686 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:29:59.352491    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 10 15:29:59.355074 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:29:59.355042    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod65aca3d0_4b3b_4f36_b3b3_456394ada2a3.slice/crio-9e64c6a8fd43b21024c2d74cc1e0c9e8ccde56b6be1a23b753f56ad23ccca6a1 WatchSource:0}: Error finding container 9e64c6a8fd43b21024c2d74cc1e0c9e8ccde56b6be1a23b753f56ad23ccca6a1: Status 404 returned error can't find the container with id 9e64c6a8fd43b21024c2d74cc1e0c9e8ccde56b6be1a23b753f56ad23ccca6a1
Sep 10 15:29:59.703394 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:29:59.703360    2567 generic.go:356] "Generic (PLEG): container finished" podID="65aca3d0-4b3b-4f36-b3b3-456394ada2a3" containerID="48f7b1d2e066bd165e108f30f3f3268d45f002455a160bf5807f49b785954916" exitCode=0
Sep 10 15:29:59.703734 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:29:59.703401    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"65aca3d0-4b3b-4f36-b3b3-456394ada2a3","Type":"ContainerDied","Data":"48f7b1d2e066bd165e108f30f3f3268d45f002455a160bf5807f49b785954916"}
Sep 10 15:29:59.703734 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:29:59.703453    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"65aca3d0-4b3b-4f36-b3b3-456394ada2a3","Type":"ContainerStarted","Data":"9e64c6a8fd43b21024c2d74cc1e0c9e8ccde56b6be1a23b753f56ad23ccca6a1"}
Sep 10 15:30:00.711639 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:30:00.711595    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"65aca3d0-4b3b-4f36-b3b3-456394ada2a3","Type":"ContainerStarted","Data":"c527aedbaf772a6755a12603031ab45a3178de037733037e706a21fe89014363"}
Sep 10 15:30:00.711993 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:30:00.711716    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"65aca3d0-4b3b-4f36-b3b3-456394ada2a3","Type":"ContainerStarted","Data":"34ad2ce9857a4e4c8f26c2a0aa8c0e00f35d5686da57db4b9cf68d0a29d97545"}
Sep 10 15:30:00.711993 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:30:00.711762    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"65aca3d0-4b3b-4f36-b3b3-456394ada2a3","Type":"ContainerStarted","Data":"230abf6be8e2a3d54768293c50801e74f5ad727e112d863e4948f868dc563a6f"}
Sep 10 15:30:00.711993 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:30:00.711798    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"65aca3d0-4b3b-4f36-b3b3-456394ada2a3","Type":"ContainerStarted","Data":"26afc3aa796b907c25accc42d22683de66b6d935eeb8746067a94de194b54143"}
Sep 10 15:30:00.711993 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:30:00.711832    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"65aca3d0-4b3b-4f36-b3b3-456394ada2a3","Type":"ContainerStarted","Data":"99884702ed9966cb72affb368e9cc8b9026d21cfcee280a2a8b99bc19d58fefa"}
Sep 10 15:30:00.711993 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:30:00.711869    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"65aca3d0-4b3b-4f36-b3b3-456394ada2a3","Type":"ContainerStarted","Data":"86f4aa3c30f5b9e37bb6d43f63004185d8cfd5ae4cd4387e4620dc239938d260"}
Sep 10 15:30:00.793095 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:30:00.793029    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.77400853 podStartE2EDuration="2.79301174s" totalImagesPullingTime="0s" totalInitContainerRuntime="19.00321ms" isStatefulPod=false podCreationTimestamp="2026-09-10 15:29:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 15:30:00.792010005 +0000 UTC m=+186.360241737" watchObservedRunningTime="2026-09-10 15:30:00.79301174 +0000 UTC m=+186.361243471"
Sep 10 15:30:00.826466 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:30:00.826427    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:30:00.887128 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:30:00.887093    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:30:04.071622 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:30:04.071563    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 15:30:07.650306 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:30:07.650266    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:30:12.820509 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:30:12.820466    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:30:12.877589 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:30:12.877553    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:30:12.939365 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:30:12.939321    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:30:46.899670 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:30:46.899610    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:30:46.954128 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:30:46.954097    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:30:59.071794 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:30:59.071753    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 15:30:59.086807 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:30:59.086778    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 15:30:59.938845 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:30:59.938817    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 15:31:07.390828 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:31:07.390786    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:31:07.459317 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:31:07.459277    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:31:10.169303 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:31:10.169255    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:31:10.272122 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:31:10.272083    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:31:19.399853 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:31:19.399773    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:31:19.732580 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:31:19.732533    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:31:21.277258 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:31:21.277194    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:31:21.333982 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:31:21.333947    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:31:36.134665 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:31:36.134625    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:31:36.234927 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:31:36.234888    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:31:54.817065 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:31:54.817029    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-t96pm_0b9d2cda-73e3-4e7c-9e17-1a3d903d95cf/console-operator/2.log"
Sep 10 15:31:54.819857 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:31:54.819837    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-t96pm_0b9d2cda-73e3-4e7c-9e17-1a3d903d95cf/console-operator/2.log"
Sep 10 15:31:54.826758 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:31:54.826735    2567 kubelet.go:1648] "Image garbage collection succeeded"
Sep 10 15:32:00.040835 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:32:00.040797    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:32:10.681179 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:32:10.681140    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:32:10.768896 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:32:10.768854    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:32:32.561622 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:32:32.561580    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:32:33.881863 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:32:33.881739    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:32:36.355946 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:32:36.355908    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:32:36.456497 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:32:36.456461    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:32:37.986508 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:32:37.986468    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:32:38.085479 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:32:38.085437    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:32:38.711156 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:32:38.711121    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:32:38.834291 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:32:38.834255    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:32:40.343896 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:32:40.343857    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:32:40.452463 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:32:40.452424    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:32:40.529390 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:32:40.529363    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-fjsgd"]
Sep 10 15:32:40.532599 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:32:40.532578    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-fjsgd"
Sep 10 15:32:40.535031 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:32:40.535006    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Sep 10 15:32:40.535979 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:32:40.535963    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-m5d68\""
Sep 10 15:32:40.536036 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:32:40.536014    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Sep 10 15:32:40.542527 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:32:40.542505    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-fjsgd"]
Sep 10 15:32:40.596741 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:32:40.596658    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j2htg\" (UniqueName: \"kubernetes.io/projected/0cccffc5-6ca3-4cc6-adb3-da15d1630a63-kube-api-access-j2htg\") pod \"cert-manager-cainjector-559cdd8c87-fjsgd\" (UID: \"0cccffc5-6ca3-4cc6-adb3-da15d1630a63\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-fjsgd"
Sep 10 15:32:40.596741 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:32:40.596716    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/0cccffc5-6ca3-4cc6-adb3-da15d1630a63-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-fjsgd\" (UID: \"0cccffc5-6ca3-4cc6-adb3-da15d1630a63\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-fjsgd"
Sep 10 15:32:40.698090 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:32:40.698038    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/0cccffc5-6ca3-4cc6-adb3-da15d1630a63-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-fjsgd\" (UID: \"0cccffc5-6ca3-4cc6-adb3-da15d1630a63\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-fjsgd"
Sep 10 15:32:40.698321 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:32:40.698125    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j2htg\" (UniqueName: \"kubernetes.io/projected/0cccffc5-6ca3-4cc6-adb3-da15d1630a63-kube-api-access-j2htg\") pod \"cert-manager-cainjector-559cdd8c87-fjsgd\" (UID: \"0cccffc5-6ca3-4cc6-adb3-da15d1630a63\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-fjsgd"
Sep 10 15:32:40.708775 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:32:40.708753    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/0cccffc5-6ca3-4cc6-adb3-da15d1630a63-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-fjsgd\" (UID: \"0cccffc5-6ca3-4cc6-adb3-da15d1630a63\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-fjsgd"
Sep 10 15:32:40.708850 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:32:40.708801    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j2htg\" (UniqueName: \"kubernetes.io/projected/0cccffc5-6ca3-4cc6-adb3-da15d1630a63-kube-api-access-j2htg\") pod \"cert-manager-cainjector-559cdd8c87-fjsgd\" (UID: \"0cccffc5-6ca3-4cc6-adb3-da15d1630a63\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-fjsgd"
Sep 10 15:32:40.852955 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:32:40.852857    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-fjsgd"
Sep 10 15:32:41.124614 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:32:41.124580    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-fjsgd"]
Sep 10 15:32:41.127099 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:32:41.127073    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0cccffc5_6ca3_4cc6_adb3_da15d1630a63.slice/crio-329025ee290d581f982385945faf98e53a2f853ac9c934ae9ace6acc71a5c44f WatchSource:0}: Error finding container 329025ee290d581f982385945faf98e53a2f853ac9c934ae9ace6acc71a5c44f: Status 404 returned error can't find the container with id 329025ee290d581f982385945faf98e53a2f853ac9c934ae9ace6acc71a5c44f
Sep 10 15:32:41.129012 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:32:41.128992    2567 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 10 15:32:41.279180 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:32:41.279149    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-fjsgd" event={"ID":"0cccffc5-6ca3-4cc6-adb3-da15d1630a63","Type":"ContainerStarted","Data":"329025ee290d581f982385945faf98e53a2f853ac9c934ae9ace6acc71a5c44f"}
Sep 10 15:32:43.155051 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:32:43.155007    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:32:43.255621 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:32:43.255576    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:32:44.294736 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:32:44.294699    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-fjsgd" event={"ID":"0cccffc5-6ca3-4cc6-adb3-da15d1630a63","Type":"ContainerStarted","Data":"be525c8418fba102654fbd52d8d76aa6ef9bf0f7bffb70bb8b77d95fbec1cf82"}
Sep 10 15:32:44.315307 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:32:44.315264    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-fjsgd" podStartSLOduration=1.276800215 podStartE2EDuration="4.315245101s" totalImagesPullingTime="3.038444886s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 15:32:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 15:32:44.314662676 +0000 UTC m=+349.882894409" watchObservedRunningTime="2026-09-10 15:32:44.315245101 +0000 UTC m=+349.883476835"
Sep 10 15:32:46.356658 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:32:46.356620    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:32:46.457333 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:32:46.457295    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:32:48.487975 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:32:48.487892    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:32:59.559191 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:32:59.559150    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:32:59.660051 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:32:59.660017    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:33:00.560910 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:33:00.560868    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:33:00.660286 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:33:00.660253    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:33:06.055830 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:33:06.055793    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:33:06.155936 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:33:06.155896    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:33:08.219788 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:33:08.219747    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:33:16.427627 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:33:16.427596    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:33:16.484367 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:33:16.484332    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-d87857f9b-hjfmt"]
Sep 10 15:33:16.487562 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:33:16.487544    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-d87857f9b-hjfmt"
Sep 10 15:33:16.491112 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:33:16.491061    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Sep 10 15:33:16.491112 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:33:16.491078    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-knkfv\""
Sep 10 15:33:16.491112 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:33:16.491090    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Sep 10 15:33:16.491386 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:33:16.491065    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Sep 10 15:33:16.491386 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:33:16.491078    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Sep 10 15:33:16.491386 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:33:16.491173    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Sep 10 15:33:16.503366 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:33:16.503343    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-d87857f9b-hjfmt"]
Sep 10 15:33:16.543422 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:33:16.543382    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:33:16.579524 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:33:16.579486    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/0c234d4b-0010-4600-820b-5fb1645b411c-metrics-cert\") pod \"lws-controller-manager-d87857f9b-hjfmt\" (UID: \"0c234d4b-0010-4600-820b-5fb1645b411c\") " pod="openshift-lws-operator/lws-controller-manager-d87857f9b-hjfmt"
Sep 10 15:33:16.579700 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:33:16.579529    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/0c234d4b-0010-4600-820b-5fb1645b411c-manager-config\") pod \"lws-controller-manager-d87857f9b-hjfmt\" (UID: \"0c234d4b-0010-4600-820b-5fb1645b411c\") " pod="openshift-lws-operator/lws-controller-manager-d87857f9b-hjfmt"
Sep 10 15:33:16.579700 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:33:16.579601    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0c234d4b-0010-4600-820b-5fb1645b411c-cert\") pod \"lws-controller-manager-d87857f9b-hjfmt\" (UID: \"0c234d4b-0010-4600-820b-5fb1645b411c\") " pod="openshift-lws-operator/lws-controller-manager-d87857f9b-hjfmt"
Sep 10 15:33:16.579700 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:33:16.579648    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qt4f5\" (UniqueName: \"kubernetes.io/projected/0c234d4b-0010-4600-820b-5fb1645b411c-kube-api-access-qt4f5\") pod \"lws-controller-manager-d87857f9b-hjfmt\" (UID: \"0c234d4b-0010-4600-820b-5fb1645b411c\") " pod="openshift-lws-operator/lws-controller-manager-d87857f9b-hjfmt"
Sep 10 15:33:16.680442 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:33:16.680356    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/0c234d4b-0010-4600-820b-5fb1645b411c-metrics-cert\") pod \"lws-controller-manager-d87857f9b-hjfmt\" (UID: \"0c234d4b-0010-4600-820b-5fb1645b411c\") " pod="openshift-lws-operator/lws-controller-manager-d87857f9b-hjfmt"
Sep 10 15:33:16.680442 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:33:16.680390    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/0c234d4b-0010-4600-820b-5fb1645b411c-manager-config\") pod \"lws-controller-manager-d87857f9b-hjfmt\" (UID: \"0c234d4b-0010-4600-820b-5fb1645b411c\") " pod="openshift-lws-operator/lws-controller-manager-d87857f9b-hjfmt"
Sep 10 15:33:16.680442 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:33:16.680406    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0c234d4b-0010-4600-820b-5fb1645b411c-cert\") pod \"lws-controller-manager-d87857f9b-hjfmt\" (UID: \"0c234d4b-0010-4600-820b-5fb1645b411c\") " pod="openshift-lws-operator/lws-controller-manager-d87857f9b-hjfmt"
Sep 10 15:33:16.680442 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:33:16.680424    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qt4f5\" (UniqueName: \"kubernetes.io/projected/0c234d4b-0010-4600-820b-5fb1645b411c-kube-api-access-qt4f5\") pod \"lws-controller-manager-d87857f9b-hjfmt\" (UID: \"0c234d4b-0010-4600-820b-5fb1645b411c\") " pod="openshift-lws-operator/lws-controller-manager-d87857f9b-hjfmt"
Sep 10 15:33:16.681113 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:33:16.681085    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/0c234d4b-0010-4600-820b-5fb1645b411c-manager-config\") pod \"lws-controller-manager-d87857f9b-hjfmt\" (UID: \"0c234d4b-0010-4600-820b-5fb1645b411c\") " pod="openshift-lws-operator/lws-controller-manager-d87857f9b-hjfmt"
Sep 10 15:33:16.682898 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:33:16.682874    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0c234d4b-0010-4600-820b-5fb1645b411c-cert\") pod \"lws-controller-manager-d87857f9b-hjfmt\" (UID: \"0c234d4b-0010-4600-820b-5fb1645b411c\") " pod="openshift-lws-operator/lws-controller-manager-d87857f9b-hjfmt"
Sep 10 15:33:16.682987 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:33:16.682948    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/0c234d4b-0010-4600-820b-5fb1645b411c-metrics-cert\") pod \"lws-controller-manager-d87857f9b-hjfmt\" (UID: \"0c234d4b-0010-4600-820b-5fb1645b411c\") " pod="openshift-lws-operator/lws-controller-manager-d87857f9b-hjfmt"
Sep 10 15:33:16.692918 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:33:16.692893    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qt4f5\" (UniqueName: \"kubernetes.io/projected/0c234d4b-0010-4600-820b-5fb1645b411c-kube-api-access-qt4f5\") pod \"lws-controller-manager-d87857f9b-hjfmt\" (UID: \"0c234d4b-0010-4600-820b-5fb1645b411c\") " pod="openshift-lws-operator/lws-controller-manager-d87857f9b-hjfmt"
Sep 10 15:33:16.797420 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:33:16.797384    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-d87857f9b-hjfmt"
Sep 10 15:33:17.081189 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:33:17.081162    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-d87857f9b-hjfmt"]
Sep 10 15:33:17.083514 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:33:17.083486    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0c234d4b_0010_4600_820b_5fb1645b411c.slice/crio-5639712e4649dc7da98312ff454ce8e10663a469bc621212eaea6b5d51545ca5 WatchSource:0}: Error finding container 5639712e4649dc7da98312ff454ce8e10663a469bc621212eaea6b5d51545ca5: Status 404 returned error can't find the container with id 5639712e4649dc7da98312ff454ce8e10663a469bc621212eaea6b5d51545ca5
Sep 10 15:33:17.410257 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:33:17.410154    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-d87857f9b-hjfmt" event={"ID":"0c234d4b-0010-4600-820b-5fb1645b411c","Type":"ContainerStarted","Data":"5639712e4649dc7da98312ff454ce8e10663a469bc621212eaea6b5d51545ca5"}
Sep 10 15:33:20.423153 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:33:20.423116    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-d87857f9b-hjfmt" event={"ID":"0c234d4b-0010-4600-820b-5fb1645b411c","Type":"ContainerStarted","Data":"010a275349dd2259f08325577d15d20ff9e2cd4da0b7a77c6211c82fbe51f2af"}
Sep 10 15:33:20.423641 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:33:20.423257    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-d87857f9b-hjfmt"
Sep 10 15:33:20.442064 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:33:20.442013    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-d87857f9b-hjfmt" podStartSLOduration=1.837521473 podStartE2EDuration="4.441995598s" totalImagesPullingTime="2.604474125s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 15:33:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 15:33:20.44144071 +0000 UTC m=+386.009672453" watchObservedRunningTime="2026-09-10 15:33:20.441995598 +0000 UTC m=+386.010227327"
Sep 10 15:33:20.557888 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:33:20.557861    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:33:20.655590 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:33:20.655552    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:33:21.182338 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:33:21.182304    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:33:21.546910 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:33:21.546830    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:33:21.649309 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:33:21.649274    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:33:21.691384 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:33:21.691351    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-797ffb6d9c-sws7t"]
Sep 10 15:33:21.693442 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:33:21.693426    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-797ffb6d9c-sws7t"
Sep 10 15:33:21.697713 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:33:21.697688    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Sep 10 15:33:21.698215 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:33:21.698195    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Sep 10 15:33:21.698521 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:33:21.698498    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Sep 10 15:33:21.698521 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:33:21.698515    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-f7nx8\""
Sep 10 15:33:21.698693 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:33:21.698508    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Sep 10 15:33:21.710524 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:33:21.710491    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-797ffb6d9c-sws7t"]
Sep 10 15:33:21.722302 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:33:21.722265    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7fb99e31-3fe0-462b-af8e-cc5d038b6377-tmp\") pod \"opendatahub-operator-controller-manager-797ffb6d9c-sws7t\" (UID: \"7fb99e31-3fe0-462b-af8e-cc5d038b6377\") " pod="opendatahub/opendatahub-operator-controller-manager-797ffb6d9c-sws7t"
Sep 10 15:33:21.725247 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:33:21.725207    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/7fb99e31-3fe0-462b-af8e-cc5d038b6377-manifests\") pod \"opendatahub-operator-controller-manager-797ffb6d9c-sws7t\" (UID: \"7fb99e31-3fe0-462b-af8e-cc5d038b6377\") " pod="opendatahub/opendatahub-operator-controller-manager-797ffb6d9c-sws7t"
Sep 10 15:33:21.826185 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:33:21.826088    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/7fb99e31-3fe0-462b-af8e-cc5d038b6377-webhook-cert\") pod \"opendatahub-operator-controller-manager-797ffb6d9c-sws7t\" (UID: \"7fb99e31-3fe0-462b-af8e-cc5d038b6377\") " pod="opendatahub/opendatahub-operator-controller-manager-797ffb6d9c-sws7t"
Sep 10 15:33:21.826185 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:33:21.826150    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7fb99e31-3fe0-462b-af8e-cc5d038b6377-tmp\") pod \"opendatahub-operator-controller-manager-797ffb6d9c-sws7t\" (UID: \"7fb99e31-3fe0-462b-af8e-cc5d038b6377\") " pod="opendatahub/opendatahub-operator-controller-manager-797ffb6d9c-sws7t"
Sep 10 15:33:21.826185 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:33:21.826170    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6st4m\" (UniqueName: \"kubernetes.io/projected/7fb99e31-3fe0-462b-af8e-cc5d038b6377-kube-api-access-6st4m\") pod \"opendatahub-operator-controller-manager-797ffb6d9c-sws7t\" (UID: \"7fb99e31-3fe0-462b-af8e-cc5d038b6377\") " pod="opendatahub/opendatahub-operator-controller-manager-797ffb6d9c-sws7t"
Sep 10 15:33:21.826422 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:33:21.826190    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/7fb99e31-3fe0-462b-af8e-cc5d038b6377-manifests\") pod \"opendatahub-operator-controller-manager-797ffb6d9c-sws7t\" (UID: \"7fb99e31-3fe0-462b-af8e-cc5d038b6377\") " pod="opendatahub/opendatahub-operator-controller-manager-797ffb6d9c-sws7t"
Sep 10 15:33:21.826422 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:33:21.826250    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/7fb99e31-3fe0-462b-af8e-cc5d038b6377-apiservice-cert\") pod \"opendatahub-operator-controller-manager-797ffb6d9c-sws7t\" (UID: \"7fb99e31-3fe0-462b-af8e-cc5d038b6377\") " pod="opendatahub/opendatahub-operator-controller-manager-797ffb6d9c-sws7t"
Sep 10 15:33:21.826640 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:33:21.826619    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/7fb99e31-3fe0-462b-af8e-cc5d038b6377-manifests\") pod \"opendatahub-operator-controller-manager-797ffb6d9c-sws7t\" (UID: \"7fb99e31-3fe0-462b-af8e-cc5d038b6377\") " pod="opendatahub/opendatahub-operator-controller-manager-797ffb6d9c-sws7t"
Sep 10 15:33:21.828397 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:33:21.828376    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7fb99e31-3fe0-462b-af8e-cc5d038b6377-tmp\") pod \"opendatahub-operator-controller-manager-797ffb6d9c-sws7t\" (UID: \"7fb99e31-3fe0-462b-af8e-cc5d038b6377\") " pod="opendatahub/opendatahub-operator-controller-manager-797ffb6d9c-sws7t"
Sep 10 15:33:21.856846 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:33:21.856812    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:33:21.927159 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:33:21.927114    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/7fb99e31-3fe0-462b-af8e-cc5d038b6377-apiservice-cert\") pod \"opendatahub-operator-controller-manager-797ffb6d9c-sws7t\" (UID: \"7fb99e31-3fe0-462b-af8e-cc5d038b6377\") " pod="opendatahub/opendatahub-operator-controller-manager-797ffb6d9c-sws7t"
Sep 10 15:33:21.927354 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:33:21.927170    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/7fb99e31-3fe0-462b-af8e-cc5d038b6377-webhook-cert\") pod \"opendatahub-operator-controller-manager-797ffb6d9c-sws7t\" (UID: \"7fb99e31-3fe0-462b-af8e-cc5d038b6377\") " pod="opendatahub/opendatahub-operator-controller-manager-797ffb6d9c-sws7t"
Sep 10 15:33:21.927354 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:33:21.927211    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6st4m\" (UniqueName: \"kubernetes.io/projected/7fb99e31-3fe0-462b-af8e-cc5d038b6377-kube-api-access-6st4m\") pod \"opendatahub-operator-controller-manager-797ffb6d9c-sws7t\" (UID: \"7fb99e31-3fe0-462b-af8e-cc5d038b6377\") " pod="opendatahub/opendatahub-operator-controller-manager-797ffb6d9c-sws7t"
Sep 10 15:33:21.929634 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:33:21.929600    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/7fb99e31-3fe0-462b-af8e-cc5d038b6377-webhook-cert\") pod \"opendatahub-operator-controller-manager-797ffb6d9c-sws7t\" (UID: \"7fb99e31-3fe0-462b-af8e-cc5d038b6377\") " pod="opendatahub/opendatahub-operator-controller-manager-797ffb6d9c-sws7t"
Sep 10 15:33:21.929756 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:33:21.929645    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/7fb99e31-3fe0-462b-af8e-cc5d038b6377-apiservice-cert\") pod \"opendatahub-operator-controller-manager-797ffb6d9c-sws7t\" (UID: \"7fb99e31-3fe0-462b-af8e-cc5d038b6377\") " pod="opendatahub/opendatahub-operator-controller-manager-797ffb6d9c-sws7t"
Sep 10 15:33:21.945065 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:33:21.945034    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6st4m\" (UniqueName: \"kubernetes.io/projected/7fb99e31-3fe0-462b-af8e-cc5d038b6377-kube-api-access-6st4m\") pod \"opendatahub-operator-controller-manager-797ffb6d9c-sws7t\" (UID: \"7fb99e31-3fe0-462b-af8e-cc5d038b6377\") " pod="opendatahub/opendatahub-operator-controller-manager-797ffb6d9c-sws7t"
Sep 10 15:33:21.959164 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:33:21.959124    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:33:22.004083 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:33:22.004039    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-797ffb6d9c-sws7t"
Sep 10 15:33:22.058206 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:33:22.058132    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:33:22.179701 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:33:22.179664    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:33:22.268156 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:33:22.268119    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:33:22.286730 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:33:22.286700    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-797ffb6d9c-sws7t"]
Sep 10 15:33:22.290179 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:33:22.290151    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7fb99e31_3fe0_462b_af8e_cc5d038b6377.slice/crio-c182771fdaf327a16aac78f3e1daf91d91ca3e619546b81371947c316b5c16d9 WatchSource:0}: Error finding container c182771fdaf327a16aac78f3e1daf91d91ca3e619546b81371947c316b5c16d9: Status 404 returned error can't find the container with id c182771fdaf327a16aac78f3e1daf91d91ca3e619546b81371947c316b5c16d9
Sep 10 15:33:22.431946 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:33:22.431852    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-797ffb6d9c-sws7t" event={"ID":"7fb99e31-3fe0-462b-af8e-cc5d038b6377","Type":"ContainerStarted","Data":"c182771fdaf327a16aac78f3e1daf91d91ca3e619546b81371947c316b5c16d9"}
Sep 10 15:33:25.446168 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:33:25.446130    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-797ffb6d9c-sws7t" event={"ID":"7fb99e31-3fe0-462b-af8e-cc5d038b6377","Type":"ContainerStarted","Data":"555e9096bc89bc991a25ef000c392b76686a8bde0156fc671bd60a65eb8565b2"}
Sep 10 15:33:26.451675 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:33:26.451641    2567 generic.go:356] "Generic (PLEG): container finished" podID="7fb99e31-3fe0-462b-af8e-cc5d038b6377" containerID="555e9096bc89bc991a25ef000c392b76686a8bde0156fc671bd60a65eb8565b2" exitCode=0
Sep 10 15:33:26.452013 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:33:26.451697    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-797ffb6d9c-sws7t" event={"ID":"7fb99e31-3fe0-462b-af8e-cc5d038b6377","Type":"ContainerDied","Data":"555e9096bc89bc991a25ef000c392b76686a8bde0156fc671bd60a65eb8565b2"}
Sep 10 15:33:27.458642 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:33:27.458599    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-797ffb6d9c-sws7t" event={"ID":"7fb99e31-3fe0-462b-af8e-cc5d038b6377","Type":"ContainerStarted","Data":"96be0ba3513fe1ca0165b31f229c0d5a1cfb6170f5aeac2d244de8105c2d2fd8"}
Sep 10 15:33:27.459114 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:33:27.458696    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-797ffb6d9c-sws7t"
Sep 10 15:33:27.497606 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:33:27.497545    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-797ffb6d9c-sws7t" podStartSLOduration=1.9950172099999999 podStartE2EDuration="6.497530071s" totalImagesPullingTime="3.44284802s" totalInitContainerRuntime="1.059664841s" isStatefulPod=false podCreationTimestamp="2026-09-10 15:33:21 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 15:33:27.496941919 +0000 UTC m=+393.065173649" watchObservedRunningTime="2026-09-10 15:33:27.497530071 +0000 UTC m=+393.065761801"
Sep 10 15:33:31.430158 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:33:31.430125    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-d87857f9b-hjfmt"
Sep 10 15:33:32.132002 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:33:32.131964    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:33:32.272908 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:33:32.272878    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:33:35.321792 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:33:35.321756    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:33:35.426161 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:33:35.426122    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:33:35.762081 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:33:35.762035    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:33:35.865457 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:33:35.865418    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:33:38.464940 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:33:38.464912    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-797ffb6d9c-sws7t"
Sep 10 15:33:41.033170 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:33:41.033133    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:33:41.151627 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:33:41.151592    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:33:41.419299 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:33:41.419259    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:33:41.596310 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:33:41.596276    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-785ff899f9-7scqc"]
Sep 10 15:33:41.601530 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:33:41.601509    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-785ff899f9-7scqc"
Sep 10 15:33:41.605342 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:33:41.605321    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-dmj8w\""
Sep 10 15:33:41.605474 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:33:41.605323    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Sep 10 15:33:41.608695 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:33:41.608351    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Sep 10 15:33:41.608695 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:33:41.608354    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Sep 10 15:33:41.609354 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:33:41.609332    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Sep 10 15:33:41.627161 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:33:41.627128    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:33:41.628472 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:33:41.628446    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-785ff899f9-7scqc"]
Sep 10 15:33:41.690840 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:33:41.690746    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cd5pd\" (UniqueName: \"kubernetes.io/projected/db9bca7a-dea3-4901-a6ce-b6b85cf40b32-kube-api-access-cd5pd\") pod \"kube-auth-proxy-785ff899f9-7scqc\" (UID: \"db9bca7a-dea3-4901-a6ce-b6b85cf40b32\") " pod="openshift-ingress/kube-auth-proxy-785ff899f9-7scqc"
Sep 10 15:33:41.690840 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:33:41.690797    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/db9bca7a-dea3-4901-a6ce-b6b85cf40b32-tls-certs\") pod \"kube-auth-proxy-785ff899f9-7scqc\" (UID: \"db9bca7a-dea3-4901-a6ce-b6b85cf40b32\") " pod="openshift-ingress/kube-auth-proxy-785ff899f9-7scqc"
Sep 10 15:33:41.691039 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:33:41.690943    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/db9bca7a-dea3-4901-a6ce-b6b85cf40b32-tmp\") pod \"kube-auth-proxy-785ff899f9-7scqc\" (UID: \"db9bca7a-dea3-4901-a6ce-b6b85cf40b32\") " pod="openshift-ingress/kube-auth-proxy-785ff899f9-7scqc"
Sep 10 15:33:41.792060 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:33:41.792017    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/db9bca7a-dea3-4901-a6ce-b6b85cf40b32-tmp\") pod \"kube-auth-proxy-785ff899f9-7scqc\" (UID: \"db9bca7a-dea3-4901-a6ce-b6b85cf40b32\") " pod="openshift-ingress/kube-auth-proxy-785ff899f9-7scqc"
Sep 10 15:33:41.792284 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:33:41.792079    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cd5pd\" (UniqueName: \"kubernetes.io/projected/db9bca7a-dea3-4901-a6ce-b6b85cf40b32-kube-api-access-cd5pd\") pod \"kube-auth-proxy-785ff899f9-7scqc\" (UID: \"db9bca7a-dea3-4901-a6ce-b6b85cf40b32\") " pod="openshift-ingress/kube-auth-proxy-785ff899f9-7scqc"
Sep 10 15:33:41.792284 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:33:41.792107    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/db9bca7a-dea3-4901-a6ce-b6b85cf40b32-tls-certs\") pod \"kube-auth-proxy-785ff899f9-7scqc\" (UID: \"db9bca7a-dea3-4901-a6ce-b6b85cf40b32\") " pod="openshift-ingress/kube-auth-proxy-785ff899f9-7scqc"
Sep 10 15:33:41.794376 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:33:41.794349    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/db9bca7a-dea3-4901-a6ce-b6b85cf40b32-tmp\") pod \"kube-auth-proxy-785ff899f9-7scqc\" (UID: \"db9bca7a-dea3-4901-a6ce-b6b85cf40b32\") " pod="openshift-ingress/kube-auth-proxy-785ff899f9-7scqc"
Sep 10 15:33:41.794654 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:33:41.794632    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/db9bca7a-dea3-4901-a6ce-b6b85cf40b32-tls-certs\") pod \"kube-auth-proxy-785ff899f9-7scqc\" (UID: \"db9bca7a-dea3-4901-a6ce-b6b85cf40b32\") " pod="openshift-ingress/kube-auth-proxy-785ff899f9-7scqc"
Sep 10 15:33:41.822814 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:33:41.822784    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cd5pd\" (UniqueName: \"kubernetes.io/projected/db9bca7a-dea3-4901-a6ce-b6b85cf40b32-kube-api-access-cd5pd\") pod \"kube-auth-proxy-785ff899f9-7scqc\" (UID: \"db9bca7a-dea3-4901-a6ce-b6b85cf40b32\") " pod="openshift-ingress/kube-auth-proxy-785ff899f9-7scqc"
Sep 10 15:33:41.911722 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:33:41.911689    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-785ff899f9-7scqc"
Sep 10 15:33:42.190400 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:33:42.190364    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-785ff899f9-7scqc"]
Sep 10 15:33:42.193621 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:33:42.193589    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddb9bca7a_dea3_4901_a6ce_b6b85cf40b32.slice/crio-28c784d4a12e6ea52fec59d991c3038756a8cbe1ce187175b3cad085f867628a WatchSource:0}: Error finding container 28c784d4a12e6ea52fec59d991c3038756a8cbe1ce187175b3cad085f867628a: Status 404 returned error can't find the container with id 28c784d4a12e6ea52fec59d991c3038756a8cbe1ce187175b3cad085f867628a
Sep 10 15:33:42.517463 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:33:42.517432    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-785ff899f9-7scqc" event={"ID":"db9bca7a-dea3-4901-a6ce-b6b85cf40b32","Type":"ContainerStarted","Data":"28c784d4a12e6ea52fec59d991c3038756a8cbe1ce187175b3cad085f867628a"}
Sep 10 15:33:43.135287 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:33:43.135251    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:33:43.249341 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:33:43.249296    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:33:44.321157 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:33:44.321105    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:33:45.530321 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:33:45.530216    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-785ff899f9-7scqc" event={"ID":"db9bca7a-dea3-4901-a6ce-b6b85cf40b32","Type":"ContainerStarted","Data":"aee32d00e03976eeb61c640f73504400a41b82a8d5ba77e1561ad812cf00ed62"}
Sep 10 15:33:45.551431 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:33:45.551368    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-785ff899f9-7scqc" podStartSLOduration=1.534740443 podStartE2EDuration="4.551348095s" totalImagesPullingTime="3.016607652s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 15:33:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 15:33:45.551198536 +0000 UTC m=+411.119430266" watchObservedRunningTime="2026-09-10 15:33:45.551348095 +0000 UTC m=+411.119579823"
Sep 10 15:33:48.924989 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:33:48.924954    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:33:49.040282 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:33:49.040226    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:33:49.776416 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:33:49.776375    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:33:50.018500 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:33:50.018462    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:33:50.286742 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:33:50.286711    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:33:52.419732 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:33:52.419696    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:33:52.495770 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:33:52.495734    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:33:52.559309 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:33:52.559275    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:33:52.694924 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:33:52.694836    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:33:53.004741 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:33:53.004650    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:33:53.271775 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:33:53.271671    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:33:54.346297 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:33:54.346259    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:33:54.447203 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:33:54.447170    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:34:01.858643 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:34:01.858605    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:34:02.031667 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:34:02.031616    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:34:02.131989 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:34:02.131898    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:34:02.251907 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:34:02.251872    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:34:03.207701 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:34:03.207660    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:34:03.308619 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:34:03.308581    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:34:03.414672 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:34:03.414634    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:34:04.815775 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:34:04.815740    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:34:47.313280 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:34:47.313225    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:34:47.380776 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:34:47.380743    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:34:53.339041 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:34:53.339006    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:34:53.443383 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:34:53.443344    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:35:09.243144 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:35:09.243106    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:35:10.041557 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:35:10.041516    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:35:10.837782 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:35:10.837746    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:35:13.064569 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:35:13.064532    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:35:13.167349 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:35:13.167303    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:35:14.436269 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:35:14.436208    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:35:16.288619 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:35:16.288568    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:35:16.451600 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:35:16.451568    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:35:18.285849 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:35:18.285816    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:35:18.475633 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:35:18.475583    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:35:18.637129 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:35:18.637033    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:35:18.741199 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:35:18.741158    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:35:19.885512 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:35:19.885478    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:35:21.457078 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:35:21.457037    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:35:21.540997 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:35:21.540965    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:35:24.275954 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:35:24.275919    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:35:24.410297 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:35:24.410220    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:35:24.699888 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:35:24.699856    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:35:24.831497 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:35:24.831469    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:35:25.095063 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:35:25.094971    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:35:25.198108 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:35:25.198064    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:35:31.019202 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:35:31.019160    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:35:31.187071 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:35:31.187037    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:35:31.462857 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:35:31.462824    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:35:31.593541 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:35:31.593506    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:35:32.330043 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:35:32.330011    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-mkltt"]
Sep 10 15:35:32.333666 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:35:32.333644    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-mkltt"
Sep 10 15:35:32.336765 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:35:32.336742    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Sep 10 15:35:32.337446 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:35:32.337426    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Sep 10 15:35:32.338835 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:35:32.338807    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-rk9mn\""
Sep 10 15:35:32.354884 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:35:32.354858    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-mkltt"]
Sep 10 15:35:32.387818 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:35:32.387774    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a18d6bf7-c326-4db6-9360-2352c000b57e-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-mkltt\" (UID: \"a18d6bf7-c326-4db6-9360-2352c000b57e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-mkltt"
Sep 10 15:35:32.387818 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:35:32.387826    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tjqr4\" (UniqueName: \"kubernetes.io/projected/a18d6bf7-c326-4db6-9360-2352c000b57e-kube-api-access-tjqr4\") pod \"kuadrant-operator-controller-manager-76f8f686bf-mkltt\" (UID: \"a18d6bf7-c326-4db6-9360-2352c000b57e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-mkltt"
Sep 10 15:35:32.488514 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:35:32.488477    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a18d6bf7-c326-4db6-9360-2352c000b57e-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-mkltt\" (UID: \"a18d6bf7-c326-4db6-9360-2352c000b57e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-mkltt"
Sep 10 15:35:32.488705 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:35:32.488529    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tjqr4\" (UniqueName: \"kubernetes.io/projected/a18d6bf7-c326-4db6-9360-2352c000b57e-kube-api-access-tjqr4\") pod \"kuadrant-operator-controller-manager-76f8f686bf-mkltt\" (UID: \"a18d6bf7-c326-4db6-9360-2352c000b57e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-mkltt"
Sep 10 15:35:32.488849 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:35:32.488827    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a18d6bf7-c326-4db6-9360-2352c000b57e-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-mkltt\" (UID: \"a18d6bf7-c326-4db6-9360-2352c000b57e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-mkltt"
Sep 10 15:35:32.507248 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:35:32.507206    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tjqr4\" (UniqueName: \"kubernetes.io/projected/a18d6bf7-c326-4db6-9360-2352c000b57e-kube-api-access-tjqr4\") pod \"kuadrant-operator-controller-manager-76f8f686bf-mkltt\" (UID: \"a18d6bf7-c326-4db6-9360-2352c000b57e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-mkltt"
Sep 10 15:35:32.645460 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:35:32.645372    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-mkltt"
Sep 10 15:35:32.925584 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:35:32.925559    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-mkltt"]
Sep 10 15:35:32.928524 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:35:32.928495    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda18d6bf7_c326_4db6_9360_2352c000b57e.slice/crio-1f40484d00485b312a236c392c9ac26040e77a54dc98afb11a701ef251a62427 WatchSource:0}: Error finding container 1f40484d00485b312a236c392c9ac26040e77a54dc98afb11a701ef251a62427: Status 404 returned error can't find the container with id 1f40484d00485b312a236c392c9ac26040e77a54dc98afb11a701ef251a62427
Sep 10 15:35:32.940836 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:35:32.940810    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-mkltt" event={"ID":"a18d6bf7-c326-4db6-9360-2352c000b57e","Type":"ContainerStarted","Data":"1f40484d00485b312a236c392c9ac26040e77a54dc98afb11a701ef251a62427"}
Sep 10 15:35:33.811343 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:35:33.811294    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:35:33.934286 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:35:33.934222    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:35:40.246903 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:35:40.246864    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:35:40.351163 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:35:40.351126    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:35:45.478805 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:35:45.478765    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:35:45.657311 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:35:45.657273    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:35:45.805641 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:35:45.805549    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:35:45.908422 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:35:45.908379    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:35:46.000478 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:35:46.000442    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-mkltt" event={"ID":"a18d6bf7-c326-4db6-9360-2352c000b57e","Type":"ContainerStarted","Data":"5914b7ee4eb22f07349756a229def9e124c0b2dddeaea9f8d975fca17732cdca"}
Sep 10 15:35:46.000658 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:35:46.000564    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-mkltt"
Sep 10 15:35:46.037339 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:35:46.037281    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-mkltt" podStartSLOduration=1.435596061 podStartE2EDuration="14.037263387s" totalImagesPullingTime="12.601667326s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 15:35:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 15:35:46.035540691 +0000 UTC m=+531.603772438" watchObservedRunningTime="2026-09-10 15:35:46.037263387 +0000 UTC m=+531.605495110"
Sep 10 15:35:46.074173 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:35:46.074081    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:35:46.178755 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:35:46.178716    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:35:47.122051 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:35:47.121966    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-mkltt"]
Sep 10 15:35:48.009315 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:35:48.009265    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-mkltt" podUID="a18d6bf7-c326-4db6-9360-2352c000b57e" containerName="manager" containerID="cri-o://5914b7ee4eb22f07349756a229def9e124c0b2dddeaea9f8d975fca17732cdca" gracePeriod=10
Sep 10 15:35:48.249143 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:35:48.249117    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-mkltt"
Sep 10 15:35:48.321539 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:35:48.321444    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tjqr4\" (UniqueName: \"kubernetes.io/projected/a18d6bf7-c326-4db6-9360-2352c000b57e-kube-api-access-tjqr4\") pod \"a18d6bf7-c326-4db6-9360-2352c000b57e\" (UID: \"a18d6bf7-c326-4db6-9360-2352c000b57e\") "
Sep 10 15:35:48.321710 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:35:48.321563    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a18d6bf7-c326-4db6-9360-2352c000b57e-extensions-socket-volume\") pod \"a18d6bf7-c326-4db6-9360-2352c000b57e\" (UID: \"a18d6bf7-c326-4db6-9360-2352c000b57e\") "
Sep 10 15:35:48.322058 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:35:48.322029    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a18d6bf7-c326-4db6-9360-2352c000b57e-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "a18d6bf7-c326-4db6-9360-2352c000b57e" (UID: "a18d6bf7-c326-4db6-9360-2352c000b57e"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 15:35:48.323672 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:35:48.323643    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a18d6bf7-c326-4db6-9360-2352c000b57e-kube-api-access-tjqr4" (OuterVolumeSpecName: "kube-api-access-tjqr4") pod "a18d6bf7-c326-4db6-9360-2352c000b57e" (UID: "a18d6bf7-c326-4db6-9360-2352c000b57e"). InnerVolumeSpecName "kube-api-access-tjqr4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 15:35:48.422653 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:35:48.422618    2567 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a18d6bf7-c326-4db6-9360-2352c000b57e-extensions-socket-volume\") on node \"ip-10-0-128-241.ec2.internal\" DevicePath \"\""
Sep 10 15:35:48.422653 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:35:48.422648    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tjqr4\" (UniqueName: \"kubernetes.io/projected/a18d6bf7-c326-4db6-9360-2352c000b57e-kube-api-access-tjqr4\") on node \"ip-10-0-128-241.ec2.internal\" DevicePath \"\""
Sep 10 15:35:49.014722 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:35:49.014684    2567 generic.go:356] "Generic (PLEG): container finished" podID="a18d6bf7-c326-4db6-9360-2352c000b57e" containerID="5914b7ee4eb22f07349756a229def9e124c0b2dddeaea9f8d975fca17732cdca" exitCode=0
Sep 10 15:35:49.014722 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:35:49.014727    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-mkltt"
Sep 10 15:35:49.014944 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:35:49.014775    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-mkltt" event={"ID":"a18d6bf7-c326-4db6-9360-2352c000b57e","Type":"ContainerDied","Data":"5914b7ee4eb22f07349756a229def9e124c0b2dddeaea9f8d975fca17732cdca"}
Sep 10 15:35:49.014944 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:35:49.014824    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-mkltt" event={"ID":"a18d6bf7-c326-4db6-9360-2352c000b57e","Type":"ContainerDied","Data":"1f40484d00485b312a236c392c9ac26040e77a54dc98afb11a701ef251a62427"}
Sep 10 15:35:49.014944 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:35:49.014855    2567 scope.go:117] "RemoveContainer" containerID="5914b7ee4eb22f07349756a229def9e124c0b2dddeaea9f8d975fca17732cdca"
Sep 10 15:35:49.022903 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:35:49.022883    2567 scope.go:117] "RemoveContainer" containerID="5914b7ee4eb22f07349756a229def9e124c0b2dddeaea9f8d975fca17732cdca"
Sep 10 15:35:49.023222 ip-10-0-128-241 kubenswrapper[2567]: E0910 15:35:49.023198    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5914b7ee4eb22f07349756a229def9e124c0b2dddeaea9f8d975fca17732cdca\": container with ID starting with 5914b7ee4eb22f07349756a229def9e124c0b2dddeaea9f8d975fca17732cdca not found: ID does not exist" containerID="5914b7ee4eb22f07349756a229def9e124c0b2dddeaea9f8d975fca17732cdca"
Sep 10 15:35:49.023328 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:35:49.023249    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5914b7ee4eb22f07349756a229def9e124c0b2dddeaea9f8d975fca17732cdca"} err="failed to get container status \"5914b7ee4eb22f07349756a229def9e124c0b2dddeaea9f8d975fca17732cdca\": rpc error: code = NotFound desc = could not find container \"5914b7ee4eb22f07349756a229def9e124c0b2dddeaea9f8d975fca17732cdca\": container with ID starting with 5914b7ee4eb22f07349756a229def9e124c0b2dddeaea9f8d975fca17732cdca not found: ID does not exist"
Sep 10 15:35:49.040933 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:35:49.040900    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-mkltt"]
Sep 10 15:35:49.048003 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:35:49.047972    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-mkltt"]
Sep 10 15:35:50.902848 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:35:50.902813    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a18d6bf7-c326-4db6-9360-2352c000b57e" path="/var/lib/kubelet/pods/a18d6bf7-c326-4db6-9360-2352c000b57e/volumes"
Sep 10 15:35:53.066779 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:35:53.066741    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:35:53.219145 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:35:53.219102    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:35:55.491305 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:35:55.491267    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:35:55.555905 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:35:55.555870    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:36:02.152202 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:36:02.152166    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:36:02.288782 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:36:02.288739    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:36:03.912490 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:36:03.912459    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:36:04.235894 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:36:04.235844    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:36:06.661780 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:36:06.661736    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:36:06.682036 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:36:06.682004    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-d6tsq"]
Sep 10 15:36:06.682349 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:36:06.682335    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a18d6bf7-c326-4db6-9360-2352c000b57e" containerName="manager"
Sep 10 15:36:06.682398 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:36:06.682352    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="a18d6bf7-c326-4db6-9360-2352c000b57e" containerName="manager"
Sep 10 15:36:06.682437 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:36:06.682425    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="a18d6bf7-c326-4db6-9360-2352c000b57e" containerName="manager"
Sep 10 15:36:06.686993 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:36:06.686970    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-8588749977-d6tsq"
Sep 10 15:36:06.689977 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:36:06.689953    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Sep 10 15:36:06.690190 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:36:06.690165    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Sep 10 15:36:06.690317 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:36:06.690174    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Sep 10 15:36:06.690317 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:36:06.690302    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-bgbdk\""
Sep 10 15:36:06.702685 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:36:06.702612    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-d6tsq"]
Sep 10 15:36:06.772744 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:36:06.772706    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-d6tsq"]
Sep 10 15:36:06.779590 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:36:06.779566    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r62z9\" (UniqueName: \"kubernetes.io/projected/eaf11499-1cc2-4df7-aefb-2a126e0c6b97-kube-api-access-r62z9\") pod \"limitador-limitador-8588749977-d6tsq\" (UID: \"eaf11499-1cc2-4df7-aefb-2a126e0c6b97\") " pod="kuadrant-system/limitador-limitador-8588749977-d6tsq"
Sep 10 15:36:06.779691 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:36:06.779599    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/eaf11499-1cc2-4df7-aefb-2a126e0c6b97-config-file\") pod \"limitador-limitador-8588749977-d6tsq\" (UID: \"eaf11499-1cc2-4df7-aefb-2a126e0c6b97\") " pod="kuadrant-system/limitador-limitador-8588749977-d6tsq"
Sep 10 15:36:06.803517 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:36:06.803483    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:36:06.880358 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:36:06.880316    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r62z9\" (UniqueName: \"kubernetes.io/projected/eaf11499-1cc2-4df7-aefb-2a126e0c6b97-kube-api-access-r62z9\") pod \"limitador-limitador-8588749977-d6tsq\" (UID: \"eaf11499-1cc2-4df7-aefb-2a126e0c6b97\") " pod="kuadrant-system/limitador-limitador-8588749977-d6tsq"
Sep 10 15:36:06.880549 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:36:06.880361    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/eaf11499-1cc2-4df7-aefb-2a126e0c6b97-config-file\") pod \"limitador-limitador-8588749977-d6tsq\" (UID: \"eaf11499-1cc2-4df7-aefb-2a126e0c6b97\") " pod="kuadrant-system/limitador-limitador-8588749977-d6tsq"
Sep 10 15:36:06.880999 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:36:06.880974    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/eaf11499-1cc2-4df7-aefb-2a126e0c6b97-config-file\") pod \"limitador-limitador-8588749977-d6tsq\" (UID: \"eaf11499-1cc2-4df7-aefb-2a126e0c6b97\") " pod="kuadrant-system/limitador-limitador-8588749977-d6tsq"
Sep 10 15:36:06.902078 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:36:06.902033    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r62z9\" (UniqueName: \"kubernetes.io/projected/eaf11499-1cc2-4df7-aefb-2a126e0c6b97-kube-api-access-r62z9\") pod \"limitador-limitador-8588749977-d6tsq\" (UID: \"eaf11499-1cc2-4df7-aefb-2a126e0c6b97\") " pod="kuadrant-system/limitador-limitador-8588749977-d6tsq"
Sep 10 15:36:06.982899 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:36:06.982861    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:36:06.999168 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:36:06.999134    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-8588749977-d6tsq"
Sep 10 15:36:07.286418 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:36:07.286340    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-d6tsq"]
Sep 10 15:36:07.289505 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:36:07.289474    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeaf11499_1cc2_4df7_aefb_2a126e0c6b97.slice/crio-7bbb98046063ae7c9ca0c2d2ecbc5affb5589b97b9568a6bafa8a28f1b67dd9a WatchSource:0}: Error finding container 7bbb98046063ae7c9ca0c2d2ecbc5affb5589b97b9568a6bafa8a28f1b67dd9a: Status 404 returned error can't find the container with id 7bbb98046063ae7c9ca0c2d2ecbc5affb5589b97b9568a6bafa8a28f1b67dd9a
Sep 10 15:36:07.621480 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:36:07.621391    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-dzlff"]
Sep 10 15:36:07.625860 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:36:07.625836    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7bcfd8f9cd-dzlff"
Sep 10 15:36:07.628709 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:36:07.628685    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-f77c8\""
Sep 10 15:36:07.635635 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:36:07.635613    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-dzlff"]
Sep 10 15:36:07.652456 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:36:07.652425    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-59c974bd48-cp6mz"]
Sep 10 15:36:07.655500 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:36:07.655482    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-59c974bd48-cp6mz"
Sep 10 15:36:07.667542 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:36:07.667516    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-59c974bd48-cp6mz"]
Sep 10 15:36:07.694249 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:36:07.694184    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:36:07.787493 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:36:07.787457    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jxs46\" (UniqueName: \"kubernetes.io/projected/2ab3ca50-ef33-44ff-9a7f-1edfa88adfbf-kube-api-access-jxs46\") pod \"authorino-7bcfd8f9cd-dzlff\" (UID: \"2ab3ca50-ef33-44ff-9a7f-1edfa88adfbf\") " pod="kuadrant-system/authorino-7bcfd8f9cd-dzlff"
Sep 10 15:36:07.787670 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:36:07.787505    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f6t54\" (UniqueName: \"kubernetes.io/projected/39f00b6f-4439-48b7-baf1-9f0b62a7f3a7-kube-api-access-f6t54\") pod \"authorino-59c974bd48-cp6mz\" (UID: \"39f00b6f-4439-48b7-baf1-9f0b62a7f3a7\") " pod="kuadrant-system/authorino-59c974bd48-cp6mz"
Sep 10 15:36:07.798210 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:36:07.798179    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:36:07.888567 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:36:07.888472    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f6t54\" (UniqueName: \"kubernetes.io/projected/39f00b6f-4439-48b7-baf1-9f0b62a7f3a7-kube-api-access-f6t54\") pod \"authorino-59c974bd48-cp6mz\" (UID: \"39f00b6f-4439-48b7-baf1-9f0b62a7f3a7\") " pod="kuadrant-system/authorino-59c974bd48-cp6mz"
Sep 10 15:36:07.888729 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:36:07.888598    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jxs46\" (UniqueName: \"kubernetes.io/projected/2ab3ca50-ef33-44ff-9a7f-1edfa88adfbf-kube-api-access-jxs46\") pod \"authorino-7bcfd8f9cd-dzlff\" (UID: \"2ab3ca50-ef33-44ff-9a7f-1edfa88adfbf\") " pod="kuadrant-system/authorino-7bcfd8f9cd-dzlff"
Sep 10 15:36:07.899156 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:36:07.899120    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f6t54\" (UniqueName: \"kubernetes.io/projected/39f00b6f-4439-48b7-baf1-9f0b62a7f3a7-kube-api-access-f6t54\") pod \"authorino-59c974bd48-cp6mz\" (UID: \"39f00b6f-4439-48b7-baf1-9f0b62a7f3a7\") " pod="kuadrant-system/authorino-59c974bd48-cp6mz"
Sep 10 15:36:07.900727 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:36:07.900706    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jxs46\" (UniqueName: \"kubernetes.io/projected/2ab3ca50-ef33-44ff-9a7f-1edfa88adfbf-kube-api-access-jxs46\") pod \"authorino-7bcfd8f9cd-dzlff\" (UID: \"2ab3ca50-ef33-44ff-9a7f-1edfa88adfbf\") " pod="kuadrant-system/authorino-7bcfd8f9cd-dzlff"
Sep 10 15:36:07.935692 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:36:07.935653    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7bcfd8f9cd-dzlff"
Sep 10 15:36:07.964881 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:36:07.964848    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-59c974bd48-cp6mz"
Sep 10 15:36:08.096445 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:36:08.096412    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-8588749977-d6tsq" event={"ID":"eaf11499-1cc2-4df7-aefb-2a126e0c6b97","Type":"ContainerStarted","Data":"7bbb98046063ae7c9ca0c2d2ecbc5affb5589b97b9568a6bafa8a28f1b67dd9a"}
Sep 10 15:36:08.208558 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:36:08.208417    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-dzlff"]
Sep 10 15:36:08.211085 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:36:08.211050    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2ab3ca50_ef33_44ff_9a7f_1edfa88adfbf.slice/crio-3c8fae4b840ad1611cd98752d404f25cbd0f6ffb2f4164787d48a28ba66e0ed0 WatchSource:0}: Error finding container 3c8fae4b840ad1611cd98752d404f25cbd0f6ffb2f4164787d48a28ba66e0ed0: Status 404 returned error can't find the container with id 3c8fae4b840ad1611cd98752d404f25cbd0f6ffb2f4164787d48a28ba66e0ed0
Sep 10 15:36:08.259067 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:36:08.259041    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-59c974bd48-cp6mz"]
Sep 10 15:36:08.432301 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:36:08.432259    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod39f00b6f_4439_48b7_baf1_9f0b62a7f3a7.slice/crio-b4433b56088b201b4a54c43f5ecb73b0a564716a998830438a5aeab76d81eee7 WatchSource:0}: Error finding container b4433b56088b201b4a54c43f5ecb73b0a564716a998830438a5aeab76d81eee7: Status 404 returned error can't find the container with id b4433b56088b201b4a54c43f5ecb73b0a564716a998830438a5aeab76d81eee7
Sep 10 15:36:09.102022 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:36:09.101985    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-59c974bd48-cp6mz" event={"ID":"39f00b6f-4439-48b7-baf1-9f0b62a7f3a7","Type":"ContainerStarted","Data":"b4433b56088b201b4a54c43f5ecb73b0a564716a998830438a5aeab76d81eee7"}
Sep 10 15:36:09.103077 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:36:09.103057    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7bcfd8f9cd-dzlff" event={"ID":"2ab3ca50-ef33-44ff-9a7f-1edfa88adfbf","Type":"ContainerStarted","Data":"3c8fae4b840ad1611cd98752d404f25cbd0f6ffb2f4164787d48a28ba66e0ed0"}
Sep 10 15:36:09.104347 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:36:09.104325    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-8588749977-d6tsq" event={"ID":"eaf11499-1cc2-4df7-aefb-2a126e0c6b97","Type":"ContainerStarted","Data":"ba8084e058aa3daf6609ced59e3f9379859640256de02ef681248b7c3f6abbac"}
Sep 10 15:36:09.104453 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:36:09.104440    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-8588749977-d6tsq"
Sep 10 15:36:09.125983 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:36:09.125938    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-8588749977-d6tsq" podStartSLOduration=1.9292355030000001 podStartE2EDuration="3.125921749s" totalImagesPullingTime="1.196686246s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 15:36:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 15:36:09.124597494 +0000 UTC m=+554.692829228" watchObservedRunningTime="2026-09-10 15:36:09.125921749 +0000 UTC m=+554.694153479"
Sep 10 15:36:11.115201 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:36:11.115163    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-59c974bd48-cp6mz" event={"ID":"39f00b6f-4439-48b7-baf1-9f0b62a7f3a7","Type":"ContainerStarted","Data":"1043a499e77319acd1bdb49f9ad23df77c668b645a0123172fbc79322860757b"}
Sep 10 15:36:11.116548 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:36:11.116526    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7bcfd8f9cd-dzlff" event={"ID":"2ab3ca50-ef33-44ff-9a7f-1edfa88adfbf","Type":"ContainerStarted","Data":"dbcf1c437b4be24b7ebfeabb9cbd9059c80594256cb7067ba3cee179563363a9"}
Sep 10 15:36:11.131866 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:36:11.131826    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-59c974bd48-cp6mz" podStartSLOduration=2.268296333 podStartE2EDuration="4.131812431s" totalImagesPullingTime="1.863516098s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 15:36:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 15:36:11.131281428 +0000 UTC m=+556.699513158" watchObservedRunningTime="2026-09-10 15:36:11.131812431 +0000 UTC m=+556.700044160"
Sep 10 15:36:11.146439 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:36:11.146390    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-7bcfd8f9cd-dzlff" podStartSLOduration=2.058013184 podStartE2EDuration="4.146373808s" totalImagesPullingTime="2.088360624s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 15:36:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 15:36:11.145419399 +0000 UTC m=+556.713651130" watchObservedRunningTime="2026-09-10 15:36:11.146373808 +0000 UTC m=+556.714605538"
Sep 10 15:36:11.174065 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:36:11.174033    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-dzlff"]
Sep 10 15:36:13.124570 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:36:13.124531    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-7bcfd8f9cd-dzlff" podUID="2ab3ca50-ef33-44ff-9a7f-1edfa88adfbf" containerName="authorino" containerID="cri-o://dbcf1c437b4be24b7ebfeabb9cbd9059c80594256cb7067ba3cee179563363a9" gracePeriod=30
Sep 10 15:36:13.366716 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:36:13.366692    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7bcfd8f9cd-dzlff"
Sep 10 15:36:13.531519 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:36:13.531486    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jxs46\" (UniqueName: \"kubernetes.io/projected/2ab3ca50-ef33-44ff-9a7f-1edfa88adfbf-kube-api-access-jxs46\") pod \"2ab3ca50-ef33-44ff-9a7f-1edfa88adfbf\" (UID: \"2ab3ca50-ef33-44ff-9a7f-1edfa88adfbf\") "
Sep 10 15:36:13.533520 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:36:13.533490    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2ab3ca50-ef33-44ff-9a7f-1edfa88adfbf-kube-api-access-jxs46" (OuterVolumeSpecName: "kube-api-access-jxs46") pod "2ab3ca50-ef33-44ff-9a7f-1edfa88adfbf" (UID: "2ab3ca50-ef33-44ff-9a7f-1edfa88adfbf"). InnerVolumeSpecName "kube-api-access-jxs46". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 15:36:13.632043 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:36:13.632006    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jxs46\" (UniqueName: \"kubernetes.io/projected/2ab3ca50-ef33-44ff-9a7f-1edfa88adfbf-kube-api-access-jxs46\") on node \"ip-10-0-128-241.ec2.internal\" DevicePath \"\""
Sep 10 15:36:14.130211 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:36:14.130175    2567 generic.go:356] "Generic (PLEG): container finished" podID="2ab3ca50-ef33-44ff-9a7f-1edfa88adfbf" containerID="dbcf1c437b4be24b7ebfeabb9cbd9059c80594256cb7067ba3cee179563363a9" exitCode=0
Sep 10 15:36:14.130606 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:36:14.130216    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7bcfd8f9cd-dzlff"
Sep 10 15:36:14.130606 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:36:14.130262    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7bcfd8f9cd-dzlff" event={"ID":"2ab3ca50-ef33-44ff-9a7f-1edfa88adfbf","Type":"ContainerDied","Data":"dbcf1c437b4be24b7ebfeabb9cbd9059c80594256cb7067ba3cee179563363a9"}
Sep 10 15:36:14.130606 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:36:14.130308    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7bcfd8f9cd-dzlff" event={"ID":"2ab3ca50-ef33-44ff-9a7f-1edfa88adfbf","Type":"ContainerDied","Data":"3c8fae4b840ad1611cd98752d404f25cbd0f6ffb2f4164787d48a28ba66e0ed0"}
Sep 10 15:36:14.130606 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:36:14.130328    2567 scope.go:117] "RemoveContainer" containerID="dbcf1c437b4be24b7ebfeabb9cbd9059c80594256cb7067ba3cee179563363a9"
Sep 10 15:36:14.138467 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:36:14.138445    2567 scope.go:117] "RemoveContainer" containerID="dbcf1c437b4be24b7ebfeabb9cbd9059c80594256cb7067ba3cee179563363a9"
Sep 10 15:36:14.138745 ip-10-0-128-241 kubenswrapper[2567]: E0910 15:36:14.138727    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dbcf1c437b4be24b7ebfeabb9cbd9059c80594256cb7067ba3cee179563363a9\": container with ID starting with dbcf1c437b4be24b7ebfeabb9cbd9059c80594256cb7067ba3cee179563363a9 not found: ID does not exist" containerID="dbcf1c437b4be24b7ebfeabb9cbd9059c80594256cb7067ba3cee179563363a9"
Sep 10 15:36:14.138797 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:36:14.138755    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dbcf1c437b4be24b7ebfeabb9cbd9059c80594256cb7067ba3cee179563363a9"} err="failed to get container status \"dbcf1c437b4be24b7ebfeabb9cbd9059c80594256cb7067ba3cee179563363a9\": rpc error: code = NotFound desc = could not find container \"dbcf1c437b4be24b7ebfeabb9cbd9059c80594256cb7067ba3cee179563363a9\": container with ID starting with dbcf1c437b4be24b7ebfeabb9cbd9059c80594256cb7067ba3cee179563363a9 not found: ID does not exist"
Sep 10 15:36:14.152266 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:36:14.152224    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-dzlff"]
Sep 10 15:36:14.156390 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:36:14.156366    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-dzlff"]
Sep 10 15:36:14.903415 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:36:14.903379    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2ab3ca50-ef33-44ff-9a7f-1edfa88adfbf" path="/var/lib/kubelet/pods/2ab3ca50-ef33-44ff-9a7f-1edfa88adfbf/volumes"
Sep 10 15:36:20.111072 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:36:20.111035    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-8588749977-d6tsq"
Sep 10 15:36:20.267952 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:36:20.267918    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-d6tsq"]
Sep 10 15:36:20.268126 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:36:20.268100    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-8588749977-d6tsq" podUID="eaf11499-1cc2-4df7-aefb-2a126e0c6b97" containerName="limitador" containerID="cri-o://ba8084e058aa3daf6609ced59e3f9379859640256de02ef681248b7c3f6abbac" gracePeriod=30
Sep 10 15:36:21.162328 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:36:21.162296    2567 generic.go:356] "Generic (PLEG): container finished" podID="eaf11499-1cc2-4df7-aefb-2a126e0c6b97" containerID="ba8084e058aa3daf6609ced59e3f9379859640256de02ef681248b7c3f6abbac" exitCode=0
Sep 10 15:36:21.162715 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:36:21.162366    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-8588749977-d6tsq" event={"ID":"eaf11499-1cc2-4df7-aefb-2a126e0c6b97","Type":"ContainerDied","Data":"ba8084e058aa3daf6609ced59e3f9379859640256de02ef681248b7c3f6abbac"}
Sep 10 15:36:21.209557 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:36:21.209533    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-8588749977-d6tsq"
Sep 10 15:36:21.394117 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:36:21.394022    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/eaf11499-1cc2-4df7-aefb-2a126e0c6b97-config-file\") pod \"eaf11499-1cc2-4df7-aefb-2a126e0c6b97\" (UID: \"eaf11499-1cc2-4df7-aefb-2a126e0c6b97\") "
Sep 10 15:36:21.394339 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:36:21.394123    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r62z9\" (UniqueName: \"kubernetes.io/projected/eaf11499-1cc2-4df7-aefb-2a126e0c6b97-kube-api-access-r62z9\") pod \"eaf11499-1cc2-4df7-aefb-2a126e0c6b97\" (UID: \"eaf11499-1cc2-4df7-aefb-2a126e0c6b97\") "
Sep 10 15:36:21.394525 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:36:21.394497    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/eaf11499-1cc2-4df7-aefb-2a126e0c6b97-config-file" (OuterVolumeSpecName: "config-file") pod "eaf11499-1cc2-4df7-aefb-2a126e0c6b97" (UID: "eaf11499-1cc2-4df7-aefb-2a126e0c6b97"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 15:36:21.396165 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:36:21.396138    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eaf11499-1cc2-4df7-aefb-2a126e0c6b97-kube-api-access-r62z9" (OuterVolumeSpecName: "kube-api-access-r62z9") pod "eaf11499-1cc2-4df7-aefb-2a126e0c6b97" (UID: "eaf11499-1cc2-4df7-aefb-2a126e0c6b97"). InnerVolumeSpecName "kube-api-access-r62z9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 15:36:21.495258 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:36:21.495195    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r62z9\" (UniqueName: \"kubernetes.io/projected/eaf11499-1cc2-4df7-aefb-2a126e0c6b97-kube-api-access-r62z9\") on node \"ip-10-0-128-241.ec2.internal\" DevicePath \"\""
Sep 10 15:36:21.495258 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:36:21.495254    2567 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/eaf11499-1cc2-4df7-aefb-2a126e0c6b97-config-file\") on node \"ip-10-0-128-241.ec2.internal\" DevicePath \"\""
Sep 10 15:36:22.167861 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:36:22.167820    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-8588749977-d6tsq" event={"ID":"eaf11499-1cc2-4df7-aefb-2a126e0c6b97","Type":"ContainerDied","Data":"7bbb98046063ae7c9ca0c2d2ecbc5affb5589b97b9568a6bafa8a28f1b67dd9a"}
Sep 10 15:36:22.167861 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:36:22.167845    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-8588749977-d6tsq"
Sep 10 15:36:22.168349 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:36:22.167896    2567 scope.go:117] "RemoveContainer" containerID="ba8084e058aa3daf6609ced59e3f9379859640256de02ef681248b7c3f6abbac"
Sep 10 15:36:22.191374 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:36:22.191345    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-d6tsq"]
Sep 10 15:36:22.196193 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:36:22.196171    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-d6tsq"]
Sep 10 15:36:22.730157 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:36:22.730127    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:36:22.829523 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:36:22.829479    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:36:22.902943 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:36:22.902907    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="eaf11499-1cc2-4df7-aefb-2a126e0c6b97" path="/var/lib/kubelet/pods/eaf11499-1cc2-4df7-aefb-2a126e0c6b97/volumes"
Sep 10 15:36:23.264676 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:36:23.264645    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-tcnpw"]
Sep 10 15:36:23.265124 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:36:23.264933    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="eaf11499-1cc2-4df7-aefb-2a126e0c6b97" containerName="limitador"
Sep 10 15:36:23.265124 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:36:23.264944    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="eaf11499-1cc2-4df7-aefb-2a126e0c6b97" containerName="limitador"
Sep 10 15:36:23.265124 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:36:23.264955    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2ab3ca50-ef33-44ff-9a7f-1edfa88adfbf" containerName="authorino"
Sep 10 15:36:23.265124 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:36:23.264961    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="2ab3ca50-ef33-44ff-9a7f-1edfa88adfbf" containerName="authorino"
Sep 10 15:36:23.265124 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:36:23.265005    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="eaf11499-1cc2-4df7-aefb-2a126e0c6b97" containerName="limitador"
Sep 10 15:36:23.265124 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:36:23.265014    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="2ab3ca50-ef33-44ff-9a7f-1edfa88adfbf" containerName="authorino"
Sep 10 15:36:23.267550 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:36:23.267530    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-tcnpw"
Sep 10 15:36:23.270032 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:36:23.270010    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"default-dockercfg-d9q4v\""
Sep 10 15:36:23.270151 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:36:23.270052    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\""
Sep 10 15:36:23.270151 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:36:23.270072    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Sep 10 15:36:23.271051 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:36:23.271036    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 10 15:36:23.277624 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:36:23.277602    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-tcnpw"]
Sep 10 15:36:23.313873 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:36:23.313842    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:36:23.410575 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:36:23.410537    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-txbgt\" (UniqueName: \"kubernetes.io/projected/c17c97c0-957e-46eb-be34-41e3f644284a-kube-api-access-txbgt\") pod \"postgres-69fb4594cf-tcnpw\" (UID: \"c17c97c0-957e-46eb-be34-41e3f644284a\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-tcnpw"
Sep 10 15:36:23.410750 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:36:23.410663    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/c17c97c0-957e-46eb-be34-41e3f644284a-data\") pod \"postgres-69fb4594cf-tcnpw\" (UID: \"c17c97c0-957e-46eb-be34-41e3f644284a\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-tcnpw"
Sep 10 15:36:23.511777 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:36:23.511731    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/c17c97c0-957e-46eb-be34-41e3f644284a-data\") pod \"postgres-69fb4594cf-tcnpw\" (UID: \"c17c97c0-957e-46eb-be34-41e3f644284a\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-tcnpw"
Sep 10 15:36:23.511777 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:36:23.511782    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-txbgt\" (UniqueName: \"kubernetes.io/projected/c17c97c0-957e-46eb-be34-41e3f644284a-kube-api-access-txbgt\") pod \"postgres-69fb4594cf-tcnpw\" (UID: \"c17c97c0-957e-46eb-be34-41e3f644284a\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-tcnpw"
Sep 10 15:36:23.512148 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:36:23.512127    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/c17c97c0-957e-46eb-be34-41e3f644284a-data\") pod \"postgres-69fb4594cf-tcnpw\" (UID: \"c17c97c0-957e-46eb-be34-41e3f644284a\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-tcnpw"
Sep 10 15:36:23.521047 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:36:23.520981    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-txbgt\" (UniqueName: \"kubernetes.io/projected/c17c97c0-957e-46eb-be34-41e3f644284a-kube-api-access-txbgt\") pod \"postgres-69fb4594cf-tcnpw\" (UID: \"c17c97c0-957e-46eb-be34-41e3f644284a\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-tcnpw"
Sep 10 15:36:23.581198 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:36:23.581165    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-tcnpw"
Sep 10 15:36:23.856938 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:36:23.856770    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-tcnpw"]
Sep 10 15:36:23.859417 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:36:23.859385    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc17c97c0_957e_46eb_be34_41e3f644284a.slice/crio-ec9470cc8b228d4e285b968d2f9de9af02fc83ea1d28cbee0ef66f13ef61f63e WatchSource:0}: Error finding container ec9470cc8b228d4e285b968d2f9de9af02fc83ea1d28cbee0ef66f13ef61f63e: Status 404 returned error can't find the container with id ec9470cc8b228d4e285b968d2f9de9af02fc83ea1d28cbee0ef66f13ef61f63e
Sep 10 15:36:24.178643 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:36:24.178607    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-tcnpw" event={"ID":"c17c97c0-957e-46eb-be34-41e3f644284a","Type":"ContainerStarted","Data":"ec9470cc8b228d4e285b968d2f9de9af02fc83ea1d28cbee0ef66f13ef61f63e"}
Sep 10 15:36:29.204162 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:36:29.204125    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-tcnpw" event={"ID":"c17c97c0-957e-46eb-be34-41e3f644284a","Type":"ContainerStarted","Data":"f0c5274225403d5b4c99ae0884fd64f6e378716a511a3477da0ee6acd618776b"}
Sep 10 15:36:29.204591 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:36:29.204260    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-tcnpw"
Sep 10 15:36:29.221561 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:36:29.221511    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/postgres-69fb4594cf-tcnpw" podStartSLOduration=1.576758689 podStartE2EDuration="6.221494067s" totalImagesPullingTime="4.644735378s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 15:36:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 15:36:29.220483742 +0000 UTC m=+574.788715474" watchObservedRunningTime="2026-09-10 15:36:29.221494067 +0000 UTC m=+574.789725797"
Sep 10 15:36:35.237142 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:36:35.237109    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-tcnpw"
Sep 10 15:36:35.593612 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:36:35.593494    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:36:35.725635 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:36:35.725598    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:36:35.820770 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:36:35.820735    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-557f876f69-fgtf6"]
Sep 10 15:36:35.823985 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:36:35.823964    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-557f876f69-fgtf6"
Sep 10 15:36:35.830878 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:36:35.830857    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-557f876f69-fgtf6"]
Sep 10 15:36:35.882518 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:36:35.882434    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-557f876f69-fgtf6"]
Sep 10 15:36:35.882693 ip-10-0-128-241 kubenswrapper[2567]: E0910 15:36:35.882675    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-wgcj6], unattached volumes=[], failed to process volumes=[kube-api-access-wgcj6]: context canceled" pod="kuadrant-system/authorino-557f876f69-fgtf6" podUID="587db4ae-5028-4552-80c2-df2b05c5986c"
Sep 10 15:36:35.908397 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:36:35.908358    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wgcj6\" (UniqueName: \"kubernetes.io/projected/587db4ae-5028-4552-80c2-df2b05c5986c-kube-api-access-wgcj6\") pod \"authorino-557f876f69-fgtf6\" (UID: \"587db4ae-5028-4552-80c2-df2b05c5986c\") " pod="kuadrant-system/authorino-557f876f69-fgtf6"
Sep 10 15:36:35.911020 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:36:35.910989    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-6d4dfc649f-8mm7m"]
Sep 10 15:36:35.914046 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:36:35.914027    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6d4dfc649f-8mm7m"
Sep 10 15:36:35.916446 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:36:35.916427    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\""
Sep 10 15:36:35.922158 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:36:35.922136    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-6d4dfc649f-8mm7m"]
Sep 10 15:36:36.009298 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:36:36.009225    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/73935e69-7aed-4169-92a2-0fb051fc94e4-tls-cert\") pod \"authorino-6d4dfc649f-8mm7m\" (UID: \"73935e69-7aed-4169-92a2-0fb051fc94e4\") " pod="kuadrant-system/authorino-6d4dfc649f-8mm7m"
Sep 10 15:36:36.009298 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:36:36.009292    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-497d5\" (UniqueName: \"kubernetes.io/projected/73935e69-7aed-4169-92a2-0fb051fc94e4-kube-api-access-497d5\") pod \"authorino-6d4dfc649f-8mm7m\" (UID: \"73935e69-7aed-4169-92a2-0fb051fc94e4\") " pod="kuadrant-system/authorino-6d4dfc649f-8mm7m"
Sep 10 15:36:36.009549 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:36:36.009426    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wgcj6\" (UniqueName: \"kubernetes.io/projected/587db4ae-5028-4552-80c2-df2b05c5986c-kube-api-access-wgcj6\") pod \"authorino-557f876f69-fgtf6\" (UID: \"587db4ae-5028-4552-80c2-df2b05c5986c\") " pod="kuadrant-system/authorino-557f876f69-fgtf6"
Sep 10 15:36:36.019127 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:36:36.019092    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wgcj6\" (UniqueName: \"kubernetes.io/projected/587db4ae-5028-4552-80c2-df2b05c5986c-kube-api-access-wgcj6\") pod \"authorino-557f876f69-fgtf6\" (UID: \"587db4ae-5028-4552-80c2-df2b05c5986c\") " pod="kuadrant-system/authorino-557f876f69-fgtf6"
Sep 10 15:36:36.064910 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:36:36.064876    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6d4dfc649f-8mm7m"]
Sep 10 15:36:36.065155 ip-10-0-128-241 kubenswrapper[2567]: E0910 15:36:36.065136    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-497d5 tls-cert], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-6d4dfc649f-8mm7m" podUID="73935e69-7aed-4169-92a2-0fb051fc94e4"
Sep 10 15:36:36.096662 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:36:36.096630    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-8477bc89d6-vwnd4"]
Sep 10 15:36:36.099921 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:36:36.099903    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-8477bc89d6-vwnd4"
Sep 10 15:36:36.108926 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:36:36.108749    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-8477bc89d6-vwnd4"]
Sep 10 15:36:36.109838 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:36:36.109810    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/9c9fa106-613a-45bd-b644-d33ea0bee161-tls-cert\") pod \"authorino-8477bc89d6-vwnd4\" (UID: \"9c9fa106-613a-45bd-b644-d33ea0bee161\") " pod="kuadrant-system/authorino-8477bc89d6-vwnd4"
Sep 10 15:36:36.109952 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:36:36.109845    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9jsb8\" (UniqueName: \"kubernetes.io/projected/9c9fa106-613a-45bd-b644-d33ea0bee161-kube-api-access-9jsb8\") pod \"authorino-8477bc89d6-vwnd4\" (UID: \"9c9fa106-613a-45bd-b644-d33ea0bee161\") " pod="kuadrant-system/authorino-8477bc89d6-vwnd4"
Sep 10 15:36:36.110001 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:36:36.109944    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/73935e69-7aed-4169-92a2-0fb051fc94e4-tls-cert\") pod \"authorino-6d4dfc649f-8mm7m\" (UID: \"73935e69-7aed-4169-92a2-0fb051fc94e4\") " pod="kuadrant-system/authorino-6d4dfc649f-8mm7m"
Sep 10 15:36:36.110001 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:36:36.109981    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-497d5\" (UniqueName: \"kubernetes.io/projected/73935e69-7aed-4169-92a2-0fb051fc94e4-kube-api-access-497d5\") pod \"authorino-6d4dfc649f-8mm7m\" (UID: \"73935e69-7aed-4169-92a2-0fb051fc94e4\") " pod="kuadrant-system/authorino-6d4dfc649f-8mm7m"
Sep 10 15:36:36.112824 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:36:36.112796    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/73935e69-7aed-4169-92a2-0fb051fc94e4-tls-cert\") pod \"authorino-6d4dfc649f-8mm7m\" (UID: \"73935e69-7aed-4169-92a2-0fb051fc94e4\") " pod="kuadrant-system/authorino-6d4dfc649f-8mm7m"
Sep 10 15:36:36.125774 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:36:36.125752    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-497d5\" (UniqueName: \"kubernetes.io/projected/73935e69-7aed-4169-92a2-0fb051fc94e4-kube-api-access-497d5\") pod \"authorino-6d4dfc649f-8mm7m\" (UID: \"73935e69-7aed-4169-92a2-0fb051fc94e4\") " pod="kuadrant-system/authorino-6d4dfc649f-8mm7m"
Sep 10 15:36:36.211276 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:36:36.211213    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/9c9fa106-613a-45bd-b644-d33ea0bee161-tls-cert\") pod \"authorino-8477bc89d6-vwnd4\" (UID: \"9c9fa106-613a-45bd-b644-d33ea0bee161\") " pod="kuadrant-system/authorino-8477bc89d6-vwnd4"
Sep 10 15:36:36.211473 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:36:36.211293    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9jsb8\" (UniqueName: \"kubernetes.io/projected/9c9fa106-613a-45bd-b644-d33ea0bee161-kube-api-access-9jsb8\") pod \"authorino-8477bc89d6-vwnd4\" (UID: \"9c9fa106-613a-45bd-b644-d33ea0bee161\") " pod="kuadrant-system/authorino-8477bc89d6-vwnd4"
Sep 10 15:36:36.213638 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:36:36.213615    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/9c9fa106-613a-45bd-b644-d33ea0bee161-tls-cert\") pod \"authorino-8477bc89d6-vwnd4\" (UID: \"9c9fa106-613a-45bd-b644-d33ea0bee161\") " pod="kuadrant-system/authorino-8477bc89d6-vwnd4"
Sep 10 15:36:36.220442 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:36:36.220419    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9jsb8\" (UniqueName: \"kubernetes.io/projected/9c9fa106-613a-45bd-b644-d33ea0bee161-kube-api-access-9jsb8\") pod \"authorino-8477bc89d6-vwnd4\" (UID: \"9c9fa106-613a-45bd-b644-d33ea0bee161\") " pod="kuadrant-system/authorino-8477bc89d6-vwnd4"
Sep 10 15:36:36.233654 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:36:36.233630    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-557f876f69-fgtf6"
Sep 10 15:36:36.233788 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:36:36.233636    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6d4dfc649f-8mm7m"
Sep 10 15:36:36.240850 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:36:36.240823    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-557f876f69-fgtf6"
Sep 10 15:36:36.243379 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:36:36.243364    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6d4dfc649f-8mm7m"
Sep 10 15:36:36.312074 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:36:36.312043    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wgcj6\" (UniqueName: \"kubernetes.io/projected/587db4ae-5028-4552-80c2-df2b05c5986c-kube-api-access-wgcj6\") pod \"587db4ae-5028-4552-80c2-df2b05c5986c\" (UID: \"587db4ae-5028-4552-80c2-df2b05c5986c\") "
Sep 10 15:36:36.312074 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:36:36.312081    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/73935e69-7aed-4169-92a2-0fb051fc94e4-tls-cert\") pod \"73935e69-7aed-4169-92a2-0fb051fc94e4\" (UID: \"73935e69-7aed-4169-92a2-0fb051fc94e4\") "
Sep 10 15:36:36.312340 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:36:36.312125    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-497d5\" (UniqueName: \"kubernetes.io/projected/73935e69-7aed-4169-92a2-0fb051fc94e4-kube-api-access-497d5\") pod \"73935e69-7aed-4169-92a2-0fb051fc94e4\" (UID: \"73935e69-7aed-4169-92a2-0fb051fc94e4\") "
Sep 10 15:36:36.314206 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:36:36.314173    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/73935e69-7aed-4169-92a2-0fb051fc94e4-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "73935e69-7aed-4169-92a2-0fb051fc94e4" (UID: "73935e69-7aed-4169-92a2-0fb051fc94e4"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 15:36:36.314327 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:36:36.314179    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/73935e69-7aed-4169-92a2-0fb051fc94e4-kube-api-access-497d5" (OuterVolumeSpecName: "kube-api-access-497d5") pod "73935e69-7aed-4169-92a2-0fb051fc94e4" (UID: "73935e69-7aed-4169-92a2-0fb051fc94e4"). InnerVolumeSpecName "kube-api-access-497d5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 15:36:36.314327 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:36:36.314207    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/587db4ae-5028-4552-80c2-df2b05c5986c-kube-api-access-wgcj6" (OuterVolumeSpecName: "kube-api-access-wgcj6") pod "587db4ae-5028-4552-80c2-df2b05c5986c" (UID: "587db4ae-5028-4552-80c2-df2b05c5986c"). InnerVolumeSpecName "kube-api-access-wgcj6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 15:36:36.412703 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:36:36.412663    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-8477bc89d6-vwnd4"
Sep 10 15:36:36.412879 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:36:36.412856    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wgcj6\" (UniqueName: \"kubernetes.io/projected/587db4ae-5028-4552-80c2-df2b05c5986c-kube-api-access-wgcj6\") on node \"ip-10-0-128-241.ec2.internal\" DevicePath \"\""
Sep 10 15:36:36.412928 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:36:36.412882    2567 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/73935e69-7aed-4169-92a2-0fb051fc94e4-tls-cert\") on node \"ip-10-0-128-241.ec2.internal\" DevicePath \"\""
Sep 10 15:36:36.412928 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:36:36.412895    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-497d5\" (UniqueName: \"kubernetes.io/projected/73935e69-7aed-4169-92a2-0fb051fc94e4-kube-api-access-497d5\") on node \"ip-10-0-128-241.ec2.internal\" DevicePath \"\""
Sep 10 15:36:36.683797 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:36:36.683760    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-8477bc89d6-vwnd4"]
Sep 10 15:36:36.686744 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:36:36.686713    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9c9fa106_613a_45bd_b644_d33ea0bee161.slice/crio-3bda675018526d0c2f92efa473eb27d86360bbf71e624444e5709988c119c01f WatchSource:0}: Error finding container 3bda675018526d0c2f92efa473eb27d86360bbf71e624444e5709988c119c01f: Status 404 returned error can't find the container with id 3bda675018526d0c2f92efa473eb27d86360bbf71e624444e5709988c119c01f
Sep 10 15:36:37.240901 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:36:37.240860    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-8477bc89d6-vwnd4" event={"ID":"9c9fa106-613a-45bd-b644-d33ea0bee161","Type":"ContainerStarted","Data":"a06a7b907484aaf5aae97aae8bb873a0411933efa15bfb1fe68da79881220763"}
Sep 10 15:36:37.240901 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:36:37.240897    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-557f876f69-fgtf6"
Sep 10 15:36:37.241383 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:36:37.240904    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-8477bc89d6-vwnd4" event={"ID":"9c9fa106-613a-45bd-b644-d33ea0bee161","Type":"ContainerStarted","Data":"3bda675018526d0c2f92efa473eb27d86360bbf71e624444e5709988c119c01f"}
Sep 10 15:36:37.241383 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:36:37.240872    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6d4dfc649f-8mm7m"
Sep 10 15:36:37.262651 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:36:37.262599    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-8477bc89d6-vwnd4" podStartSLOduration=0.817192775 podStartE2EDuration="1.262582166s" totalImagesPullingTime="445.389391ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 15:36:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 15:36:37.260894699 +0000 UTC m=+582.829126441" watchObservedRunningTime="2026-09-10 15:36:37.262582166 +0000 UTC m=+582.830813896"
Sep 10 15:36:37.286162 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:36:37.286088    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-557f876f69-fgtf6"]
Sep 10 15:36:37.289910 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:36:37.289882    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-59c974bd48-cp6mz"]
Sep 10 15:36:37.290116 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:36:37.290090    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-59c974bd48-cp6mz" podUID="39f00b6f-4439-48b7-baf1-9f0b62a7f3a7" containerName="authorino" containerID="cri-o://1043a499e77319acd1bdb49f9ad23df77c668b645a0123172fbc79322860757b" gracePeriod=30
Sep 10 15:36:37.292299 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:36:37.292277    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-557f876f69-fgtf6"]
Sep 10 15:36:37.323297 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:36:37.323264    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6d4dfc649f-8mm7m"]
Sep 10 15:36:37.328258 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:36:37.328216    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-6d4dfc649f-8mm7m"]
Sep 10 15:36:37.525704 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:36:37.525676    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-59c974bd48-cp6mz"
Sep 10 15:36:37.623177 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:36:37.623077    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-f6t54\" (UniqueName: \"kubernetes.io/projected/39f00b6f-4439-48b7-baf1-9f0b62a7f3a7-kube-api-access-f6t54\") pod \"39f00b6f-4439-48b7-baf1-9f0b62a7f3a7\" (UID: \"39f00b6f-4439-48b7-baf1-9f0b62a7f3a7\") "
Sep 10 15:36:37.625129 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:36:37.625106    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/39f00b6f-4439-48b7-baf1-9f0b62a7f3a7-kube-api-access-f6t54" (OuterVolumeSpecName: "kube-api-access-f6t54") pod "39f00b6f-4439-48b7-baf1-9f0b62a7f3a7" (UID: "39f00b6f-4439-48b7-baf1-9f0b62a7f3a7"). InnerVolumeSpecName "kube-api-access-f6t54". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 15:36:37.724327 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:36:37.724288    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f6t54\" (UniqueName: \"kubernetes.io/projected/39f00b6f-4439-48b7-baf1-9f0b62a7f3a7-kube-api-access-f6t54\") on node \"ip-10-0-128-241.ec2.internal\" DevicePath \"\""
Sep 10 15:36:38.246155 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:36:38.246117    2567 generic.go:356] "Generic (PLEG): container finished" podID="39f00b6f-4439-48b7-baf1-9f0b62a7f3a7" containerID="1043a499e77319acd1bdb49f9ad23df77c668b645a0123172fbc79322860757b" exitCode=0
Sep 10 15:36:38.246629 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:36:38.246162    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-59c974bd48-cp6mz"
Sep 10 15:36:38.246629 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:36:38.246197    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-59c974bd48-cp6mz" event={"ID":"39f00b6f-4439-48b7-baf1-9f0b62a7f3a7","Type":"ContainerDied","Data":"1043a499e77319acd1bdb49f9ad23df77c668b645a0123172fbc79322860757b"}
Sep 10 15:36:38.246629 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:36:38.246249    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-59c974bd48-cp6mz" event={"ID":"39f00b6f-4439-48b7-baf1-9f0b62a7f3a7","Type":"ContainerDied","Data":"b4433b56088b201b4a54c43f5ecb73b0a564716a998830438a5aeab76d81eee7"}
Sep 10 15:36:38.246629 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:36:38.246268    2567 scope.go:117] "RemoveContainer" containerID="1043a499e77319acd1bdb49f9ad23df77c668b645a0123172fbc79322860757b"
Sep 10 15:36:38.255256 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:36:38.255222    2567 scope.go:117] "RemoveContainer" containerID="1043a499e77319acd1bdb49f9ad23df77c668b645a0123172fbc79322860757b"
Sep 10 15:36:38.255545 ip-10-0-128-241 kubenswrapper[2567]: E0910 15:36:38.255520    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1043a499e77319acd1bdb49f9ad23df77c668b645a0123172fbc79322860757b\": container with ID starting with 1043a499e77319acd1bdb49f9ad23df77c668b645a0123172fbc79322860757b not found: ID does not exist" containerID="1043a499e77319acd1bdb49f9ad23df77c668b645a0123172fbc79322860757b"
Sep 10 15:36:38.255643 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:36:38.255552    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1043a499e77319acd1bdb49f9ad23df77c668b645a0123172fbc79322860757b"} err="failed to get container status \"1043a499e77319acd1bdb49f9ad23df77c668b645a0123172fbc79322860757b\": rpc error: code = NotFound desc = could not find container \"1043a499e77319acd1bdb49f9ad23df77c668b645a0123172fbc79322860757b\": container with ID starting with 1043a499e77319acd1bdb49f9ad23df77c668b645a0123172fbc79322860757b not found: ID does not exist"
Sep 10 15:36:38.272820 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:36:38.269913    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-59c974bd48-cp6mz"]
Sep 10 15:36:38.275582 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:36:38.275525    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-59c974bd48-cp6mz"]
Sep 10 15:36:38.903318 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:36:38.903289    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="39f00b6f-4439-48b7-baf1-9f0b62a7f3a7" path="/var/lib/kubelet/pods/39f00b6f-4439-48b7-baf1-9f0b62a7f3a7/volumes"
Sep 10 15:36:38.903619 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:36:38.903606    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="587db4ae-5028-4552-80c2-df2b05c5986c" path="/var/lib/kubelet/pods/587db4ae-5028-4552-80c2-df2b05c5986c/volumes"
Sep 10 15:36:38.903824 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:36:38.903813    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="73935e69-7aed-4169-92a2-0fb051fc94e4" path="/var/lib/kubelet/pods/73935e69-7aed-4169-92a2-0fb051fc94e4/volumes"
Sep 10 15:36:42.145890 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:36:42.145857    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-57c78f46d9-q6rds"]
Sep 10 15:36:42.146318 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:36:42.146180    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="39f00b6f-4439-48b7-baf1-9f0b62a7f3a7" containerName="authorino"
Sep 10 15:36:42.146318 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:36:42.146194    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="39f00b6f-4439-48b7-baf1-9f0b62a7f3a7" containerName="authorino"
Sep 10 15:36:42.146318 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:36:42.146272    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="39f00b6f-4439-48b7-baf1-9f0b62a7f3a7" containerName="authorino"
Sep 10 15:36:42.148895 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:36:42.148872    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-57c78f46d9-q6rds"
Sep 10 15:36:42.152130 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:36:42.152086    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\""
Sep 10 15:36:42.152313 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:36:42.152298    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Sep 10 15:36:42.153189 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:36:42.153169    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-dsv9b\""
Sep 10 15:36:42.153296 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:36:42.153177    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Sep 10 15:36:42.163049 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:36:42.163016    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-57c78f46d9-q6rds"]
Sep 10 15:36:42.261461 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:36:42.261416    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6msqv\" (UniqueName: \"kubernetes.io/projected/f3ec778c-797f-4e19-8aaa-5834ff2f5718-kube-api-access-6msqv\") pod \"maas-controller-57c78f46d9-q6rds\" (UID: \"f3ec778c-797f-4e19-8aaa-5834ff2f5718\") " pod="opendatahub/maas-controller-57c78f46d9-q6rds"
Sep 10 15:36:42.261638 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:36:42.261500    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f3ec778c-797f-4e19-8aaa-5834ff2f5718-webhook-cert\") pod \"maas-controller-57c78f46d9-q6rds\" (UID: \"f3ec778c-797f-4e19-8aaa-5834ff2f5718\") " pod="opendatahub/maas-controller-57c78f46d9-q6rds"
Sep 10 15:36:42.261638 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:36:42.261546    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f3ec778c-797f-4e19-8aaa-5834ff2f5718-metrics-certs\") pod \"maas-controller-57c78f46d9-q6rds\" (UID: \"f3ec778c-797f-4e19-8aaa-5834ff2f5718\") " pod="opendatahub/maas-controller-57c78f46d9-q6rds"
Sep 10 15:36:42.262805 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:36:42.262779    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:36:42.307996 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:36:42.307962    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-68569f6669-754p8"]
Sep 10 15:36:42.310257 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:36:42.310217    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-68569f6669-754p8"
Sep 10 15:36:42.322217 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:36:42.322191    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-68569f6669-754p8"]
Sep 10 15:36:42.342032 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:36:42.341997    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:36:42.361941 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:36:42.361890    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f3ec778c-797f-4e19-8aaa-5834ff2f5718-webhook-cert\") pod \"maas-controller-57c78f46d9-q6rds\" (UID: \"f3ec778c-797f-4e19-8aaa-5834ff2f5718\") " pod="opendatahub/maas-controller-57c78f46d9-q6rds"
Sep 10 15:36:42.362152 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:36:42.361948    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0181721a-e1e2-4991-8970-8073aebda063-metrics-certs\") pod \"maas-controller-68569f6669-754p8\" (UID: \"0181721a-e1e2-4991-8970-8073aebda063\") " pod="opendatahub/maas-controller-68569f6669-754p8"
Sep 10 15:36:42.362152 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:36:42.361982    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f3ec778c-797f-4e19-8aaa-5834ff2f5718-metrics-certs\") pod \"maas-controller-57c78f46d9-q6rds\" (UID: \"f3ec778c-797f-4e19-8aaa-5834ff2f5718\") " pod="opendatahub/maas-controller-57c78f46d9-q6rds"
Sep 10 15:36:42.362152 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:36:42.361999    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b6wbq\" (UniqueName: \"kubernetes.io/projected/0181721a-e1e2-4991-8970-8073aebda063-kube-api-access-b6wbq\") pod \"maas-controller-68569f6669-754p8\" (UID: \"0181721a-e1e2-4991-8970-8073aebda063\") " pod="opendatahub/maas-controller-68569f6669-754p8"
Sep 10 15:36:42.362152 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:36:42.362055    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6msqv\" (UniqueName: \"kubernetes.io/projected/f3ec778c-797f-4e19-8aaa-5834ff2f5718-kube-api-access-6msqv\") pod \"maas-controller-57c78f46d9-q6rds\" (UID: \"f3ec778c-797f-4e19-8aaa-5834ff2f5718\") " pod="opendatahub/maas-controller-57c78f46d9-q6rds"
Sep 10 15:36:42.362152 ip-10-0-128-241 kubenswrapper[2567]: E0910 15:36:42.362145    2567 secret.go:189] Couldn't get secret opendatahub/maas-controller-metrics-tls: secret "maas-controller-metrics-tls" not found
Sep 10 15:36:42.362468 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:36:42.362161    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/0181721a-e1e2-4991-8970-8073aebda063-webhook-cert\") pod \"maas-controller-68569f6669-754p8\" (UID: \"0181721a-e1e2-4991-8970-8073aebda063\") " pod="opendatahub/maas-controller-68569f6669-754p8"
Sep 10 15:36:42.362468 ip-10-0-128-241 kubenswrapper[2567]: E0910 15:36:42.362210    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f3ec778c-797f-4e19-8aaa-5834ff2f5718-metrics-certs podName:f3ec778c-797f-4e19-8aaa-5834ff2f5718 nodeName:}" failed. No retries permitted until 2026-09-10 15:36:42.86219082 +0000 UTC m=+588.430422544 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f3ec778c-797f-4e19-8aaa-5834ff2f5718-metrics-certs") pod "maas-controller-57c78f46d9-q6rds" (UID: "f3ec778c-797f-4e19-8aaa-5834ff2f5718") : secret "maas-controller-metrics-tls" not found
Sep 10 15:36:42.364525 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:36:42.364495    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f3ec778c-797f-4e19-8aaa-5834ff2f5718-webhook-cert\") pod \"maas-controller-57c78f46d9-q6rds\" (UID: \"f3ec778c-797f-4e19-8aaa-5834ff2f5718\") " pod="opendatahub/maas-controller-57c78f46d9-q6rds"
Sep 10 15:36:42.371711 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:36:42.371681    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6msqv\" (UniqueName: \"kubernetes.io/projected/f3ec778c-797f-4e19-8aaa-5834ff2f5718-kube-api-access-6msqv\") pod \"maas-controller-57c78f46d9-q6rds\" (UID: \"f3ec778c-797f-4e19-8aaa-5834ff2f5718\") " pod="opendatahub/maas-controller-57c78f46d9-q6rds"
Sep 10 15:36:42.463295 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:36:42.463226    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/0181721a-e1e2-4991-8970-8073aebda063-webhook-cert\") pod \"maas-controller-68569f6669-754p8\" (UID: \"0181721a-e1e2-4991-8970-8073aebda063\") " pod="opendatahub/maas-controller-68569f6669-754p8"
Sep 10 15:36:42.463472 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:36:42.463318    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0181721a-e1e2-4991-8970-8073aebda063-metrics-certs\") pod \"maas-controller-68569f6669-754p8\" (UID: \"0181721a-e1e2-4991-8970-8073aebda063\") " pod="opendatahub/maas-controller-68569f6669-754p8"
Sep 10 15:36:42.463472 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:36:42.463358    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b6wbq\" (UniqueName: \"kubernetes.io/projected/0181721a-e1e2-4991-8970-8073aebda063-kube-api-access-b6wbq\") pod \"maas-controller-68569f6669-754p8\" (UID: \"0181721a-e1e2-4991-8970-8073aebda063\") " pod="opendatahub/maas-controller-68569f6669-754p8"
Sep 10 15:36:42.465698 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:36:42.465665    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0181721a-e1e2-4991-8970-8073aebda063-metrics-certs\") pod \"maas-controller-68569f6669-754p8\" (UID: \"0181721a-e1e2-4991-8970-8073aebda063\") " pod="opendatahub/maas-controller-68569f6669-754p8"
Sep 10 15:36:42.465811 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:36:42.465699    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/0181721a-e1e2-4991-8970-8073aebda063-webhook-cert\") pod \"maas-controller-68569f6669-754p8\" (UID: \"0181721a-e1e2-4991-8970-8073aebda063\") " pod="opendatahub/maas-controller-68569f6669-754p8"
Sep 10 15:36:42.474475 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:36:42.474450    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b6wbq\" (UniqueName: \"kubernetes.io/projected/0181721a-e1e2-4991-8970-8073aebda063-kube-api-access-b6wbq\") pod \"maas-controller-68569f6669-754p8\" (UID: \"0181721a-e1e2-4991-8970-8073aebda063\") " pod="opendatahub/maas-controller-68569f6669-754p8"
Sep 10 15:36:42.492175 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:36:42.492145    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:36:42.620635 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:36:42.620601    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-68569f6669-754p8"
Sep 10 15:36:42.866961 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:36:42.866865    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f3ec778c-797f-4e19-8aaa-5834ff2f5718-metrics-certs\") pod \"maas-controller-57c78f46d9-q6rds\" (UID: \"f3ec778c-797f-4e19-8aaa-5834ff2f5718\") " pod="opendatahub/maas-controller-57c78f46d9-q6rds"
Sep 10 15:36:42.869336 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:36:42.869312    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f3ec778c-797f-4e19-8aaa-5834ff2f5718-metrics-certs\") pod \"maas-controller-57c78f46d9-q6rds\" (UID: \"f3ec778c-797f-4e19-8aaa-5834ff2f5718\") " pod="opendatahub/maas-controller-57c78f46d9-q6rds"
Sep 10 15:36:42.890164 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:36:42.890136    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-68569f6669-754p8"]
Sep 10 15:36:42.892381 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:36:42.892349    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0181721a_e1e2_4991_8970_8073aebda063.slice/crio-6f50448d35d6e03baec64ff8911265e5f473372242f18e9c372ce92725533bf1 WatchSource:0}: Error finding container 6f50448d35d6e03baec64ff8911265e5f473372242f18e9c372ce92725533bf1: Status 404 returned error can't find the container with id 6f50448d35d6e03baec64ff8911265e5f473372242f18e9c372ce92725533bf1
Sep 10 15:36:43.060483 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:36:43.060441    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-57c78f46d9-q6rds"
Sep 10 15:36:43.269340 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:36:43.269306    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-68569f6669-754p8" event={"ID":"0181721a-e1e2-4991-8970-8073aebda063","Type":"ContainerStarted","Data":"6f50448d35d6e03baec64ff8911265e5f473372242f18e9c372ce92725533bf1"}
Sep 10 15:36:43.330340 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:36:43.330312    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-57c78f46d9-q6rds"]
Sep 10 15:36:43.333192 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:36:43.333157    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf3ec778c_797f_4e19_8aaa_5834ff2f5718.slice/crio-a5d710611a9b20f594246b0c8af12a66ff82e92cbab75277e420ae2d34da41fb WatchSource:0}: Error finding container a5d710611a9b20f594246b0c8af12a66ff82e92cbab75277e420ae2d34da41fb: Status 404 returned error can't find the container with id a5d710611a9b20f594246b0c8af12a66ff82e92cbab75277e420ae2d34da41fb
Sep 10 15:36:44.278403 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:36:44.278306    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-57c78f46d9-q6rds" event={"ID":"f3ec778c-797f-4e19-8aaa-5834ff2f5718","Type":"ContainerStarted","Data":"a5d710611a9b20f594246b0c8af12a66ff82e92cbab75277e420ae2d34da41fb"}
Sep 10 15:36:45.961460 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:36:45.961427    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:36:46.064905 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:36:46.064799    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:36:46.289017 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:36:46.288972    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-68569f6669-754p8" event={"ID":"0181721a-e1e2-4991-8970-8073aebda063","Type":"ContainerStarted","Data":"b6dce6c6bfa1cca606aa45b3058bf43600088ab879ea6f93b005c794f5a94ead"}
Sep 10 15:36:46.289306 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:36:46.289124    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-68569f6669-754p8"
Sep 10 15:36:46.290741 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:36:46.290707    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-57c78f46d9-q6rds" event={"ID":"f3ec778c-797f-4e19-8aaa-5834ff2f5718","Type":"ContainerStarted","Data":"353ce9cea610497c0e5c45e84a7e532a415fb03b547d9a2f003db8ff10c6ddf4"}
Sep 10 15:36:46.290841 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:36:46.290776    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-57c78f46d9-q6rds"
Sep 10 15:36:46.308070 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:36:46.308014    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-68569f6669-754p8" podStartSLOduration=1.409255063 podStartE2EDuration="4.307997279s" totalImagesPullingTime="2.898742216s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 15:36:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 15:36:46.30634009 +0000 UTC m=+591.874571833" watchObservedRunningTime="2026-09-10 15:36:46.307997279 +0000 UTC m=+591.876229024"
Sep 10 15:36:46.327488 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:36:46.327381    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-57c78f46d9-q6rds" podStartSLOduration=1.881579485 podStartE2EDuration="4.32736438s" totalImagesPullingTime="2.445784895s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 15:36:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 15:36:46.325732068 +0000 UTC m=+591.893963798" watchObservedRunningTime="2026-09-10 15:36:46.32736438 +0000 UTC m=+591.895596158"
Sep 10 15:36:54.846441 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:36:54.846413    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-t96pm_0b9d2cda-73e3-4e7c-9e17-1a3d903d95cf/console-operator/2.log"
Sep 10 15:36:54.846963 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:36:54.846478    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-t96pm_0b9d2cda-73e3-4e7c-9e17-1a3d903d95cf/console-operator/2.log"
Sep 10 15:36:57.300296 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:36:57.300261    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-57c78f46d9-q6rds"
Sep 10 15:36:57.300676 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:36:57.300341    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-68569f6669-754p8"
Sep 10 15:36:57.376705 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:36:57.376666    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-57c78f46d9-q6rds"]
Sep 10 15:36:57.376942 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:36:57.376891    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-57c78f46d9-q6rds" podUID="f3ec778c-797f-4e19-8aaa-5834ff2f5718" containerName="manager" containerID="cri-o://353ce9cea610497c0e5c45e84a7e532a415fb03b547d9a2f003db8ff10c6ddf4" gracePeriod=30
Sep 10 15:36:57.647990 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:36:57.647951    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-57c78f46d9-q6rds"
Sep 10 15:36:57.702468 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:36:57.702424    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f3ec778c-797f-4e19-8aaa-5834ff2f5718-metrics-certs\") pod \"f3ec778c-797f-4e19-8aaa-5834ff2f5718\" (UID: \"f3ec778c-797f-4e19-8aaa-5834ff2f5718\") "
Sep 10 15:36:57.702651 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:36:57.702533    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f3ec778c-797f-4e19-8aaa-5834ff2f5718-webhook-cert\") pod \"f3ec778c-797f-4e19-8aaa-5834ff2f5718\" (UID: \"f3ec778c-797f-4e19-8aaa-5834ff2f5718\") "
Sep 10 15:36:57.702651 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:36:57.702565    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6msqv\" (UniqueName: \"kubernetes.io/projected/f3ec778c-797f-4e19-8aaa-5834ff2f5718-kube-api-access-6msqv\") pod \"f3ec778c-797f-4e19-8aaa-5834ff2f5718\" (UID: \"f3ec778c-797f-4e19-8aaa-5834ff2f5718\") "
Sep 10 15:36:57.704869 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:36:57.704829    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f3ec778c-797f-4e19-8aaa-5834ff2f5718-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "f3ec778c-797f-4e19-8aaa-5834ff2f5718" (UID: "f3ec778c-797f-4e19-8aaa-5834ff2f5718"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 15:36:57.704869 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:36:57.704854    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f3ec778c-797f-4e19-8aaa-5834ff2f5718-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "f3ec778c-797f-4e19-8aaa-5834ff2f5718" (UID: "f3ec778c-797f-4e19-8aaa-5834ff2f5718"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 15:36:57.704869 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:36:57.704850    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f3ec778c-797f-4e19-8aaa-5834ff2f5718-kube-api-access-6msqv" (OuterVolumeSpecName: "kube-api-access-6msqv") pod "f3ec778c-797f-4e19-8aaa-5834ff2f5718" (UID: "f3ec778c-797f-4e19-8aaa-5834ff2f5718"). InnerVolumeSpecName "kube-api-access-6msqv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 15:36:57.803182 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:36:57.803139    2567 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f3ec778c-797f-4e19-8aaa-5834ff2f5718-metrics-certs\") on node \"ip-10-0-128-241.ec2.internal\" DevicePath \"\""
Sep 10 15:36:57.803182 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:36:57.803171    2567 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f3ec778c-797f-4e19-8aaa-5834ff2f5718-webhook-cert\") on node \"ip-10-0-128-241.ec2.internal\" DevicePath \"\""
Sep 10 15:36:57.803182 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:36:57.803182    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6msqv\" (UniqueName: \"kubernetes.io/projected/f3ec778c-797f-4e19-8aaa-5834ff2f5718-kube-api-access-6msqv\") on node \"ip-10-0-128-241.ec2.internal\" DevicePath \"\""
Sep 10 15:36:58.346073 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:36:58.346035    2567 generic.go:356] "Generic (PLEG): container finished" podID="f3ec778c-797f-4e19-8aaa-5834ff2f5718" containerID="353ce9cea610497c0e5c45e84a7e532a415fb03b547d9a2f003db8ff10c6ddf4" exitCode=0
Sep 10 15:36:58.346519 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:36:58.346082    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-57c78f46d9-q6rds"
Sep 10 15:36:58.346519 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:36:58.346124    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-57c78f46d9-q6rds" event={"ID":"f3ec778c-797f-4e19-8aaa-5834ff2f5718","Type":"ContainerDied","Data":"353ce9cea610497c0e5c45e84a7e532a415fb03b547d9a2f003db8ff10c6ddf4"}
Sep 10 15:36:58.346519 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:36:58.346180    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-57c78f46d9-q6rds" event={"ID":"f3ec778c-797f-4e19-8aaa-5834ff2f5718","Type":"ContainerDied","Data":"a5d710611a9b20f594246b0c8af12a66ff82e92cbab75277e420ae2d34da41fb"}
Sep 10 15:36:58.346519 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:36:58.346205    2567 scope.go:117] "RemoveContainer" containerID="353ce9cea610497c0e5c45e84a7e532a415fb03b547d9a2f003db8ff10c6ddf4"
Sep 10 15:36:58.354794 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:36:58.354775    2567 scope.go:117] "RemoveContainer" containerID="353ce9cea610497c0e5c45e84a7e532a415fb03b547d9a2f003db8ff10c6ddf4"
Sep 10 15:36:58.355115 ip-10-0-128-241 kubenswrapper[2567]: E0910 15:36:58.355082    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"353ce9cea610497c0e5c45e84a7e532a415fb03b547d9a2f003db8ff10c6ddf4\": container with ID starting with 353ce9cea610497c0e5c45e84a7e532a415fb03b547d9a2f003db8ff10c6ddf4 not found: ID does not exist" containerID="353ce9cea610497c0e5c45e84a7e532a415fb03b547d9a2f003db8ff10c6ddf4"
Sep 10 15:36:58.355188 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:36:58.355120    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"353ce9cea610497c0e5c45e84a7e532a415fb03b547d9a2f003db8ff10c6ddf4"} err="failed to get container status \"353ce9cea610497c0e5c45e84a7e532a415fb03b547d9a2f003db8ff10c6ddf4\": rpc error: code = NotFound desc = could not find container \"353ce9cea610497c0e5c45e84a7e532a415fb03b547d9a2f003db8ff10c6ddf4\": container with ID starting with 353ce9cea610497c0e5c45e84a7e532a415fb03b547d9a2f003db8ff10c6ddf4 not found: ID does not exist"
Sep 10 15:36:58.372515 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:36:58.372484    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-57c78f46d9-q6rds"]
Sep 10 15:36:58.376671 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:36:58.376642    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-57c78f46d9-q6rds"]
Sep 10 15:36:58.904028 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:36:58.903992    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f3ec778c-797f-4e19-8aaa-5834ff2f5718" path="/var/lib/kubelet/pods/f3ec778c-797f-4e19-8aaa-5834ff2f5718/volumes"
Sep 10 15:37:09.091069 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:37:09.091027    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:37:18.647807 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:37:18.647699    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:37:18.746610 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:37:18.746572    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:37:20.068554 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:37:20.068515    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:37:20.290741 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:37:20.290704    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:37:20.358261 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:37:20.358158    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-66f7f9db8f-s6ktb"]
Sep 10 15:37:20.358601 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:37:20.358587    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f3ec778c-797f-4e19-8aaa-5834ff2f5718" containerName="manager"
Sep 10 15:37:20.358646 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:37:20.358604    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="f3ec778c-797f-4e19-8aaa-5834ff2f5718" containerName="manager"
Sep 10 15:37:20.358680 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:37:20.358663    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="f3ec778c-797f-4e19-8aaa-5834ff2f5718" containerName="manager"
Sep 10 15:37:20.364655 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:37:20.364632    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-66f7f9db8f-s6ktb"
Sep 10 15:37:20.368109 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:37:20.368086    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-5qzmv\""
Sep 10 15:37:20.368272 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:37:20.368110    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\""
Sep 10 15:37:20.368272 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:37:20.368121    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Sep 10 15:37:20.372412 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:37:20.372390    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-66f7f9db8f-s6ktb"]
Sep 10 15:37:20.375973 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:37:20.375944    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:37:20.399938 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:37:20.399904    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5cb4993c-6ce8-49d6-9f34-8aa7c46200f3-maas-api-tls\") pod \"maas-api-66f7f9db8f-s6ktb\" (UID: \"5cb4993c-6ce8-49d6-9f34-8aa7c46200f3\") " pod="odh-ai-gateway-infra/maas-api-66f7f9db8f-s6ktb"
Sep 10 15:37:20.400107 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:37:20.399954    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7x9gh\" (UniqueName: \"kubernetes.io/projected/5cb4993c-6ce8-49d6-9f34-8aa7c46200f3-kube-api-access-7x9gh\") pod \"maas-api-66f7f9db8f-s6ktb\" (UID: \"5cb4993c-6ce8-49d6-9f34-8aa7c46200f3\") " pod="odh-ai-gateway-infra/maas-api-66f7f9db8f-s6ktb"
Sep 10 15:37:20.400107 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:37:20.400004    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5cb4993c-6ce8-49d6-9f34-8aa7c46200f3-openshift-service-ca\") pod \"maas-api-66f7f9db8f-s6ktb\" (UID: \"5cb4993c-6ce8-49d6-9f34-8aa7c46200f3\") " pod="odh-ai-gateway-infra/maas-api-66f7f9db8f-s6ktb"
Sep 10 15:37:20.400107 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:37:20.400077    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5cb4993c-6ce8-49d6-9f34-8aa7c46200f3-metrics-certs\") pod \"maas-api-66f7f9db8f-s6ktb\" (UID: \"5cb4993c-6ce8-49d6-9f34-8aa7c46200f3\") " pod="odh-ai-gateway-infra/maas-api-66f7f9db8f-s6ktb"
Sep 10 15:37:20.447251 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:37:20.447197    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:37:20.500686 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:37:20.500655    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5cb4993c-6ce8-49d6-9f34-8aa7c46200f3-metrics-certs\") pod \"maas-api-66f7f9db8f-s6ktb\" (UID: \"5cb4993c-6ce8-49d6-9f34-8aa7c46200f3\") " pod="odh-ai-gateway-infra/maas-api-66f7f9db8f-s6ktb"
Sep 10 15:37:20.500862 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:37:20.500715    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5cb4993c-6ce8-49d6-9f34-8aa7c46200f3-maas-api-tls\") pod \"maas-api-66f7f9db8f-s6ktb\" (UID: \"5cb4993c-6ce8-49d6-9f34-8aa7c46200f3\") " pod="odh-ai-gateway-infra/maas-api-66f7f9db8f-s6ktb"
Sep 10 15:37:20.500862 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:37:20.500753    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7x9gh\" (UniqueName: \"kubernetes.io/projected/5cb4993c-6ce8-49d6-9f34-8aa7c46200f3-kube-api-access-7x9gh\") pod \"maas-api-66f7f9db8f-s6ktb\" (UID: \"5cb4993c-6ce8-49d6-9f34-8aa7c46200f3\") " pod="odh-ai-gateway-infra/maas-api-66f7f9db8f-s6ktb"
Sep 10 15:37:20.500862 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:37:20.500776    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5cb4993c-6ce8-49d6-9f34-8aa7c46200f3-openshift-service-ca\") pod \"maas-api-66f7f9db8f-s6ktb\" (UID: \"5cb4993c-6ce8-49d6-9f34-8aa7c46200f3\") " pod="odh-ai-gateway-infra/maas-api-66f7f9db8f-s6ktb"
Sep 10 15:37:20.501423 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:37:20.501395    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5cb4993c-6ce8-49d6-9f34-8aa7c46200f3-openshift-service-ca\") pod \"maas-api-66f7f9db8f-s6ktb\" (UID: \"5cb4993c-6ce8-49d6-9f34-8aa7c46200f3\") " pod="odh-ai-gateway-infra/maas-api-66f7f9db8f-s6ktb"
Sep 10 15:37:20.503151 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:37:20.503127    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5cb4993c-6ce8-49d6-9f34-8aa7c46200f3-maas-api-tls\") pod \"maas-api-66f7f9db8f-s6ktb\" (UID: \"5cb4993c-6ce8-49d6-9f34-8aa7c46200f3\") " pod="odh-ai-gateway-infra/maas-api-66f7f9db8f-s6ktb"
Sep 10 15:37:20.503266 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:37:20.503166    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5cb4993c-6ce8-49d6-9f34-8aa7c46200f3-metrics-certs\") pod \"maas-api-66f7f9db8f-s6ktb\" (UID: \"5cb4993c-6ce8-49d6-9f34-8aa7c46200f3\") " pod="odh-ai-gateway-infra/maas-api-66f7f9db8f-s6ktb"
Sep 10 15:37:20.510224 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:37:20.510198    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7x9gh\" (UniqueName: \"kubernetes.io/projected/5cb4993c-6ce8-49d6-9f34-8aa7c46200f3-kube-api-access-7x9gh\") pod \"maas-api-66f7f9db8f-s6ktb\" (UID: \"5cb4993c-6ce8-49d6-9f34-8aa7c46200f3\") " pod="odh-ai-gateway-infra/maas-api-66f7f9db8f-s6ktb"
Sep 10 15:37:20.676338 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:37:20.676300    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-66f7f9db8f-s6ktb"
Sep 10 15:37:20.732814 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:37:20.732540    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:37:20.834962 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:37:20.834925    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:37:20.846604 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:37:20.846565    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-59fbc47bf8-kbxnm"]
Sep 10 15:37:20.850718 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:37:20.850701    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-kbxnm"
Sep 10 15:37:20.853126 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:37:20.853100    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\""
Sep 10 15:37:20.861710 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:37:20.861676    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-59fbc47bf8-kbxnm"]
Sep 10 15:37:20.904469 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:37:20.904431    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tkgt2\" (UniqueName: \"kubernetes.io/projected/66268cd1-5727-4ec6-8832-873f3132d1e2-kube-api-access-tkgt2\") pod \"payload-pre-processing-59fbc47bf8-kbxnm\" (UID: \"66268cd1-5727-4ec6-8832-873f3132d1e2\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-kbxnm"
Sep 10 15:37:20.904840 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:37:20.904810    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/66268cd1-5727-4ec6-8832-873f3132d1e2-plugins-config-volume\") pod \"payload-pre-processing-59fbc47bf8-kbxnm\" (UID: \"66268cd1-5727-4ec6-8832-873f3132d1e2\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-kbxnm"
Sep 10 15:37:20.955082 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:37:20.954985    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-66f7f9db8f-s6ktb"]
Sep 10 15:37:20.958088 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:37:20.958053    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5cb4993c_6ce8_49d6_9f34_8aa7c46200f3.slice/crio-a41ca198eaef2fa9cb4d7e3685092f1a9914cb1cee05748925e3849a794129d8 WatchSource:0}: Error finding container a41ca198eaef2fa9cb4d7e3685092f1a9914cb1cee05748925e3849a794129d8: Status 404 returned error can't find the container with id a41ca198eaef2fa9cb4d7e3685092f1a9914cb1cee05748925e3849a794129d8
Sep 10 15:37:21.005506 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:37:21.005466    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/66268cd1-5727-4ec6-8832-873f3132d1e2-plugins-config-volume\") pod \"payload-pre-processing-59fbc47bf8-kbxnm\" (UID: \"66268cd1-5727-4ec6-8832-873f3132d1e2\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-kbxnm"
Sep 10 15:37:21.005692 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:37:21.005575    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tkgt2\" (UniqueName: \"kubernetes.io/projected/66268cd1-5727-4ec6-8832-873f3132d1e2-kube-api-access-tkgt2\") pod \"payload-pre-processing-59fbc47bf8-kbxnm\" (UID: \"66268cd1-5727-4ec6-8832-873f3132d1e2\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-kbxnm"
Sep 10 15:37:21.006372 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:37:21.006345    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/66268cd1-5727-4ec6-8832-873f3132d1e2-plugins-config-volume\") pod \"payload-pre-processing-59fbc47bf8-kbxnm\" (UID: \"66268cd1-5727-4ec6-8832-873f3132d1e2\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-kbxnm"
Sep 10 15:37:21.016277 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:37:21.016251    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tkgt2\" (UniqueName: \"kubernetes.io/projected/66268cd1-5727-4ec6-8832-873f3132d1e2-kube-api-access-tkgt2\") pod \"payload-pre-processing-59fbc47bf8-kbxnm\" (UID: \"66268cd1-5727-4ec6-8832-873f3132d1e2\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-kbxnm"
Sep 10 15:37:21.164017 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:37:21.163972    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-kbxnm"
Sep 10 15:37:21.440218 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:37:21.440186    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-59fbc47bf8-kbxnm"]
Sep 10 15:37:21.442287 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:37:21.442259    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod66268cd1_5727_4ec6_8832_873f3132d1e2.slice/crio-6b034d5b63aa194db04d91568f389597cd1c5bc23b201f95dcfd887ed8a96245 WatchSource:0}: Error finding container 6b034d5b63aa194db04d91568f389597cd1c5bc23b201f95dcfd887ed8a96245: Status 404 returned error can't find the container with id 6b034d5b63aa194db04d91568f389597cd1c5bc23b201f95dcfd887ed8a96245
Sep 10 15:37:21.444396 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:37:21.444297    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-66f7f9db8f-s6ktb" event={"ID":"5cb4993c-6ce8-49d6-9f34-8aa7c46200f3","Type":"ContainerStarted","Data":"a41ca198eaef2fa9cb4d7e3685092f1a9914cb1cee05748925e3849a794129d8"}
Sep 10 15:37:22.453519 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:37:22.453439    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-kbxnm" event={"ID":"66268cd1-5727-4ec6-8832-873f3132d1e2","Type":"ContainerStarted","Data":"6b034d5b63aa194db04d91568f389597cd1c5bc23b201f95dcfd887ed8a96245"}
Sep 10 15:37:24.463252 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:37:24.463196    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-kbxnm" event={"ID":"66268cd1-5727-4ec6-8832-873f3132d1e2","Type":"ContainerStarted","Data":"9672ece3584f0ee9f50b8ba6c830f00617a340ff979d74bb5b36b1247dd4c19f"}
Sep 10 15:37:24.463729 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:37:24.463295    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-kbxnm"
Sep 10 15:37:24.464719 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:37:24.464698    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-66f7f9db8f-s6ktb" event={"ID":"5cb4993c-6ce8-49d6-9f34-8aa7c46200f3","Type":"ContainerStarted","Data":"850d1cc58c4a3e2213ecdfb075389845c3bf5a17694a06badcf22be5cb9df6e2"}
Sep 10 15:37:24.464845 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:37:24.464828    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-66f7f9db8f-s6ktb"
Sep 10 15:37:24.489830 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:37:24.489778    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-kbxnm" podStartSLOduration=2.509645201 podStartE2EDuration="4.489762501s" totalImagesPullingTime="1.9801173s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 15:37:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 15:37:24.488654899 +0000 UTC m=+630.056886630" watchObservedRunningTime="2026-09-10 15:37:24.489762501 +0000 UTC m=+630.057994230"
Sep 10 15:37:24.506590 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:37:24.506535    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-66f7f9db8f-s6ktb" podStartSLOduration=2.049581096 podStartE2EDuration="4.506517657s" totalImagesPullingTime="2.456936561s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 15:37:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 15:37:24.505607626 +0000 UTC m=+630.073839348" watchObservedRunningTime="2026-09-10 15:37:24.506517657 +0000 UTC m=+630.074749386"
Sep 10 15:37:30.475043 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:37:30.475012    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-66f7f9db8f-s6ktb"
Sep 10 15:37:31.544091 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:37:31.544052    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:37:31.649091 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:37:31.649056    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:37:34.206060 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:37:34.206024    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:37:35.472134 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:37:35.472102    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-kbxnm"
Sep 10 15:37:47.099156 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:37:47.099114    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-vq88d"]
Sep 10 15:37:47.103537 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:37:47.103514    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-vq88d"
Sep 10 15:37:47.106941 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:37:47.106923    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Sep 10 15:37:47.107059 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:37:47.106936    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-vwb6m\""
Sep 10 15:37:47.107930 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:37:47.107905    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\""
Sep 10 15:37:47.108079 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:37:47.107947    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Sep 10 15:37:47.122889 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:37:47.122839    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-vq88d"]
Sep 10 15:37:47.144616 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:37:47.144577    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:37:47.233815 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:37:47.233784    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d61c9899-c724-4ebe-8a03-f90000c3b568-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-vq88d\" (UID: \"d61c9899-c724-4ebe-8a03-f90000c3b568\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-vq88d"
Sep 10 15:37:47.233992 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:37:47.233828    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4z2ns\" (UniqueName: \"kubernetes.io/projected/d61c9899-c724-4ebe-8a03-f90000c3b568-kube-api-access-4z2ns\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-vq88d\" (UID: \"d61c9899-c724-4ebe-8a03-f90000c3b568\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-vq88d"
Sep 10 15:37:47.233992 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:37:47.233925    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d61c9899-c724-4ebe-8a03-f90000c3b568-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-vq88d\" (UID: \"d61c9899-c724-4ebe-8a03-f90000c3b568\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-vq88d"
Sep 10 15:37:47.234064 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:37:47.233990    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d61c9899-c724-4ebe-8a03-f90000c3b568-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-vq88d\" (UID: \"d61c9899-c724-4ebe-8a03-f90000c3b568\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-vq88d"
Sep 10 15:37:47.234064 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:37:47.234014    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d61c9899-c724-4ebe-8a03-f90000c3b568-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-vq88d\" (UID: \"d61c9899-c724-4ebe-8a03-f90000c3b568\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-vq88d"
Sep 10 15:37:47.234064 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:37:47.234036    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d61c9899-c724-4ebe-8a03-f90000c3b568-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-vq88d\" (UID: \"d61c9899-c724-4ebe-8a03-f90000c3b568\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-vq88d"
Sep 10 15:37:47.335196 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:37:47.335154    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d61c9899-c724-4ebe-8a03-f90000c3b568-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-vq88d\" (UID: \"d61c9899-c724-4ebe-8a03-f90000c3b568\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-vq88d"
Sep 10 15:37:47.335419 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:37:47.335216    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d61c9899-c724-4ebe-8a03-f90000c3b568-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-vq88d\" (UID: \"d61c9899-c724-4ebe-8a03-f90000c3b568\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-vq88d"
Sep 10 15:37:47.335419 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:37:47.335261    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d61c9899-c724-4ebe-8a03-f90000c3b568-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-vq88d\" (UID: \"d61c9899-c724-4ebe-8a03-f90000c3b568\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-vq88d"
Sep 10 15:37:47.335419 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:37:47.335287    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d61c9899-c724-4ebe-8a03-f90000c3b568-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-vq88d\" (UID: \"d61c9899-c724-4ebe-8a03-f90000c3b568\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-vq88d"
Sep 10 15:37:47.335419 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:37:47.335316    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d61c9899-c724-4ebe-8a03-f90000c3b568-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-vq88d\" (UID: \"d61c9899-c724-4ebe-8a03-f90000c3b568\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-vq88d"
Sep 10 15:37:47.335419 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:37:47.335359    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4z2ns\" (UniqueName: \"kubernetes.io/projected/d61c9899-c724-4ebe-8a03-f90000c3b568-kube-api-access-4z2ns\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-vq88d\" (UID: \"d61c9899-c724-4ebe-8a03-f90000c3b568\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-vq88d"
Sep 10 15:37:47.335731 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:37:47.335711    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d61c9899-c724-4ebe-8a03-f90000c3b568-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-vq88d\" (UID: \"d61c9899-c724-4ebe-8a03-f90000c3b568\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-vq88d"
Sep 10 15:37:47.335811 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:37:47.335786    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d61c9899-c724-4ebe-8a03-f90000c3b568-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-vq88d\" (UID: \"d61c9899-c724-4ebe-8a03-f90000c3b568\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-vq88d"
Sep 10 15:37:47.335871 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:37:47.335858    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d61c9899-c724-4ebe-8a03-f90000c3b568-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-vq88d\" (UID: \"d61c9899-c724-4ebe-8a03-f90000c3b568\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-vq88d"
Sep 10 15:37:47.337453 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:37:47.337429    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d61c9899-c724-4ebe-8a03-f90000c3b568-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-vq88d\" (UID: \"d61c9899-c724-4ebe-8a03-f90000c3b568\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-vq88d"
Sep 10 15:37:47.337883 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:37:47.337862    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d61c9899-c724-4ebe-8a03-f90000c3b568-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-vq88d\" (UID: \"d61c9899-c724-4ebe-8a03-f90000c3b568\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-vq88d"
Sep 10 15:37:47.347812 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:37:47.347777    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4z2ns\" (UniqueName: \"kubernetes.io/projected/d61c9899-c724-4ebe-8a03-f90000c3b568-kube-api-access-4z2ns\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-vq88d\" (UID: \"d61c9899-c724-4ebe-8a03-f90000c3b568\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-vq88d"
Sep 10 15:37:47.413854 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:37:47.413758    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-vq88d"
Sep 10 15:37:47.691209 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:37:47.691152    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-vq88d"]
Sep 10 15:37:47.694113 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:37:47.694085    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd61c9899_c724_4ebe_8a03_f90000c3b568.slice/crio-46c9e6aa2de6050d66c36c21d18f89cc80c0f53e76487e8cec875f03eefdb87b WatchSource:0}: Error finding container 46c9e6aa2de6050d66c36c21d18f89cc80c0f53e76487e8cec875f03eefdb87b: Status 404 returned error can't find the container with id 46c9e6aa2de6050d66c36c21d18f89cc80c0f53e76487e8cec875f03eefdb87b
Sep 10 15:37:47.695936 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:37:47.695916    2567 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 10 15:37:48.575048 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:37:48.575006    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-vq88d" event={"ID":"d61c9899-c724-4ebe-8a03-f90000c3b568","Type":"ContainerStarted","Data":"46c9e6aa2de6050d66c36c21d18f89cc80c0f53e76487e8cec875f03eefdb87b"}
Sep 10 15:37:49.580712 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:37:49.580673    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-vq88d" event={"ID":"d61c9899-c724-4ebe-8a03-f90000c3b568","Type":"ContainerStarted","Data":"95c7daae5191c8e1ec4dc568a7e1e528deda01bdac2d362cecd8e1770d05e768"}
Sep 10 15:37:49.601290 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:37:49.601217    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-vq88d" podStartSLOduration=1.447506833 podStartE2EDuration="2.60119655s" totalImagesPullingTime="1.153689717s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 15:37:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 15:37:49.600965952 +0000 UTC m=+655.169197685" watchObservedRunningTime="2026-09-10 15:37:49.60119655 +0000 UTC m=+655.169428280"
Sep 10 15:37:57.414720 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:37:57.414671    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-vq88d"
Sep 10 15:37:57.415452 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:37:57.414783    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-vq88d"
Sep 10 15:37:57.427415 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:37:57.427384    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-vq88d"
Sep 10 15:37:57.626177 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:37:57.626150    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-vq88d"
Sep 10 15:38:20.908628 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:38:20.908596    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-nxp96"]
Sep 10 15:38:20.911855 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:38:20.911835    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-nxp96"
Sep 10 15:38:20.914424 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:38:20.914399    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\""
Sep 10 15:38:20.921353 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:38:20.921325    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-nxp96"]
Sep 10 15:38:20.936928 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:38:20.936841    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:38:21.018605 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:38:21.018562    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/01606484-2402-43a3-98de-8bff3c7cdbc9-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-nxp96\" (UID: \"01606484-2402-43a3-98de-8bff3c7cdbc9\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-nxp96"
Sep 10 15:38:21.018605 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:38:21.018601    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/01606484-2402-43a3-98de-8bff3c7cdbc9-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-nxp96\" (UID: \"01606484-2402-43a3-98de-8bff3c7cdbc9\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-nxp96"
Sep 10 15:38:21.018820 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:38:21.018676    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/01606484-2402-43a3-98de-8bff3c7cdbc9-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-nxp96\" (UID: \"01606484-2402-43a3-98de-8bff3c7cdbc9\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-nxp96"
Sep 10 15:38:21.018820 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:38:21.018726    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/01606484-2402-43a3-98de-8bff3c7cdbc9-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-nxp96\" (UID: \"01606484-2402-43a3-98de-8bff3c7cdbc9\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-nxp96"
Sep 10 15:38:21.018820 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:38:21.018744    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/01606484-2402-43a3-98de-8bff3c7cdbc9-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-nxp96\" (UID: \"01606484-2402-43a3-98de-8bff3c7cdbc9\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-nxp96"
Sep 10 15:38:21.018820 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:38:21.018760    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-86zht\" (UniqueName: \"kubernetes.io/projected/01606484-2402-43a3-98de-8bff3c7cdbc9-kube-api-access-86zht\") pod \"e2e-distinct-simulated-kserve-59975d69f9-nxp96\" (UID: \"01606484-2402-43a3-98de-8bff3c7cdbc9\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-nxp96"
Sep 10 15:38:21.120284 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:38:21.120217    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/01606484-2402-43a3-98de-8bff3c7cdbc9-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-nxp96\" (UID: \"01606484-2402-43a3-98de-8bff3c7cdbc9\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-nxp96"
Sep 10 15:38:21.120284 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:38:21.120282    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/01606484-2402-43a3-98de-8bff3c7cdbc9-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-nxp96\" (UID: \"01606484-2402-43a3-98de-8bff3c7cdbc9\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-nxp96"
Sep 10 15:38:21.120548 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:38:21.120323    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/01606484-2402-43a3-98de-8bff3c7cdbc9-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-nxp96\" (UID: \"01606484-2402-43a3-98de-8bff3c7cdbc9\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-nxp96"
Sep 10 15:38:21.120548 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:38:21.120348    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/01606484-2402-43a3-98de-8bff3c7cdbc9-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-nxp96\" (UID: \"01606484-2402-43a3-98de-8bff3c7cdbc9\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-nxp96"
Sep 10 15:38:21.120548 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:38:21.120364    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/01606484-2402-43a3-98de-8bff3c7cdbc9-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-nxp96\" (UID: \"01606484-2402-43a3-98de-8bff3c7cdbc9\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-nxp96"
Sep 10 15:38:21.120548 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:38:21.120379    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-86zht\" (UniqueName: \"kubernetes.io/projected/01606484-2402-43a3-98de-8bff3c7cdbc9-kube-api-access-86zht\") pod \"e2e-distinct-simulated-kserve-59975d69f9-nxp96\" (UID: \"01606484-2402-43a3-98de-8bff3c7cdbc9\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-nxp96"
Sep 10 15:38:21.120759 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:38:21.120724    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/01606484-2402-43a3-98de-8bff3c7cdbc9-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-nxp96\" (UID: \"01606484-2402-43a3-98de-8bff3c7cdbc9\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-nxp96"
Sep 10 15:38:21.120808 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:38:21.120757    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/01606484-2402-43a3-98de-8bff3c7cdbc9-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-nxp96\" (UID: \"01606484-2402-43a3-98de-8bff3c7cdbc9\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-nxp96"
Sep 10 15:38:21.120870 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:38:21.120846    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/01606484-2402-43a3-98de-8bff3c7cdbc9-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-nxp96\" (UID: \"01606484-2402-43a3-98de-8bff3c7cdbc9\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-nxp96"
Sep 10 15:38:21.122731 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:38:21.122710    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/01606484-2402-43a3-98de-8bff3c7cdbc9-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-nxp96\" (UID: \"01606484-2402-43a3-98de-8bff3c7cdbc9\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-nxp96"
Sep 10 15:38:21.123008 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:38:21.122992    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/01606484-2402-43a3-98de-8bff3c7cdbc9-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-nxp96\" (UID: \"01606484-2402-43a3-98de-8bff3c7cdbc9\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-nxp96"
Sep 10 15:38:21.129676 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:38:21.129653    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-86zht\" (UniqueName: \"kubernetes.io/projected/01606484-2402-43a3-98de-8bff3c7cdbc9-kube-api-access-86zht\") pod \"e2e-distinct-simulated-kserve-59975d69f9-nxp96\" (UID: \"01606484-2402-43a3-98de-8bff3c7cdbc9\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-nxp96"
Sep 10 15:38:21.223755 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:38:21.223712    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-nxp96"
Sep 10 15:38:21.494701 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:38:21.494674    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-nxp96"]
Sep 10 15:38:21.496803 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:38:21.496764    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod01606484_2402_43a3_98de_8bff3c7cdbc9.slice/crio-5eb1afbef182585f614be2720eb1531e4dac2c40fb0aa8aecb5a4d8869f19d35 WatchSource:0}: Error finding container 5eb1afbef182585f614be2720eb1531e4dac2c40fb0aa8aecb5a4d8869f19d35: Status 404 returned error can't find the container with id 5eb1afbef182585f614be2720eb1531e4dac2c40fb0aa8aecb5a4d8869f19d35
Sep 10 15:38:21.722365 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:38:21.722327    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-nxp96" event={"ID":"01606484-2402-43a3-98de-8bff3c7cdbc9","Type":"ContainerStarted","Data":"5eb1afbef182585f614be2720eb1531e4dac2c40fb0aa8aecb5a4d8869f19d35"}
Sep 10 15:38:22.728306 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:38:22.728269    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-nxp96" event={"ID":"01606484-2402-43a3-98de-8bff3c7cdbc9","Type":"ContainerStarted","Data":"3037c2cd5a50db325d41692ec8dc4a8621f8e1164e259915c73798e60e6baf5c"}
Sep 10 15:38:22.747553 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:38:22.747498    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-nxp96" podStartSLOduration=2.547672378 podStartE2EDuration="2.747479215s" totalImagesPullingTime="199.806837ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 15:38:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 15:38:22.745755983 +0000 UTC m=+688.313987717" watchObservedRunningTime="2026-09-10 15:38:22.747479215 +0000 UTC m=+688.315710944"
Sep 10 15:38:25.741271 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:38:25.741220    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:38:31.224567 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:38:31.224527    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-nxp96"
Sep 10 15:38:31.225035 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:38:31.224591    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-nxp96"
Sep 10 15:38:31.236891 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:38:31.236860    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-nxp96"
Sep 10 15:38:31.781632 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:38:31.781596    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-nxp96"
Sep 10 15:38:38.904994 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:38:38.904961    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-b6999"]
Sep 10 15:38:38.909979 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:38:38.909957    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-b6999"
Sep 10 15:38:38.917414 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:38:38.917019    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\""
Sep 10 15:38:38.917611 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:38:38.917584    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-b6999"]
Sep 10 15:38:38.931205 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:38:38.931179    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:38:38.979038 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:38:38.978999    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/623a26a6-a6e2-4939-ac5c-a6e847393d10-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-b6999\" (UID: \"623a26a6-a6e2-4939-ac5c-a6e847393d10\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-b6999"
Sep 10 15:38:38.979226 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:38:38.979119    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/623a26a6-a6e2-4939-ac5c-a6e847393d10-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-b6999\" (UID: \"623a26a6-a6e2-4939-ac5c-a6e847393d10\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-b6999"
Sep 10 15:38:38.979226 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:38:38.979156    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/623a26a6-a6e2-4939-ac5c-a6e847393d10-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-b6999\" (UID: \"623a26a6-a6e2-4939-ac5c-a6e847393d10\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-b6999"
Sep 10 15:38:38.979226 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:38:38.979218    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/623a26a6-a6e2-4939-ac5c-a6e847393d10-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-b6999\" (UID: \"623a26a6-a6e2-4939-ac5c-a6e847393d10\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-b6999"
Sep 10 15:38:38.979401 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:38:38.979268    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/623a26a6-a6e2-4939-ac5c-a6e847393d10-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-b6999\" (UID: \"623a26a6-a6e2-4939-ac5c-a6e847393d10\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-b6999"
Sep 10 15:38:38.979401 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:38:38.979323    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t8b6z\" (UniqueName: \"kubernetes.io/projected/623a26a6-a6e2-4939-ac5c-a6e847393d10-kube-api-access-t8b6z\") pod \"e2e-trlp-test-simulated-kserve-598466548d-b6999\" (UID: \"623a26a6-a6e2-4939-ac5c-a6e847393d10\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-b6999"
Sep 10 15:38:39.079857 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:38:39.079827    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/623a26a6-a6e2-4939-ac5c-a6e847393d10-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-b6999\" (UID: \"623a26a6-a6e2-4939-ac5c-a6e847393d10\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-b6999"
Sep 10 15:38:39.079857 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:38:39.079860    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/623a26a6-a6e2-4939-ac5c-a6e847393d10-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-b6999\" (UID: \"623a26a6-a6e2-4939-ac5c-a6e847393d10\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-b6999"
Sep 10 15:38:39.080109 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:38:39.079894    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/623a26a6-a6e2-4939-ac5c-a6e847393d10-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-b6999\" (UID: \"623a26a6-a6e2-4939-ac5c-a6e847393d10\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-b6999"
Sep 10 15:38:39.080109 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:38:39.079915    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/623a26a6-a6e2-4939-ac5c-a6e847393d10-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-b6999\" (UID: \"623a26a6-a6e2-4939-ac5c-a6e847393d10\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-b6999"
Sep 10 15:38:39.080109 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:38:39.079954    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t8b6z\" (UniqueName: \"kubernetes.io/projected/623a26a6-a6e2-4939-ac5c-a6e847393d10-kube-api-access-t8b6z\") pod \"e2e-trlp-test-simulated-kserve-598466548d-b6999\" (UID: \"623a26a6-a6e2-4939-ac5c-a6e847393d10\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-b6999"
Sep 10 15:38:39.080109 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:38:39.080000    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/623a26a6-a6e2-4939-ac5c-a6e847393d10-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-b6999\" (UID: \"623a26a6-a6e2-4939-ac5c-a6e847393d10\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-b6999"
Sep 10 15:38:39.080409 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:38:39.080330    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/623a26a6-a6e2-4939-ac5c-a6e847393d10-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-b6999\" (UID: \"623a26a6-a6e2-4939-ac5c-a6e847393d10\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-b6999"
Sep 10 15:38:39.080471 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:38:39.080421    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/623a26a6-a6e2-4939-ac5c-a6e847393d10-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-b6999\" (UID: \"623a26a6-a6e2-4939-ac5c-a6e847393d10\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-b6999"
Sep 10 15:38:39.080629 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:38:39.080601    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/623a26a6-a6e2-4939-ac5c-a6e847393d10-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-b6999\" (UID: \"623a26a6-a6e2-4939-ac5c-a6e847393d10\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-b6999"
Sep 10 15:38:39.082424 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:38:39.082398    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/623a26a6-a6e2-4939-ac5c-a6e847393d10-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-b6999\" (UID: \"623a26a6-a6e2-4939-ac5c-a6e847393d10\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-b6999"
Sep 10 15:38:39.082544 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:38:39.082482    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/623a26a6-a6e2-4939-ac5c-a6e847393d10-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-b6999\" (UID: \"623a26a6-a6e2-4939-ac5c-a6e847393d10\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-b6999"
Sep 10 15:38:39.089285 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:38:39.089263    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t8b6z\" (UniqueName: \"kubernetes.io/projected/623a26a6-a6e2-4939-ac5c-a6e847393d10-kube-api-access-t8b6z\") pod \"e2e-trlp-test-simulated-kserve-598466548d-b6999\" (UID: \"623a26a6-a6e2-4939-ac5c-a6e847393d10\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-b6999"
Sep 10 15:38:39.221775 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:38:39.221748    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-b6999"
Sep 10 15:38:39.493605 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:38:39.493577    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-b6999"]
Sep 10 15:38:39.496061 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:38:39.496033    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod623a26a6_a6e2_4939_ac5c_a6e847393d10.slice/crio-3711e1f502583a29ff82ca98d785c641cf10b14324889fcdd89e280d39e3363a WatchSource:0}: Error finding container 3711e1f502583a29ff82ca98d785c641cf10b14324889fcdd89e280d39e3363a: Status 404 returned error can't find the container with id 3711e1f502583a29ff82ca98d785c641cf10b14324889fcdd89e280d39e3363a
Sep 10 15:38:39.807324 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:38:39.807294    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-b6999" event={"ID":"623a26a6-a6e2-4939-ac5c-a6e847393d10","Type":"ContainerStarted","Data":"3711e1f502583a29ff82ca98d785c641cf10b14324889fcdd89e280d39e3363a"}
Sep 10 15:38:40.813962 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:38:40.813928    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-b6999" event={"ID":"623a26a6-a6e2-4939-ac5c-a6e847393d10","Type":"ContainerStarted","Data":"7405f5d29ca4d82a8ef29419fa4cfb10a9a0a757db1da16bcf745555b1834199"}
Sep 10 15:38:40.841622 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:38:40.841567    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-b6999" podStartSLOduration=2.54771568 podStartE2EDuration="2.841548652s" totalImagesPullingTime="293.832972ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 15:38:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 15:38:40.840163594 +0000 UTC m=+706.408395325" watchObservedRunningTime="2026-09-10 15:38:40.841548652 +0000 UTC m=+706.409780382"
Sep 10 15:38:49.221946 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:38:49.221854    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-b6999"
Sep 10 15:38:49.221946 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:38:49.221933    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-b6999"
Sep 10 15:38:49.234548 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:38:49.234521    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-b6999"
Sep 10 15:38:49.864948 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:38:49.864918    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-b6999"
Sep 10 15:38:52.302432 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:38:52.302394    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:39:11.267525 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:39:11.267486    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:39:20.332061 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:39:20.332022    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:39:22.750452 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:39:22.750420    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:40:53.118321 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:40:53.118162    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:40:53.204478 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:40:53.204441    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:41:08.745274 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:41:08.745225    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-557cbbbb8c-hn8vs"]
Sep 10 15:41:08.748689 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:41:08.748671    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-557cbbbb8c-hn8vs"
Sep 10 15:41:08.763680 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:41:08.763646    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-557cbbbb8c-hn8vs"]
Sep 10 15:41:08.795810 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:41:08.795775    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/82379701-1582-41e4-b444-129c64f9ee97-tls-cert\") pod \"authorino-557cbbbb8c-hn8vs\" (UID: \"82379701-1582-41e4-b444-129c64f9ee97\") " pod="kuadrant-system/authorino-557cbbbb8c-hn8vs"
Sep 10 15:41:08.795985 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:41:08.795834    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vhv99\" (UniqueName: \"kubernetes.io/projected/82379701-1582-41e4-b444-129c64f9ee97-kube-api-access-vhv99\") pod \"authorino-557cbbbb8c-hn8vs\" (UID: \"82379701-1582-41e4-b444-129c64f9ee97\") " pod="kuadrant-system/authorino-557cbbbb8c-hn8vs"
Sep 10 15:41:08.897132 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:41:08.897093    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vhv99\" (UniqueName: \"kubernetes.io/projected/82379701-1582-41e4-b444-129c64f9ee97-kube-api-access-vhv99\") pod \"authorino-557cbbbb8c-hn8vs\" (UID: \"82379701-1582-41e4-b444-129c64f9ee97\") " pod="kuadrant-system/authorino-557cbbbb8c-hn8vs"
Sep 10 15:41:08.897371 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:41:08.897181    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/82379701-1582-41e4-b444-129c64f9ee97-tls-cert\") pod \"authorino-557cbbbb8c-hn8vs\" (UID: \"82379701-1582-41e4-b444-129c64f9ee97\") " pod="kuadrant-system/authorino-557cbbbb8c-hn8vs"
Sep 10 15:41:08.899627 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:41:08.899601    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/82379701-1582-41e4-b444-129c64f9ee97-tls-cert\") pod \"authorino-557cbbbb8c-hn8vs\" (UID: \"82379701-1582-41e4-b444-129c64f9ee97\") " pod="kuadrant-system/authorino-557cbbbb8c-hn8vs"
Sep 10 15:41:08.905851 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:41:08.905828    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vhv99\" (UniqueName: \"kubernetes.io/projected/82379701-1582-41e4-b444-129c64f9ee97-kube-api-access-vhv99\") pod \"authorino-557cbbbb8c-hn8vs\" (UID: \"82379701-1582-41e4-b444-129c64f9ee97\") " pod="kuadrant-system/authorino-557cbbbb8c-hn8vs"
Sep 10 15:41:09.056977 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:41:09.056870    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-557cbbbb8c-hn8vs"
Sep 10 15:41:09.325146 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:41:09.324974    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-557cbbbb8c-hn8vs"]
Sep 10 15:41:09.327779 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:41:09.327749    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod82379701_1582_41e4_b444_129c64f9ee97.slice/crio-ab4e24e1abd1d7ceb2321dc9daac7cd53e02c41eeed9aa7cd2fb7d542bfb32c2 WatchSource:0}: Error finding container ab4e24e1abd1d7ceb2321dc9daac7cd53e02c41eeed9aa7cd2fb7d542bfb32c2: Status 404 returned error can't find the container with id ab4e24e1abd1d7ceb2321dc9daac7cd53e02c41eeed9aa7cd2fb7d542bfb32c2
Sep 10 15:41:09.471252 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:41:09.471184    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-557cbbbb8c-hn8vs" event={"ID":"82379701-1582-41e4-b444-129c64f9ee97","Type":"ContainerStarted","Data":"ab4e24e1abd1d7ceb2321dc9daac7cd53e02c41eeed9aa7cd2fb7d542bfb32c2"}
Sep 10 15:41:10.479898 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:41:10.479854    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-557cbbbb8c-hn8vs" event={"ID":"82379701-1582-41e4-b444-129c64f9ee97","Type":"ContainerStarted","Data":"271c80ea18afc4dcd717552eff7beac149130d49e7d6c6bbbc9473fd4e833428"}
Sep 10 15:41:10.507957 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:41:10.507873    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-557cbbbb8c-hn8vs" podStartSLOduration=1.9722860949999999 podStartE2EDuration="2.507851741s" totalImagesPullingTime="535.565646ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 15:41:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 15:41:10.505749951 +0000 UTC m=+856.073981683" watchObservedRunningTime="2026-09-10 15:41:10.507851741 +0000 UTC m=+856.076083470"
Sep 10 15:41:10.539972 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:41:10.539910    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-8477bc89d6-vwnd4"]
Sep 10 15:41:10.540542 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:41:10.540506    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-8477bc89d6-vwnd4" podUID="9c9fa106-613a-45bd-b644-d33ea0bee161" containerName="authorino" containerID="cri-o://a06a7b907484aaf5aae97aae8bb873a0411933efa15bfb1fe68da79881220763" gracePeriod=30
Sep 10 15:41:10.805695 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:41:10.805666    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-8477bc89d6-vwnd4"
Sep 10 15:41:10.919492 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:41:10.919459    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/9c9fa106-613a-45bd-b644-d33ea0bee161-tls-cert\") pod \"9c9fa106-613a-45bd-b644-d33ea0bee161\" (UID: \"9c9fa106-613a-45bd-b644-d33ea0bee161\") "
Sep 10 15:41:10.919691 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:41:10.919519    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9jsb8\" (UniqueName: \"kubernetes.io/projected/9c9fa106-613a-45bd-b644-d33ea0bee161-kube-api-access-9jsb8\") pod \"9c9fa106-613a-45bd-b644-d33ea0bee161\" (UID: \"9c9fa106-613a-45bd-b644-d33ea0bee161\") "
Sep 10 15:41:10.921680 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:41:10.921650    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9c9fa106-613a-45bd-b644-d33ea0bee161-kube-api-access-9jsb8" (OuterVolumeSpecName: "kube-api-access-9jsb8") pod "9c9fa106-613a-45bd-b644-d33ea0bee161" (UID: "9c9fa106-613a-45bd-b644-d33ea0bee161"). InnerVolumeSpecName "kube-api-access-9jsb8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 15:41:10.929505 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:41:10.929477    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9c9fa106-613a-45bd-b644-d33ea0bee161-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "9c9fa106-613a-45bd-b644-d33ea0bee161" (UID: "9c9fa106-613a-45bd-b644-d33ea0bee161"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 15:41:11.021127 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:41:11.021089    2567 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/9c9fa106-613a-45bd-b644-d33ea0bee161-tls-cert\") on node \"ip-10-0-128-241.ec2.internal\" DevicePath \"\""
Sep 10 15:41:11.021127 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:41:11.021127    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9jsb8\" (UniqueName: \"kubernetes.io/projected/9c9fa106-613a-45bd-b644-d33ea0bee161-kube-api-access-9jsb8\") on node \"ip-10-0-128-241.ec2.internal\" DevicePath \"\""
Sep 10 15:41:11.105026 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:41:11.104769    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:41:11.238428 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:41:11.238392    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:41:11.486539 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:41:11.486502    2567 generic.go:356] "Generic (PLEG): container finished" podID="9c9fa106-613a-45bd-b644-d33ea0bee161" containerID="a06a7b907484aaf5aae97aae8bb873a0411933efa15bfb1fe68da79881220763" exitCode=0
Sep 10 15:41:11.486956 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:41:11.486539    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-8477bc89d6-vwnd4" event={"ID":"9c9fa106-613a-45bd-b644-d33ea0bee161","Type":"ContainerDied","Data":"a06a7b907484aaf5aae97aae8bb873a0411933efa15bfb1fe68da79881220763"}
Sep 10 15:41:11.486956 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:41:11.486582    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-8477bc89d6-vwnd4" event={"ID":"9c9fa106-613a-45bd-b644-d33ea0bee161","Type":"ContainerDied","Data":"3bda675018526d0c2f92efa473eb27d86360bbf71e624444e5709988c119c01f"}
Sep 10 15:41:11.486956 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:41:11.486582    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-8477bc89d6-vwnd4"
Sep 10 15:41:11.486956 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:41:11.486636    2567 scope.go:117] "RemoveContainer" containerID="a06a7b907484aaf5aae97aae8bb873a0411933efa15bfb1fe68da79881220763"
Sep 10 15:41:11.495693 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:41:11.495665    2567 scope.go:117] "RemoveContainer" containerID="a06a7b907484aaf5aae97aae8bb873a0411933efa15bfb1fe68da79881220763"
Sep 10 15:41:11.496001 ip-10-0-128-241 kubenswrapper[2567]: E0910 15:41:11.495980    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a06a7b907484aaf5aae97aae8bb873a0411933efa15bfb1fe68da79881220763\": container with ID starting with a06a7b907484aaf5aae97aae8bb873a0411933efa15bfb1fe68da79881220763 not found: ID does not exist" containerID="a06a7b907484aaf5aae97aae8bb873a0411933efa15bfb1fe68da79881220763"
Sep 10 15:41:11.496072 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:41:11.496011    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a06a7b907484aaf5aae97aae8bb873a0411933efa15bfb1fe68da79881220763"} err="failed to get container status \"a06a7b907484aaf5aae97aae8bb873a0411933efa15bfb1fe68da79881220763\": rpc error: code = NotFound desc = could not find container \"a06a7b907484aaf5aae97aae8bb873a0411933efa15bfb1fe68da79881220763\": container with ID starting with a06a7b907484aaf5aae97aae8bb873a0411933efa15bfb1fe68da79881220763 not found: ID does not exist"
Sep 10 15:41:11.511918 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:41:11.511881    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-8477bc89d6-vwnd4"]
Sep 10 15:41:11.516737 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:41:11.516711    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-8477bc89d6-vwnd4"]
Sep 10 15:41:12.208632 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:41:12.208597    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:41:12.302653 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:41:12.302615    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:41:12.904806 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:41:12.904765    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9c9fa106-613a-45bd-b644-d33ea0bee161" path="/var/lib/kubelet/pods/9c9fa106-613a-45bd-b644-d33ea0bee161/volumes"
Sep 10 15:41:27.463167 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:41:27.463133    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:41:27.569920 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:41:27.569885    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:41:27.805830 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:41:27.805725    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:41:27.909076 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:41:27.909033    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:41:28.887937 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:41:28.887885    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:41:29.061948 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:41:29.061918    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:41:29.397193 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:41:29.397156    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:41:30.896190 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:41:30.896155    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:41:30.995963 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:41:30.995929    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:41:32.038353 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:41:32.038323    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-139155-584bcf7687-nhzlf"]
Sep 10 15:41:32.038709 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:41:32.038669    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9c9fa106-613a-45bd-b644-d33ea0bee161" containerName="authorino"
Sep 10 15:41:32.038709 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:41:32.038680    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="9c9fa106-613a-45bd-b644-d33ea0bee161" containerName="authorino"
Sep 10 15:41:32.038784 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:41:32.038738    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="9c9fa106-613a-45bd-b644-d33ea0bee161" containerName="authorino"
Sep 10 15:41:32.041679 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:41:32.041662    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-139155-584bcf7687-nhzlf"
Sep 10 15:41:32.044317 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:41:32.044292    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-a-139155-serving-cert\""
Sep 10 15:41:32.052223 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:41:32.052199    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-139155-584bcf7687-nhzlf"]
Sep 10 15:41:32.102993 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:41:32.102946    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:41:32.195926 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:41:32.195884    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:41:32.221120 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:41:32.221083    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/53bb17de-fec9-4485-9850-f2d9a1c33d9e-openshift-service-ca\") pod \"maas-api-e2e-inf-a-139155-584bcf7687-nhzlf\" (UID: \"53bb17de-fec9-4485-9850-f2d9a1c33d9e\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-139155-584bcf7687-nhzlf"
Sep 10 15:41:32.221323 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:41:32.221171    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zfldt\" (UniqueName: \"kubernetes.io/projected/53bb17de-fec9-4485-9850-f2d9a1c33d9e-kube-api-access-zfldt\") pod \"maas-api-e2e-inf-a-139155-584bcf7687-nhzlf\" (UID: \"53bb17de-fec9-4485-9850-f2d9a1c33d9e\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-139155-584bcf7687-nhzlf"
Sep 10 15:41:32.221323 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:41:32.221194    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/53bb17de-fec9-4485-9850-f2d9a1c33d9e-maas-api-tls\") pod \"maas-api-e2e-inf-a-139155-584bcf7687-nhzlf\" (UID: \"53bb17de-fec9-4485-9850-f2d9a1c33d9e\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-139155-584bcf7687-nhzlf"
Sep 10 15:41:32.221323 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:41:32.221216    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/53bb17de-fec9-4485-9850-f2d9a1c33d9e-metrics-certs\") pod \"maas-api-e2e-inf-a-139155-584bcf7687-nhzlf\" (UID: \"53bb17de-fec9-4485-9850-f2d9a1c33d9e\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-139155-584bcf7687-nhzlf"
Sep 10 15:41:32.298925 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:41:32.298796    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:41:32.321847 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:41:32.321811    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/53bb17de-fec9-4485-9850-f2d9a1c33d9e-openshift-service-ca\") pod \"maas-api-e2e-inf-a-139155-584bcf7687-nhzlf\" (UID: \"53bb17de-fec9-4485-9850-f2d9a1c33d9e\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-139155-584bcf7687-nhzlf"
Sep 10 15:41:32.322006 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:41:32.321914    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zfldt\" (UniqueName: \"kubernetes.io/projected/53bb17de-fec9-4485-9850-f2d9a1c33d9e-kube-api-access-zfldt\") pod \"maas-api-e2e-inf-a-139155-584bcf7687-nhzlf\" (UID: \"53bb17de-fec9-4485-9850-f2d9a1c33d9e\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-139155-584bcf7687-nhzlf"
Sep 10 15:41:32.322006 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:41:32.321960    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/53bb17de-fec9-4485-9850-f2d9a1c33d9e-maas-api-tls\") pod \"maas-api-e2e-inf-a-139155-584bcf7687-nhzlf\" (UID: \"53bb17de-fec9-4485-9850-f2d9a1c33d9e\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-139155-584bcf7687-nhzlf"
Sep 10 15:41:32.322084 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:41:32.322051    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/53bb17de-fec9-4485-9850-f2d9a1c33d9e-metrics-certs\") pod \"maas-api-e2e-inf-a-139155-584bcf7687-nhzlf\" (UID: \"53bb17de-fec9-4485-9850-f2d9a1c33d9e\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-139155-584bcf7687-nhzlf"
Sep 10 15:41:32.322581 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:41:32.322556    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/53bb17de-fec9-4485-9850-f2d9a1c33d9e-openshift-service-ca\") pod \"maas-api-e2e-inf-a-139155-584bcf7687-nhzlf\" (UID: \"53bb17de-fec9-4485-9850-f2d9a1c33d9e\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-139155-584bcf7687-nhzlf"
Sep 10 15:41:32.324778 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:41:32.324743    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/53bb17de-fec9-4485-9850-f2d9a1c33d9e-maas-api-tls\") pod \"maas-api-e2e-inf-a-139155-584bcf7687-nhzlf\" (UID: \"53bb17de-fec9-4485-9850-f2d9a1c33d9e\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-139155-584bcf7687-nhzlf"
Sep 10 15:41:32.325128 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:41:32.325105    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/53bb17de-fec9-4485-9850-f2d9a1c33d9e-metrics-certs\") pod \"maas-api-e2e-inf-a-139155-584bcf7687-nhzlf\" (UID: \"53bb17de-fec9-4485-9850-f2d9a1c33d9e\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-139155-584bcf7687-nhzlf"
Sep 10 15:41:32.331484 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:41:32.331458    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zfldt\" (UniqueName: \"kubernetes.io/projected/53bb17de-fec9-4485-9850-f2d9a1c33d9e-kube-api-access-zfldt\") pod \"maas-api-e2e-inf-a-139155-584bcf7687-nhzlf\" (UID: \"53bb17de-fec9-4485-9850-f2d9a1c33d9e\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-139155-584bcf7687-nhzlf"
Sep 10 15:41:32.353539 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:41:32.353511    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-139155-584bcf7687-nhzlf"
Sep 10 15:41:32.631129 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:41:32.631102    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-139155-584bcf7687-nhzlf"]
Sep 10 15:41:32.633766 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:41:32.633736    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod53bb17de_fec9_4485_9850_f2d9a1c33d9e.slice/crio-6a58eea1ae8306e1d4b59f722368517aa35f9261b0fb8836429cf39ad381e6aa WatchSource:0}: Error finding container 6a58eea1ae8306e1d4b59f722368517aa35f9261b0fb8836429cf39ad381e6aa: Status 404 returned error can't find the container with id 6a58eea1ae8306e1d4b59f722368517aa35f9261b0fb8836429cf39ad381e6aa
Sep 10 15:41:32.732571 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:41:32.732530    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:41:32.831113 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:41:32.831081    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-139155-7d477d9786-vhwg2"]
Sep 10 15:41:32.833411 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:41:32.833380    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:41:32.835784 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:41:32.835769    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-139155-7d477d9786-vhwg2"
Sep 10 15:41:32.838222 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:41:32.838201    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-a-139155\""
Sep 10 15:41:32.838366 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:41:32.838284    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-a-139155-dockercfg-hrtz5\""
Sep 10 15:41:32.845251 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:41:32.845211    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-139155-7d477d9786-vhwg2"]
Sep 10 15:41:33.029404 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:41:33.029370    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2r2cl\" (UniqueName: \"kubernetes.io/projected/8ad3cae7-ba50-4435-8ee1-7494b6899ee1-kube-api-access-2r2cl\") pod \"payload-pre-processing-e2e-inf-a-139155-7d477d9786-vhwg2\" (UID: \"8ad3cae7-ba50-4435-8ee1-7494b6899ee1\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-139155-7d477d9786-vhwg2"
Sep 10 15:41:33.029598 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:41:33.029528    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8ad3cae7-ba50-4435-8ee1-7494b6899ee1-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-139155-7d477d9786-vhwg2\" (UID: \"8ad3cae7-ba50-4435-8ee1-7494b6899ee1\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-139155-7d477d9786-vhwg2"
Sep 10 15:41:33.130293 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:41:33.130264    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8ad3cae7-ba50-4435-8ee1-7494b6899ee1-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-139155-7d477d9786-vhwg2\" (UID: \"8ad3cae7-ba50-4435-8ee1-7494b6899ee1\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-139155-7d477d9786-vhwg2"
Sep 10 15:41:33.130670 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:41:33.130322    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2r2cl\" (UniqueName: \"kubernetes.io/projected/8ad3cae7-ba50-4435-8ee1-7494b6899ee1-kube-api-access-2r2cl\") pod \"payload-pre-processing-e2e-inf-a-139155-7d477d9786-vhwg2\" (UID: \"8ad3cae7-ba50-4435-8ee1-7494b6899ee1\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-139155-7d477d9786-vhwg2"
Sep 10 15:41:33.131035 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:41:33.131009    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8ad3cae7-ba50-4435-8ee1-7494b6899ee1-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-139155-7d477d9786-vhwg2\" (UID: \"8ad3cae7-ba50-4435-8ee1-7494b6899ee1\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-139155-7d477d9786-vhwg2"
Sep 10 15:41:33.140380 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:41:33.140351    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2r2cl\" (UniqueName: \"kubernetes.io/projected/8ad3cae7-ba50-4435-8ee1-7494b6899ee1-kube-api-access-2r2cl\") pod \"payload-pre-processing-e2e-inf-a-139155-7d477d9786-vhwg2\" (UID: \"8ad3cae7-ba50-4435-8ee1-7494b6899ee1\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-139155-7d477d9786-vhwg2"
Sep 10 15:41:33.148098 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:41:33.148068    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-139155-7d477d9786-vhwg2"
Sep 10 15:41:33.422881 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:41:33.422856    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-139155-7d477d9786-vhwg2"]
Sep 10 15:41:33.425209 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:41:33.425179    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8ad3cae7_ba50_4435_8ee1_7494b6899ee1.slice/crio-7c830ef6a39478005a2e2a069697e5928b093afae4e4d462febfde85551b4ac5 WatchSource:0}: Error finding container 7c830ef6a39478005a2e2a069697e5928b093afae4e4d462febfde85551b4ac5: Status 404 returned error can't find the container with id 7c830ef6a39478005a2e2a069697e5928b093afae4e4d462febfde85551b4ac5
Sep 10 15:41:33.590299 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:41:33.590265    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-139155-584bcf7687-nhzlf" event={"ID":"53bb17de-fec9-4485-9850-f2d9a1c33d9e","Type":"ContainerStarted","Data":"6a58eea1ae8306e1d4b59f722368517aa35f9261b0fb8836429cf39ad381e6aa"}
Sep 10 15:41:33.591845 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:41:33.591818    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-139155-7d477d9786-vhwg2" event={"ID":"8ad3cae7-ba50-4435-8ee1-7494b6899ee1","Type":"ContainerStarted","Data":"fca38a64649665697f165114d6e23973e3e8a95b6e56fe10e66802c353fee1fe"}
Sep 10 15:41:33.591978 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:41:33.591859    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-139155-7d477d9786-vhwg2" event={"ID":"8ad3cae7-ba50-4435-8ee1-7494b6899ee1","Type":"ContainerStarted","Data":"7c830ef6a39478005a2e2a069697e5928b093afae4e4d462febfde85551b4ac5"}
Sep 10 15:41:33.591978 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:41:33.591956    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-139155-7d477d9786-vhwg2"
Sep 10 15:41:33.609423 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:41:33.609370    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-139155-7d477d9786-vhwg2" podStartSLOduration=1.609351864 podStartE2EDuration="1.609351864s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 15:41:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 15:41:33.607491387 +0000 UTC m=+879.175723118" watchObservedRunningTime="2026-09-10 15:41:33.609351864 +0000 UTC m=+879.177583598"
Sep 10 15:41:35.602227 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:41:35.602185    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-139155-584bcf7687-nhzlf" event={"ID":"53bb17de-fec9-4485-9850-f2d9a1c33d9e","Type":"ContainerStarted","Data":"ae4f548bcbd1f79998a621d3f767055a44e7c19006169d656c242c99da895606"}
Sep 10 15:41:35.602648 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:41:35.602267    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-139155-584bcf7687-nhzlf"
Sep 10 15:41:35.619714 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:41:35.619661    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-139155-584bcf7687-nhzlf" podStartSLOduration=1.190342743 podStartE2EDuration="3.619646264s" totalImagesPullingTime="2.429303521s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 15:41:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 15:41:35.61934864 +0000 UTC m=+881.187580377" watchObservedRunningTime="2026-09-10 15:41:35.619646264 +0000 UTC m=+881.187877994"
Sep 10 15:41:35.756170 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:41:35.756136    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:41:35.823908 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:41:35.823873    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:41:35.983735 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:41:35.983696    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:41:36.315704 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:41:36.315628    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:41:36.982102 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:41:36.982068    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:41:37.082517 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:41:37.082478    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:41:37.301134 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:41:37.301046    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:41:37.407714 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:41:37.407680    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:41:37.740882 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:41:37.740847    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:41:38.399453 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:41:38.399425    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:41:38.498171 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:41:38.498133    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:41:38.823052 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:41:38.823015    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:41:39.919304 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:41:39.919251    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-7a711aa8-59ccd69dfc-47t9r"]
Sep 10 15:41:39.923768 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:41:39.923744    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-7a711aa8-59ccd69dfc-47t9r"
Sep 10 15:41:39.926283 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:41:39.926257    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-7a711aa8-serving-cert\""
Sep 10 15:41:39.936431 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:41:39.935841    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-7a711aa8-59ccd69dfc-47t9r"]
Sep 10 15:41:39.967247 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:41:39.967198    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:41:39.989433 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:41:39.989401    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/eb3192ca-7107-41a8-a249-a1afd0791495-maas-api-tls\") pod \"maas-api-e2e-ait-7a711aa8-59ccd69dfc-47t9r\" (UID: \"eb3192ca-7107-41a8-a249-a1afd0791495\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-7a711aa8-59ccd69dfc-47t9r"
Sep 10 15:41:39.989625 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:41:39.989523    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/eb3192ca-7107-41a8-a249-a1afd0791495-openshift-service-ca\") pod \"maas-api-e2e-ait-7a711aa8-59ccd69dfc-47t9r\" (UID: \"eb3192ca-7107-41a8-a249-a1afd0791495\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-7a711aa8-59ccd69dfc-47t9r"
Sep 10 15:41:39.989625 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:41:39.989556    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/eb3192ca-7107-41a8-a249-a1afd0791495-metrics-certs\") pod \"maas-api-e2e-ait-7a711aa8-59ccd69dfc-47t9r\" (UID: \"eb3192ca-7107-41a8-a249-a1afd0791495\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-7a711aa8-59ccd69dfc-47t9r"
Sep 10 15:41:39.989738 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:41:39.989640    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sgf6f\" (UniqueName: \"kubernetes.io/projected/eb3192ca-7107-41a8-a249-a1afd0791495-kube-api-access-sgf6f\") pod \"maas-api-e2e-ait-7a711aa8-59ccd69dfc-47t9r\" (UID: \"eb3192ca-7107-41a8-a249-a1afd0791495\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-7a711aa8-59ccd69dfc-47t9r"
Sep 10 15:41:40.069609 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:41:40.069579    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:41:40.090716 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:41:40.090678    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/eb3192ca-7107-41a8-a249-a1afd0791495-openshift-service-ca\") pod \"maas-api-e2e-ait-7a711aa8-59ccd69dfc-47t9r\" (UID: \"eb3192ca-7107-41a8-a249-a1afd0791495\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-7a711aa8-59ccd69dfc-47t9r"
Sep 10 15:41:40.090716 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:41:40.090710    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/eb3192ca-7107-41a8-a249-a1afd0791495-metrics-certs\") pod \"maas-api-e2e-ait-7a711aa8-59ccd69dfc-47t9r\" (UID: \"eb3192ca-7107-41a8-a249-a1afd0791495\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-7a711aa8-59ccd69dfc-47t9r"
Sep 10 15:41:40.090973 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:41:40.090780    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sgf6f\" (UniqueName: \"kubernetes.io/projected/eb3192ca-7107-41a8-a249-a1afd0791495-kube-api-access-sgf6f\") pod \"maas-api-e2e-ait-7a711aa8-59ccd69dfc-47t9r\" (UID: \"eb3192ca-7107-41a8-a249-a1afd0791495\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-7a711aa8-59ccd69dfc-47t9r"
Sep 10 15:41:40.090973 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:41:40.090816    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/eb3192ca-7107-41a8-a249-a1afd0791495-maas-api-tls\") pod \"maas-api-e2e-ait-7a711aa8-59ccd69dfc-47t9r\" (UID: \"eb3192ca-7107-41a8-a249-a1afd0791495\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-7a711aa8-59ccd69dfc-47t9r"
Sep 10 15:41:40.090973 ip-10-0-128-241 kubenswrapper[2567]: E0910 15:41:40.090918    2567 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-7a711aa8-serving-cert: secret "maas-api-e2e-ait-7a711aa8-serving-cert" not found
Sep 10 15:41:40.091132 ip-10-0-128-241 kubenswrapper[2567]: E0910 15:41:40.090983    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/eb3192ca-7107-41a8-a249-a1afd0791495-maas-api-tls podName:eb3192ca-7107-41a8-a249-a1afd0791495 nodeName:}" failed. No retries permitted until 2026-09-10 15:41:40.590964131 +0000 UTC m=+886.159195842 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/eb3192ca-7107-41a8-a249-a1afd0791495-maas-api-tls") pod "maas-api-e2e-ait-7a711aa8-59ccd69dfc-47t9r" (UID: "eb3192ca-7107-41a8-a249-a1afd0791495") : secret "maas-api-e2e-ait-7a711aa8-serving-cert" not found
Sep 10 15:41:40.091518 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:41:40.091493    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/eb3192ca-7107-41a8-a249-a1afd0791495-openshift-service-ca\") pod \"maas-api-e2e-ait-7a711aa8-59ccd69dfc-47t9r\" (UID: \"eb3192ca-7107-41a8-a249-a1afd0791495\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-7a711aa8-59ccd69dfc-47t9r"
Sep 10 15:41:40.093418 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:41:40.093394    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/eb3192ca-7107-41a8-a249-a1afd0791495-metrics-certs\") pod \"maas-api-e2e-ait-7a711aa8-59ccd69dfc-47t9r\" (UID: \"eb3192ca-7107-41a8-a249-a1afd0791495\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-7a711aa8-59ccd69dfc-47t9r"
Sep 10 15:41:40.104705 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:41:40.104671    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sgf6f\" (UniqueName: \"kubernetes.io/projected/eb3192ca-7107-41a8-a249-a1afd0791495-kube-api-access-sgf6f\") pod \"maas-api-e2e-ait-7a711aa8-59ccd69dfc-47t9r\" (UID: \"eb3192ca-7107-41a8-a249-a1afd0791495\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-7a711aa8-59ccd69dfc-47t9r"
Sep 10 15:41:40.167527 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:41:40.167486    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:41:40.271620 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:41:40.271524    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:41:40.517656 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:41:40.517622    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:41:40.595271 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:41:40.595139    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/eb3192ca-7107-41a8-a249-a1afd0791495-maas-api-tls\") pod \"maas-api-e2e-ait-7a711aa8-59ccd69dfc-47t9r\" (UID: \"eb3192ca-7107-41a8-a249-a1afd0791495\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-7a711aa8-59ccd69dfc-47t9r"
Sep 10 15:41:40.597569 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:41:40.597548    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/eb3192ca-7107-41a8-a249-a1afd0791495-maas-api-tls\") pod \"maas-api-e2e-ait-7a711aa8-59ccd69dfc-47t9r\" (UID: \"eb3192ca-7107-41a8-a249-a1afd0791495\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-7a711aa8-59ccd69dfc-47t9r"
Sep 10 15:41:40.683104 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:41:40.683077    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:41:40.785159 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:41:40.785118    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:41:40.790065 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:41:40.790027    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-7a711aa8-6679cb8df4-slfdf"]
Sep 10 15:41:40.796272 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:41:40.796215    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-7a711aa8-6679cb8df4-slfdf"
Sep 10 15:41:40.797716 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:41:40.797681    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hcd6f\" (UniqueName: \"kubernetes.io/projected/71dc332d-0da7-480e-8757-43821adc9365-kube-api-access-hcd6f\") pod \"payload-pre-processing-e2e-ait-7a711aa8-6679cb8df4-slfdf\" (UID: \"71dc332d-0da7-480e-8757-43821adc9365\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-7a711aa8-6679cb8df4-slfdf"
Sep 10 15:41:40.797864 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:41:40.797807    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/71dc332d-0da7-480e-8757-43821adc9365-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-7a711aa8-6679cb8df4-slfdf\" (UID: \"71dc332d-0da7-480e-8757-43821adc9365\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-7a711aa8-6679cb8df4-slfdf"
Sep 10 15:41:40.798904 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:41:40.798867    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-7a711aa8-5cf84dd5df-64g8h"]
Sep 10 15:41:40.799407 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:41:40.799383    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-7a711aa8\""
Sep 10 15:41:40.802945 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:41:40.802920    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-7a711aa8-5cf84dd5df-64g8h"
Sep 10 15:41:40.804615 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:41:40.804591    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-7a711aa8-6679cb8df4-slfdf"]
Sep 10 15:41:40.813138 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:41:40.813108    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-7a711aa8-5cf84dd5df-64g8h"]
Sep 10 15:41:40.839381 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:41:40.839338    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-7a711aa8-59ccd69dfc-47t9r"
Sep 10 15:41:40.899210 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:41:40.899145    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xvwbl\" (UniqueName: \"kubernetes.io/projected/37106482-a471-4b7e-b641-2a4bd544cf72-kube-api-access-xvwbl\") pod \"payload-processing-e2e-ait-7a711aa8-5cf84dd5df-64g8h\" (UID: \"37106482-a471-4b7e-b641-2a4bd544cf72\") " pod="openshift-ingress/payload-processing-e2e-ait-7a711aa8-5cf84dd5df-64g8h"
Sep 10 15:41:40.899386 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:41:40.899299    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/71dc332d-0da7-480e-8757-43821adc9365-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-7a711aa8-6679cb8df4-slfdf\" (UID: \"71dc332d-0da7-480e-8757-43821adc9365\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-7a711aa8-6679cb8df4-slfdf"
Sep 10 15:41:40.899386 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:41:40.899381    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hcd6f\" (UniqueName: \"kubernetes.io/projected/71dc332d-0da7-480e-8757-43821adc9365-kube-api-access-hcd6f\") pod \"payload-pre-processing-e2e-ait-7a711aa8-6679cb8df4-slfdf\" (UID: \"71dc332d-0da7-480e-8757-43821adc9365\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-7a711aa8-6679cb8df4-slfdf"
Sep 10 15:41:40.899518 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:41:40.899412    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/37106482-a471-4b7e-b641-2a4bd544cf72-plugins-config-volume\") pod \"payload-processing-e2e-ait-7a711aa8-5cf84dd5df-64g8h\" (UID: \"37106482-a471-4b7e-b641-2a4bd544cf72\") " pod="openshift-ingress/payload-processing-e2e-ait-7a711aa8-5cf84dd5df-64g8h"
Sep 10 15:41:40.900089 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:41:40.900063    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/71dc332d-0da7-480e-8757-43821adc9365-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-7a711aa8-6679cb8df4-slfdf\" (UID: \"71dc332d-0da7-480e-8757-43821adc9365\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-7a711aa8-6679cb8df4-slfdf"
Sep 10 15:41:40.917644 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:41:40.917590    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hcd6f\" (UniqueName: \"kubernetes.io/projected/71dc332d-0da7-480e-8757-43821adc9365-kube-api-access-hcd6f\") pod \"payload-pre-processing-e2e-ait-7a711aa8-6679cb8df4-slfdf\" (UID: \"71dc332d-0da7-480e-8757-43821adc9365\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-7a711aa8-6679cb8df4-slfdf"
Sep 10 15:41:41.000128 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:41:41.000089    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xvwbl\" (UniqueName: \"kubernetes.io/projected/37106482-a471-4b7e-b641-2a4bd544cf72-kube-api-access-xvwbl\") pod \"payload-processing-e2e-ait-7a711aa8-5cf84dd5df-64g8h\" (UID: \"37106482-a471-4b7e-b641-2a4bd544cf72\") " pod="openshift-ingress/payload-processing-e2e-ait-7a711aa8-5cf84dd5df-64g8h"
Sep 10 15:41:41.000548 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:41:41.000187    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/37106482-a471-4b7e-b641-2a4bd544cf72-plugins-config-volume\") pod \"payload-processing-e2e-ait-7a711aa8-5cf84dd5df-64g8h\" (UID: \"37106482-a471-4b7e-b641-2a4bd544cf72\") " pod="openshift-ingress/payload-processing-e2e-ait-7a711aa8-5cf84dd5df-64g8h"
Sep 10 15:41:41.000898 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:41:41.000876    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/37106482-a471-4b7e-b641-2a4bd544cf72-plugins-config-volume\") pod \"payload-processing-e2e-ait-7a711aa8-5cf84dd5df-64g8h\" (UID: \"37106482-a471-4b7e-b641-2a4bd544cf72\") " pod="openshift-ingress/payload-processing-e2e-ait-7a711aa8-5cf84dd5df-64g8h"
Sep 10 15:41:41.011007 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:41:41.010977    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xvwbl\" (UniqueName: \"kubernetes.io/projected/37106482-a471-4b7e-b641-2a4bd544cf72-kube-api-access-xvwbl\") pod \"payload-processing-e2e-ait-7a711aa8-5cf84dd5df-64g8h\" (UID: \"37106482-a471-4b7e-b641-2a4bd544cf72\") " pod="openshift-ingress/payload-processing-e2e-ait-7a711aa8-5cf84dd5df-64g8h"
Sep 10 15:41:41.110982 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:41:41.110893    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-7a711aa8-6679cb8df4-slfdf"
Sep 10 15:41:41.118545 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:41:41.118512    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-7a711aa8-5cf84dd5df-64g8h"
Sep 10 15:41:41.119600 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:41:41.119580    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-7a711aa8-59ccd69dfc-47t9r"]
Sep 10 15:41:41.122617 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:41:41.122593    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeb3192ca_7107_41a8_a249_a1afd0791495.slice/crio-15a17efe1cd7fa057c32fe9c4a566e8d53ab44c8f23bda3e796b3d812619a6a2 WatchSource:0}: Error finding container 15a17efe1cd7fa057c32fe9c4a566e8d53ab44c8f23bda3e796b3d812619a6a2: Status 404 returned error can't find the container with id 15a17efe1cd7fa057c32fe9c4a566e8d53ab44c8f23bda3e796b3d812619a6a2
Sep 10 15:41:41.308462 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:41:41.308422    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:41:41.393561 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:41:41.393532    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-7a711aa8-6679cb8df4-slfdf"]
Sep 10 15:41:41.395657 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:41:41.395629    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod71dc332d_0da7_480e_8757_43821adc9365.slice/crio-979ed9273dc8446aa985b24f3f8e25e8cda4f33b361c1a4925f511e06f76181c WatchSource:0}: Error finding container 979ed9273dc8446aa985b24f3f8e25e8cda4f33b361c1a4925f511e06f76181c: Status 404 returned error can't find the container with id 979ed9273dc8446aa985b24f3f8e25e8cda4f33b361c1a4925f511e06f76181c
Sep 10 15:41:41.412325 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:41:41.412296    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-7a711aa8-5cf84dd5df-64g8h"]
Sep 10 15:41:41.414086 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:41:41.414047    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod37106482_a471_4b7e_b641_2a4bd544cf72.slice/crio-3b9a31c374cab6d4377b34bc1a81a8655759a1aa3f514a249d3cd874fa9e01d5 WatchSource:0}: Error finding container 3b9a31c374cab6d4377b34bc1a81a8655759a1aa3f514a249d3cd874fa9e01d5: Status 404 returned error can't find the container with id 3b9a31c374cab6d4377b34bc1a81a8655759a1aa3f514a249d3cd874fa9e01d5
Sep 10 15:41:41.613620 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:41:41.613591    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-139155-584bcf7687-nhzlf"
Sep 10 15:41:41.635101 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:41:41.635065    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-7a711aa8-5cf84dd5df-64g8h" event={"ID":"37106482-a471-4b7e-b641-2a4bd544cf72","Type":"ContainerStarted","Data":"e8bc1efa88a290a7ea686606c3a3c8d26e3ccdb39a2c9531a3280a690ee706f2"}
Sep 10 15:41:41.635324 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:41:41.635119    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-7a711aa8-5cf84dd5df-64g8h" event={"ID":"37106482-a471-4b7e-b641-2a4bd544cf72","Type":"ContainerStarted","Data":"3b9a31c374cab6d4377b34bc1a81a8655759a1aa3f514a249d3cd874fa9e01d5"}
Sep 10 15:41:41.635324 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:41:41.635159    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ait-7a711aa8-5cf84dd5df-64g8h"
Sep 10 15:41:41.636889 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:41:41.636851    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-7a711aa8-59ccd69dfc-47t9r" event={"ID":"eb3192ca-7107-41a8-a249-a1afd0791495","Type":"ContainerStarted","Data":"d8a5c19d2b5ca9e50120fa7730585fc5624d3555bb3c181a690b0e962a4bbb39"}
Sep 10 15:41:41.637016 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:41:41.636903    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-7a711aa8-59ccd69dfc-47t9r"
Sep 10 15:41:41.637016 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:41:41.636926    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-7a711aa8-59ccd69dfc-47t9r" event={"ID":"eb3192ca-7107-41a8-a249-a1afd0791495","Type":"ContainerStarted","Data":"15a17efe1cd7fa057c32fe9c4a566e8d53ab44c8f23bda3e796b3d812619a6a2"}
Sep 10 15:41:41.638394 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:41:41.638366    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-7a711aa8-6679cb8df4-slfdf" event={"ID":"71dc332d-0da7-480e-8757-43821adc9365","Type":"ContainerStarted","Data":"6f99a8e03cd843589aca79bab3ad1b16c5cbe647f0f8bed1155a6c19050a4ffd"}
Sep 10 15:41:41.638508 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:41:41.638406    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-7a711aa8-6679cb8df4-slfdf" event={"ID":"71dc332d-0da7-480e-8757-43821adc9365","Type":"ContainerStarted","Data":"979ed9273dc8446aa985b24f3f8e25e8cda4f33b361c1a4925f511e06f76181c"}
Sep 10 15:41:41.638508 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:41:41.638426    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-7a711aa8-6679cb8df4-slfdf"
Sep 10 15:41:41.657661 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:41:41.657561    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ait-7a711aa8-5cf84dd5df-64g8h" podStartSLOduration=1.657543103 podStartE2EDuration="1.657543103s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 15:41:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 15:41:41.65640669 +0000 UTC m=+887.224638423" watchObservedRunningTime="2026-09-10 15:41:41.657543103 +0000 UTC m=+887.225774866"
Sep 10 15:41:41.677736 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:41:41.677661    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ait-7a711aa8-6679cb8df4-slfdf" podStartSLOduration=1.67764438 podStartE2EDuration="1.67764438s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 15:41:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 15:41:41.676468693 +0000 UTC m=+887.244700424" watchObservedRunningTime="2026-09-10 15:41:41.67764438 +0000 UTC m=+887.245876110"
Sep 10 15:41:41.694982 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:41:41.694931    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-7a711aa8-59ccd69dfc-47t9r" podStartSLOduration=2.278338501 podStartE2EDuration="2.694916878s" totalImagesPullingTime="416.578377ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 15:41:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 15:41:41.693034233 +0000 UTC m=+887.261265962" watchObservedRunningTime="2026-09-10 15:41:41.694916878 +0000 UTC m=+887.263148644"
Sep 10 15:41:42.127890 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:41:42.127847    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:41:43.138526 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:41:43.138496    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:41:43.942363 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:41:43.942322    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:41:44.009929 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:41:44.009894    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:41:44.110221 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:41:44.110183    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:41:44.515049 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:41:44.515017    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:41:44.598666 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:41:44.598637    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-139155-7d477d9786-vhwg2"
Sep 10 15:41:44.608289 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:41:44.608248    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:41:44.706579 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:41:44.706532    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:41:44.745603 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:41:44.745567    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-bd5806-647db94487-xv97q"]
Sep 10 15:41:44.749190 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:41:44.749170    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-autores-bd5806-647db94487-xv97q"
Sep 10 15:41:44.751691 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:41:44.751663    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-autores-bd5806\""
Sep 10 15:41:44.751846 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:41:44.751784    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-autores-bd5806-dockercfg-f2q22\""
Sep 10 15:41:44.760392 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:41:44.760361    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-bd5806-647db94487-xv97q"]
Sep 10 15:41:44.830161 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:41:44.830070    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vv9pt\" (UniqueName: \"kubernetes.io/projected/a8268b2f-1ba6-42c8-bb45-bcc2c389a765-kube-api-access-vv9pt\") pod \"payload-processing-e2e-autores-bd5806-647db94487-xv97q\" (UID: \"a8268b2f-1ba6-42c8-bb45-bcc2c389a765\") " pod="openshift-ingress/payload-processing-e2e-autores-bd5806-647db94487-xv97q"
Sep 10 15:41:44.830161 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:41:44.830137    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a8268b2f-1ba6-42c8-bb45-bcc2c389a765-plugins-config-volume\") pod \"payload-processing-e2e-autores-bd5806-647db94487-xv97q\" (UID: \"a8268b2f-1ba6-42c8-bb45-bcc2c389a765\") " pod="openshift-ingress/payload-processing-e2e-autores-bd5806-647db94487-xv97q"
Sep 10 15:41:44.931130 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:41:44.931068    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vv9pt\" (UniqueName: \"kubernetes.io/projected/a8268b2f-1ba6-42c8-bb45-bcc2c389a765-kube-api-access-vv9pt\") pod \"payload-processing-e2e-autores-bd5806-647db94487-xv97q\" (UID: \"a8268b2f-1ba6-42c8-bb45-bcc2c389a765\") " pod="openshift-ingress/payload-processing-e2e-autores-bd5806-647db94487-xv97q"
Sep 10 15:41:44.931339 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:41:44.931225    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a8268b2f-1ba6-42c8-bb45-bcc2c389a765-plugins-config-volume\") pod \"payload-processing-e2e-autores-bd5806-647db94487-xv97q\" (UID: \"a8268b2f-1ba6-42c8-bb45-bcc2c389a765\") " pod="openshift-ingress/payload-processing-e2e-autores-bd5806-647db94487-xv97q"
Sep 10 15:41:44.931926 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:41:44.931904    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a8268b2f-1ba6-42c8-bb45-bcc2c389a765-plugins-config-volume\") pod \"payload-processing-e2e-autores-bd5806-647db94487-xv97q\" (UID: \"a8268b2f-1ba6-42c8-bb45-bcc2c389a765\") " pod="openshift-ingress/payload-processing-e2e-autores-bd5806-647db94487-xv97q"
Sep 10 15:41:44.940100 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:41:44.940076    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vv9pt\" (UniqueName: \"kubernetes.io/projected/a8268b2f-1ba6-42c8-bb45-bcc2c389a765-kube-api-access-vv9pt\") pod \"payload-processing-e2e-autores-bd5806-647db94487-xv97q\" (UID: \"a8268b2f-1ba6-42c8-bb45-bcc2c389a765\") " pod="openshift-ingress/payload-processing-e2e-autores-bd5806-647db94487-xv97q"
Sep 10 15:41:45.062158 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:41:45.062120    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-autores-bd5806-647db94487-xv97q"
Sep 10 15:41:45.158738 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:41:45.158704    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:41:45.338919 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:41:45.338889    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-bd5806-647db94487-xv97q"]
Sep 10 15:41:45.341368 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:41:45.341339    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda8268b2f_1ba6_42c8_bb45_bcc2c389a765.slice/crio-828de96c78bc54ef5f937ab6124c6b3282b148b956667262e9999970f6c12fdc WatchSource:0}: Error finding container 828de96c78bc54ef5f937ab6124c6b3282b148b956667262e9999970f6c12fdc: Status 404 returned error can't find the container with id 828de96c78bc54ef5f937ab6124c6b3282b148b956667262e9999970f6c12fdc
Sep 10 15:41:45.612104 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:41:45.612065    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:41:45.665433 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:41:45.665305    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-autores-bd5806-647db94487-xv97q" event={"ID":"a8268b2f-1ba6-42c8-bb45-bcc2c389a765","Type":"ContainerStarted","Data":"b07b4ed8fb8b3e24bda01e2e9cc6f3307b41c0c0267f2e1b56d7bce4f3eb19c9"}
Sep 10 15:41:45.665433 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:41:45.665373    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-autores-bd5806-647db94487-xv97q" event={"ID":"a8268b2f-1ba6-42c8-bb45-bcc2c389a765","Type":"ContainerStarted","Data":"828de96c78bc54ef5f937ab6124c6b3282b148b956667262e9999970f6c12fdc"}
Sep 10 15:41:45.665433 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:41:45.665403    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-autores-bd5806-647db94487-xv97q"
Sep 10 15:41:45.684537 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:41:45.684479    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-autores-bd5806-647db94487-xv97q" podStartSLOduration=1.684459048 podStartE2EDuration="1.684459048s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 15:41:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 15:41:45.682119933 +0000 UTC m=+891.250351663" watchObservedRunningTime="2026-09-10 15:41:45.684459048 +0000 UTC m=+891.252690782"
Sep 10 15:41:45.956380 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:41:45.956342    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:41:46.054064 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:41:46.054026    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:41:46.164863 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:41:46.164832    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:41:46.273660 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:41:46.273550    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:41:46.374893 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:41:46.374844    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:41:46.471063 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:41:46.471018    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:41:46.571885 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:41:46.571803    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:41:46.753199 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:41:46.753163    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:41:46.978573 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:41:46.978536    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:41:47.650645 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:41:47.650612    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-7a711aa8-59ccd69dfc-47t9r"
Sep 10 15:41:47.784098 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:41:47.784063    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:41:49.202625 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:41:49.202574    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:41:50.595476 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:41:50.595445    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:41:50.699490 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:41:50.699456    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:41:50.819490 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:41:50.819451    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:41:50.939633 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:41:50.939598    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:41:52.356518 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:41:52.356480    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:41:52.646389 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:41:52.646281    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ait-7a711aa8-5cf84dd5df-64g8h"
Sep 10 15:41:52.646598 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:41:52.646503    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-7a711aa8-6679cb8df4-slfdf"
Sep 10 15:41:52.959882 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:41:52.959838    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:41:53.064573 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:41:53.064540    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:41:53.512724 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:41:53.512688    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:41:54.865628 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:41:54.865597    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:41:54.877743 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:41:54.877717    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-t96pm_0b9d2cda-73e3-4e7c-9e17-1a3d903d95cf/console-operator/2.log"
Sep 10 15:41:54.879058 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:41:54.879039    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-t96pm_0b9d2cda-73e3-4e7c-9e17-1a3d903d95cf/console-operator/2.log"
Sep 10 15:41:55.725414 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:41:55.725379    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:41:56.672659 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:41:56.672629    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-autores-bd5806-647db94487-xv97q"
Sep 10 15:41:57.023786 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:41:57.023688    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:41:57.124522 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:41:57.124489    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:41:57.544528 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:41:57.544488    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:41:57.693902 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:41:57.693862    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:41:57.789554 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:41:57.789518    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:41:57.877808 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:41:57.877720    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-90ca97-846c64b6fd-frz4x"]
Sep 10 15:41:57.882745 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:41:57.882722    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-90ca97-846c64b6fd-frz4x"
Sep 10 15:41:57.885010 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:41:57.884981    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-b-90ca97-dockercfg-9vtq7\""
Sep 10 15:41:57.885120 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:41:57.885025    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-b-90ca97\""
Sep 10 15:41:57.891944 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:41:57.891920    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-90ca97-846c64b6fd-frz4x"]
Sep 10 15:41:57.955473 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:41:57.955441    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0931ea56-b8ce-43cc-861c-07b7c1a51893-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-90ca97-846c64b6fd-frz4x\" (UID: \"0931ea56-b8ce-43cc-861c-07b7c1a51893\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-90ca97-846c64b6fd-frz4x"
Sep 10 15:41:57.955473 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:41:57.955477    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-phck5\" (UniqueName: \"kubernetes.io/projected/0931ea56-b8ce-43cc-861c-07b7c1a51893-kube-api-access-phck5\") pod \"payload-pre-processing-e2e-inf-b-90ca97-846c64b6fd-frz4x\" (UID: \"0931ea56-b8ce-43cc-861c-07b7c1a51893\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-90ca97-846c64b6fd-frz4x"
Sep 10 15:41:58.057179 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:41:58.057132    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0931ea56-b8ce-43cc-861c-07b7c1a51893-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-90ca97-846c64b6fd-frz4x\" (UID: \"0931ea56-b8ce-43cc-861c-07b7c1a51893\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-90ca97-846c64b6fd-frz4x"
Sep 10 15:41:58.057409 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:41:58.057189    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-phck5\" (UniqueName: \"kubernetes.io/projected/0931ea56-b8ce-43cc-861c-07b7c1a51893-kube-api-access-phck5\") pod \"payload-pre-processing-e2e-inf-b-90ca97-846c64b6fd-frz4x\" (UID: \"0931ea56-b8ce-43cc-861c-07b7c1a51893\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-90ca97-846c64b6fd-frz4x"
Sep 10 15:41:58.057956 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:41:58.057928    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0931ea56-b8ce-43cc-861c-07b7c1a51893-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-90ca97-846c64b6fd-frz4x\" (UID: \"0931ea56-b8ce-43cc-861c-07b7c1a51893\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-90ca97-846c64b6fd-frz4x"
Sep 10 15:41:58.067171 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:41:58.067142    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-phck5\" (UniqueName: \"kubernetes.io/projected/0931ea56-b8ce-43cc-861c-07b7c1a51893-kube-api-access-phck5\") pod \"payload-pre-processing-e2e-inf-b-90ca97-846c64b6fd-frz4x\" (UID: \"0931ea56-b8ce-43cc-861c-07b7c1a51893\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-90ca97-846c64b6fd-frz4x"
Sep 10 15:41:58.194642 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:41:58.194605    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-90ca97-846c64b6fd-frz4x"
Sep 10 15:41:58.316471 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:41:58.316439    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:41:58.464792 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:41:58.464709    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-90ca97-846c64b6fd-frz4x"]
Sep 10 15:41:58.467875 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:41:58.467846    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0931ea56_b8ce_43cc_861c_07b7c1a51893.slice/crio-15b50f426aec34e7f7417cfb67eb4c5a4360684bc336a06565c49c2abcf1e1f4 WatchSource:0}: Error finding container 15b50f426aec34e7f7417cfb67eb4c5a4360684bc336a06565c49c2abcf1e1f4: Status 404 returned error can't find the container with id 15b50f426aec34e7f7417cfb67eb4c5a4360684bc336a06565c49c2abcf1e1f4
Sep 10 15:41:58.736943 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:41:58.736856    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-90ca97-846c64b6fd-frz4x" event={"ID":"0931ea56-b8ce-43cc-861c-07b7c1a51893","Type":"ContainerStarted","Data":"d7c6e741dfcf675d9eca93b0cb47ddd4262c084945f1dde30980906e28a85f72"}
Sep 10 15:41:58.736943 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:41:58.736905    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-90ca97-846c64b6fd-frz4x" event={"ID":"0931ea56-b8ce-43cc-861c-07b7c1a51893","Type":"ContainerStarted","Data":"15b50f426aec34e7f7417cfb67eb4c5a4360684bc336a06565c49c2abcf1e1f4"}
Sep 10 15:41:58.737390 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:41:58.736975    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-90ca97-846c64b6fd-frz4x"
Sep 10 15:41:58.760988 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:41:58.760930    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-90ca97-846c64b6fd-frz4x" podStartSLOduration=1.760912524 podStartE2EDuration="1.760912524s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 15:41:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 15:41:58.760566675 +0000 UTC m=+904.328798405" watchObservedRunningTime="2026-09-10 15:41:58.760912524 +0000 UTC m=+904.329144253"
Sep 10 15:41:59.314548 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:41:59.314519    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:41:59.425428 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:41:59.425387    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:41:59.535273 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:41:59.535217    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:41:59.649090 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:41:59.648994    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:41:59.801524 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:41:59.801489    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:41:59.942477 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:41:59.942446    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:42:00.469727 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:42:00.469688    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:42:01.359573 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:42:01.359537    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:42:02.277905 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:42:02.277867    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:42:03.187245 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:42:03.187210    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:42:04.143540 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:42:04.143504    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:42:04.868314 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:42:04.868267    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:42:05.724076 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:42:05.724039    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:42:06.738500 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:42:06.738451    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:42:06.841679 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:42:06.841640    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:42:08.932467 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:42:08.932425    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:42:09.566138 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:42:09.566106    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-7a711aa8-59ccd69dfc-47t9r"]
Sep 10 15:42:09.568222 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:42:09.566335    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-7a711aa8-59ccd69dfc-47t9r" podUID="eb3192ca-7107-41a8-a249-a1afd0791495" containerName="maas-api" containerID="cri-o://d8a5c19d2b5ca9e50120fa7730585fc5624d3555bb3c181a690b0e962a4bbb39" gracePeriod=30
Sep 10 15:42:09.587961 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:42:09.587884    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:42:09.635944 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:42:09.635909    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-7a711aa8-5cf84dd5df-64g8h"]
Sep 10 15:42:09.636192 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:42:09.636161    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ait-7a711aa8-5cf84dd5df-64g8h" podUID="37106482-a471-4b7e-b641-2a4bd544cf72" containerName="payload-processing" containerID="cri-o://e8bc1efa88a290a7ea686606c3a3c8d26e3ccdb39a2c9531a3280a690ee706f2" gracePeriod=30
Sep 10 15:42:09.650024 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:42:09.649988    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-7a711aa8-6679cb8df4-slfdf"]
Sep 10 15:42:09.650337 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:42:09.650227    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ait-7a711aa8-6679cb8df4-slfdf" podUID="71dc332d-0da7-480e-8757-43821adc9365" containerName="payload-pre-processing" containerID="cri-o://6f99a8e03cd843589aca79bab3ad1b16c5cbe647f0f8bed1155a6c19050a4ffd" gracePeriod=30
Sep 10 15:42:09.682384 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:42:09.682350    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:42:09.744460 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:42:09.744426    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-90ca97-846c64b6fd-frz4x"
Sep 10 15:42:09.793845 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:42:09.793722    2567 generic.go:356] "Generic (PLEG): container finished" podID="37106482-a471-4b7e-b641-2a4bd544cf72" containerID="e8bc1efa88a290a7ea686606c3a3c8d26e3ccdb39a2c9531a3280a690ee706f2" exitCode=0
Sep 10 15:42:09.793845 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:42:09.793799    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-7a711aa8-5cf84dd5df-64g8h" event={"ID":"37106482-a471-4b7e-b641-2a4bd544cf72","Type":"ContainerDied","Data":"e8bc1efa88a290a7ea686606c3a3c8d26e3ccdb39a2c9531a3280a690ee706f2"}
Sep 10 15:42:09.796759 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:42:09.795995    2567 generic.go:356] "Generic (PLEG): container finished" podID="eb3192ca-7107-41a8-a249-a1afd0791495" containerID="d8a5c19d2b5ca9e50120fa7730585fc5624d3555bb3c181a690b0e962a4bbb39" exitCode=0
Sep 10 15:42:09.796759 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:42:09.796157    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-7a711aa8-59ccd69dfc-47t9r" event={"ID":"eb3192ca-7107-41a8-a249-a1afd0791495","Type":"ContainerDied","Data":"d8a5c19d2b5ca9e50120fa7730585fc5624d3555bb3c181a690b0e962a4bbb39"}
Sep 10 15:42:09.798682 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:42:09.798618    2567 generic.go:356] "Generic (PLEG): container finished" podID="71dc332d-0da7-480e-8757-43821adc9365" containerID="6f99a8e03cd843589aca79bab3ad1b16c5cbe647f0f8bed1155a6c19050a4ffd" exitCode=0
Sep 10 15:42:09.798682 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:42:09.798657    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-7a711aa8-6679cb8df4-slfdf" event={"ID":"71dc332d-0da7-480e-8757-43821adc9365","Type":"ContainerDied","Data":"6f99a8e03cd843589aca79bab3ad1b16c5cbe647f0f8bed1155a6c19050a4ffd"}
Sep 10 15:42:09.950716 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:42:09.950685    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-7a711aa8-5cf84dd5df-64g8h"
Sep 10 15:42:09.969896 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:42:09.969868    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-7a711aa8-59ccd69dfc-47t9r"
Sep 10 15:42:09.975952 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:42:09.975927    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-7a711aa8-6679cb8df4-slfdf"
Sep 10 15:42:10.072308 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:42:10.072202    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xvwbl\" (UniqueName: \"kubernetes.io/projected/37106482-a471-4b7e-b641-2a4bd544cf72-kube-api-access-xvwbl\") pod \"37106482-a471-4b7e-b641-2a4bd544cf72\" (UID: \"37106482-a471-4b7e-b641-2a4bd544cf72\") "
Sep 10 15:42:10.072308 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:42:10.072289    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sgf6f\" (UniqueName: \"kubernetes.io/projected/eb3192ca-7107-41a8-a249-a1afd0791495-kube-api-access-sgf6f\") pod \"eb3192ca-7107-41a8-a249-a1afd0791495\" (UID: \"eb3192ca-7107-41a8-a249-a1afd0791495\") "
Sep 10 15:42:10.072543 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:42:10.072319    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/eb3192ca-7107-41a8-a249-a1afd0791495-maas-api-tls\") pod \"eb3192ca-7107-41a8-a249-a1afd0791495\" (UID: \"eb3192ca-7107-41a8-a249-a1afd0791495\") "
Sep 10 15:42:10.072543 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:42:10.072348    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hcd6f\" (UniqueName: \"kubernetes.io/projected/71dc332d-0da7-480e-8757-43821adc9365-kube-api-access-hcd6f\") pod \"71dc332d-0da7-480e-8757-43821adc9365\" (UID: \"71dc332d-0da7-480e-8757-43821adc9365\") "
Sep 10 15:42:10.072543 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:42:10.072395    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/eb3192ca-7107-41a8-a249-a1afd0791495-openshift-service-ca\") pod \"eb3192ca-7107-41a8-a249-a1afd0791495\" (UID: \"eb3192ca-7107-41a8-a249-a1afd0791495\") "
Sep 10 15:42:10.072543 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:42:10.072418    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/eb3192ca-7107-41a8-a249-a1afd0791495-metrics-certs\") pod \"eb3192ca-7107-41a8-a249-a1afd0791495\" (UID: \"eb3192ca-7107-41a8-a249-a1afd0791495\") "
Sep 10 15:42:10.072543 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:42:10.072444    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/37106482-a471-4b7e-b641-2a4bd544cf72-plugins-config-volume\") pod \"37106482-a471-4b7e-b641-2a4bd544cf72\" (UID: \"37106482-a471-4b7e-b641-2a4bd544cf72\") "
Sep 10 15:42:10.072543 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:42:10.072472    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/71dc332d-0da7-480e-8757-43821adc9365-plugins-config-volume\") pod \"71dc332d-0da7-480e-8757-43821adc9365\" (UID: \"71dc332d-0da7-480e-8757-43821adc9365\") "
Sep 10 15:42:10.073110 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:42:10.073082    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/71dc332d-0da7-480e-8757-43821adc9365-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "71dc332d-0da7-480e-8757-43821adc9365" (UID: "71dc332d-0da7-480e-8757-43821adc9365"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 15:42:10.073214 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:42:10.073101    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/eb3192ca-7107-41a8-a249-a1afd0791495-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "eb3192ca-7107-41a8-a249-a1afd0791495" (UID: "eb3192ca-7107-41a8-a249-a1afd0791495"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 15:42:10.073214 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:42:10.073082    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/37106482-a471-4b7e-b641-2a4bd544cf72-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "37106482-a471-4b7e-b641-2a4bd544cf72" (UID: "37106482-a471-4b7e-b641-2a4bd544cf72"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 15:42:10.074829 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:42:10.074797    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eb3192ca-7107-41a8-a249-a1afd0791495-kube-api-access-sgf6f" (OuterVolumeSpecName: "kube-api-access-sgf6f") pod "eb3192ca-7107-41a8-a249-a1afd0791495" (UID: "eb3192ca-7107-41a8-a249-a1afd0791495"). InnerVolumeSpecName "kube-api-access-sgf6f". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 15:42:10.074989 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:42:10.074962    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/eb3192ca-7107-41a8-a249-a1afd0791495-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "eb3192ca-7107-41a8-a249-a1afd0791495" (UID: "eb3192ca-7107-41a8-a249-a1afd0791495"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 15:42:10.075346 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:42:10.075324    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/eb3192ca-7107-41a8-a249-a1afd0791495-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "eb3192ca-7107-41a8-a249-a1afd0791495" (UID: "eb3192ca-7107-41a8-a249-a1afd0791495"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 15:42:10.075490 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:42:10.075467    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/71dc332d-0da7-480e-8757-43821adc9365-kube-api-access-hcd6f" (OuterVolumeSpecName: "kube-api-access-hcd6f") pod "71dc332d-0da7-480e-8757-43821adc9365" (UID: "71dc332d-0da7-480e-8757-43821adc9365"). InnerVolumeSpecName "kube-api-access-hcd6f". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 15:42:10.075541 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:42:10.075499    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/37106482-a471-4b7e-b641-2a4bd544cf72-kube-api-access-xvwbl" (OuterVolumeSpecName: "kube-api-access-xvwbl") pod "37106482-a471-4b7e-b641-2a4bd544cf72" (UID: "37106482-a471-4b7e-b641-2a4bd544cf72"). InnerVolumeSpecName "kube-api-access-xvwbl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 15:42:10.173213 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:42:10.173177    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sgf6f\" (UniqueName: \"kubernetes.io/projected/eb3192ca-7107-41a8-a249-a1afd0791495-kube-api-access-sgf6f\") on node \"ip-10-0-128-241.ec2.internal\" DevicePath \"\""
Sep 10 15:42:10.173213 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:42:10.173206    2567 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/eb3192ca-7107-41a8-a249-a1afd0791495-maas-api-tls\") on node \"ip-10-0-128-241.ec2.internal\" DevicePath \"\""
Sep 10 15:42:10.173213 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:42:10.173217    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hcd6f\" (UniqueName: \"kubernetes.io/projected/71dc332d-0da7-480e-8757-43821adc9365-kube-api-access-hcd6f\") on node \"ip-10-0-128-241.ec2.internal\" DevicePath \"\""
Sep 10 15:42:10.173466 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:42:10.173225    2567 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/eb3192ca-7107-41a8-a249-a1afd0791495-openshift-service-ca\") on node \"ip-10-0-128-241.ec2.internal\" DevicePath \"\""
Sep 10 15:42:10.173466 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:42:10.173246    2567 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/eb3192ca-7107-41a8-a249-a1afd0791495-metrics-certs\") on node \"ip-10-0-128-241.ec2.internal\" DevicePath \"\""
Sep 10 15:42:10.173466 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:42:10.173256    2567 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/37106482-a471-4b7e-b641-2a4bd544cf72-plugins-config-volume\") on node \"ip-10-0-128-241.ec2.internal\" DevicePath \"\""
Sep 10 15:42:10.173466 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:42:10.173266    2567 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/71dc332d-0da7-480e-8757-43821adc9365-plugins-config-volume\") on node \"ip-10-0-128-241.ec2.internal\" DevicePath \"\""
Sep 10 15:42:10.173466 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:42:10.173274    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xvwbl\" (UniqueName: \"kubernetes.io/projected/37106482-a471-4b7e-b641-2a4bd544cf72-kube-api-access-xvwbl\") on node \"ip-10-0-128-241.ec2.internal\" DevicePath \"\""
Sep 10 15:42:10.805708 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:42:10.805680    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-7a711aa8-6679cb8df4-slfdf"
Sep 10 15:42:10.805908 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:42:10.805726    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-7a711aa8-6679cb8df4-slfdf" event={"ID":"71dc332d-0da7-480e-8757-43821adc9365","Type":"ContainerDied","Data":"979ed9273dc8446aa985b24f3f8e25e8cda4f33b361c1a4925f511e06f76181c"}
Sep 10 15:42:10.805908 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:42:10.805784    2567 scope.go:117] "RemoveContainer" containerID="6f99a8e03cd843589aca79bab3ad1b16c5cbe647f0f8bed1155a6c19050a4ffd"
Sep 10 15:42:10.807214 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:42:10.807194    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-7a711aa8-5cf84dd5df-64g8h"
Sep 10 15:42:10.807323 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:42:10.807201    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-7a711aa8-5cf84dd5df-64g8h" event={"ID":"37106482-a471-4b7e-b641-2a4bd544cf72","Type":"ContainerDied","Data":"3b9a31c374cab6d4377b34bc1a81a8655759a1aa3f514a249d3cd874fa9e01d5"}
Sep 10 15:42:10.808847 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:42:10.808817    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-7a711aa8-59ccd69dfc-47t9r" event={"ID":"eb3192ca-7107-41a8-a249-a1afd0791495","Type":"ContainerDied","Data":"15a17efe1cd7fa057c32fe9c4a566e8d53ab44c8f23bda3e796b3d812619a6a2"}
Sep 10 15:42:10.808847 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:42:10.808847    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-7a711aa8-59ccd69dfc-47t9r"
Sep 10 15:42:10.818390 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:42:10.818258    2567 scope.go:117] "RemoveContainer" containerID="e8bc1efa88a290a7ea686606c3a3c8d26e3ccdb39a2c9531a3280a690ee706f2"
Sep 10 15:42:10.827426 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:42:10.827408    2567 scope.go:117] "RemoveContainer" containerID="d8a5c19d2b5ca9e50120fa7730585fc5624d3555bb3c181a690b0e962a4bbb39"
Sep 10 15:42:10.840331 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:42:10.840302    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-7a711aa8-5cf84dd5df-64g8h"]
Sep 10 15:42:10.849385 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:42:10.849359    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-7a711aa8-5cf84dd5df-64g8h"]
Sep 10 15:42:10.863187 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:42:10.863159    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-7a711aa8-6679cb8df4-slfdf"]
Sep 10 15:42:10.871703 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:42:10.871673    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-7a711aa8-6679cb8df4-slfdf"]
Sep 10 15:42:10.882880 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:42:10.882853    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-7a711aa8-59ccd69dfc-47t9r"]
Sep 10 15:42:10.890905 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:42:10.890877    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-7a711aa8-59ccd69dfc-47t9r"]
Sep 10 15:42:10.905374 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:42:10.905346    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="37106482-a471-4b7e-b641-2a4bd544cf72" path="/var/lib/kubelet/pods/37106482-a471-4b7e-b641-2a4bd544cf72/volumes"
Sep 10 15:42:10.905798 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:42:10.905778    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="71dc332d-0da7-480e-8757-43821adc9365" path="/var/lib/kubelet/pods/71dc332d-0da7-480e-8757-43821adc9365/volumes"
Sep 10 15:42:10.906223 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:42:10.906207    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="eb3192ca-7107-41a8-a249-a1afd0791495" path="/var/lib/kubelet/pods/eb3192ca-7107-41a8-a249-a1afd0791495/volumes"
Sep 10 15:42:11.085669 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:42:11.085586    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:42:11.196946 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:42:11.196914    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:42:11.557826 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:42:11.557788    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:42:11.645921 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:42:11.645877    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:42:12.454061 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:42:12.454026    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:42:12.772643 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:42:12.772555    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:42:14.594683 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:42:14.594647    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:42:14.696948 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:42:14.696910    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:42:15.316167 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:42:15.316133    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:42:16.121475 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:42:16.121433    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:42:16.222473 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:42:16.222435    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:42:16.321480 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:42:16.321445    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:42:16.725868 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:42:16.725831    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:42:16.822928 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:42:16.822899    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:42:16.870690 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:42:16.870654    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-f89c4f3a-56cdb7b74-r522s"]
Sep 10 15:42:16.871085 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:42:16.871059    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="37106482-a471-4b7e-b641-2a4bd544cf72" containerName="payload-processing"
Sep 10 15:42:16.871085 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:42:16.871083    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="37106482-a471-4b7e-b641-2a4bd544cf72" containerName="payload-processing"
Sep 10 15:42:16.871292 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:42:16.871121    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="71dc332d-0da7-480e-8757-43821adc9365" containerName="payload-pre-processing"
Sep 10 15:42:16.871292 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:42:16.871131    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="71dc332d-0da7-480e-8757-43821adc9365" containerName="payload-pre-processing"
Sep 10 15:42:16.871292 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:42:16.871152    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="eb3192ca-7107-41a8-a249-a1afd0791495" containerName="maas-api"
Sep 10 15:42:16.871292 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:42:16.871161    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="eb3192ca-7107-41a8-a249-a1afd0791495" containerName="maas-api"
Sep 10 15:42:16.871292 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:42:16.871251    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="71dc332d-0da7-480e-8757-43821adc9365" containerName="payload-pre-processing"
Sep 10 15:42:16.871292 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:42:16.871259    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="37106482-a471-4b7e-b641-2a4bd544cf72" containerName="payload-processing"
Sep 10 15:42:16.871292 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:42:16.871268    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="eb3192ca-7107-41a8-a249-a1afd0791495" containerName="maas-api"
Sep 10 15:42:16.875567 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:42:16.875552    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-f89c4f3a-56cdb7b74-r522s"
Sep 10 15:42:16.877845 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:42:16.877824    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-migrate-f89c4f3a\""
Sep 10 15:42:16.877945 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:42:16.877891    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-migrate-f89c4f3a-dockercfg-l2gnt\""
Sep 10 15:42:16.888213 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:42:16.888177    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-f89c4f3a-56cdb7b74-r522s"]
Sep 10 15:42:16.915645 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:42:16.915605    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-f89c4f3a-67df946f46-5nb6t"]
Sep 10 15:42:16.919307 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:42:16.919283    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-f89c4f3a-67df946f46-5nb6t"
Sep 10 15:42:16.933219 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:42:16.933188    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-f89c4f3a-67df946f46-5nb6t"]
Sep 10 15:42:17.031773 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:42:17.031683    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sdp47\" (UniqueName: \"kubernetes.io/projected/283d5a52-97d0-4c1a-8d4e-05c1c9aa592e-kube-api-access-sdp47\") pod \"payload-pre-processing-e2e-migrate-f89c4f3a-67df946f46-5nb6t\" (UID: \"283d5a52-97d0-4c1a-8d4e-05c1c9aa592e\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-f89c4f3a-67df946f46-5nb6t"
Sep 10 15:42:17.031773 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:42:17.031746    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/283d5a52-97d0-4c1a-8d4e-05c1c9aa592e-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-f89c4f3a-67df946f46-5nb6t\" (UID: \"283d5a52-97d0-4c1a-8d4e-05c1c9aa592e\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-f89c4f3a-67df946f46-5nb6t"
Sep 10 15:42:17.031966 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:42:17.031815    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/69ac2a3c-12b7-4e49-9ae7-11025fdc236c-plugins-config-volume\") pod \"payload-processing-e2e-migrate-f89c4f3a-56cdb7b74-r522s\" (UID: \"69ac2a3c-12b7-4e49-9ae7-11025fdc236c\") " pod="openshift-ingress/payload-processing-e2e-migrate-f89c4f3a-56cdb7b74-r522s"
Sep 10 15:42:17.031966 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:42:17.031872    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s275d\" (UniqueName: \"kubernetes.io/projected/69ac2a3c-12b7-4e49-9ae7-11025fdc236c-kube-api-access-s275d\") pod \"payload-processing-e2e-migrate-f89c4f3a-56cdb7b74-r522s\" (UID: \"69ac2a3c-12b7-4e49-9ae7-11025fdc236c\") " pod="openshift-ingress/payload-processing-e2e-migrate-f89c4f3a-56cdb7b74-r522s"
Sep 10 15:42:17.133215 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:42:17.133174    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sdp47\" (UniqueName: \"kubernetes.io/projected/283d5a52-97d0-4c1a-8d4e-05c1c9aa592e-kube-api-access-sdp47\") pod \"payload-pre-processing-e2e-migrate-f89c4f3a-67df946f46-5nb6t\" (UID: \"283d5a52-97d0-4c1a-8d4e-05c1c9aa592e\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-f89c4f3a-67df946f46-5nb6t"
Sep 10 15:42:17.133784 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:42:17.133757    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/283d5a52-97d0-4c1a-8d4e-05c1c9aa592e-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-f89c4f3a-67df946f46-5nb6t\" (UID: \"283d5a52-97d0-4c1a-8d4e-05c1c9aa592e\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-f89c4f3a-67df946f46-5nb6t"
Sep 10 15:42:17.133945 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:42:17.133930    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/69ac2a3c-12b7-4e49-9ae7-11025fdc236c-plugins-config-volume\") pod \"payload-processing-e2e-migrate-f89c4f3a-56cdb7b74-r522s\" (UID: \"69ac2a3c-12b7-4e49-9ae7-11025fdc236c\") " pod="openshift-ingress/payload-processing-e2e-migrate-f89c4f3a-56cdb7b74-r522s"
Sep 10 15:42:17.134058 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:42:17.134043    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s275d\" (UniqueName: \"kubernetes.io/projected/69ac2a3c-12b7-4e49-9ae7-11025fdc236c-kube-api-access-s275d\") pod \"payload-processing-e2e-migrate-f89c4f3a-56cdb7b74-r522s\" (UID: \"69ac2a3c-12b7-4e49-9ae7-11025fdc236c\") " pod="openshift-ingress/payload-processing-e2e-migrate-f89c4f3a-56cdb7b74-r522s"
Sep 10 15:42:17.135563 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:42:17.135536    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/69ac2a3c-12b7-4e49-9ae7-11025fdc236c-plugins-config-volume\") pod \"payload-processing-e2e-migrate-f89c4f3a-56cdb7b74-r522s\" (UID: \"69ac2a3c-12b7-4e49-9ae7-11025fdc236c\") " pod="openshift-ingress/payload-processing-e2e-migrate-f89c4f3a-56cdb7b74-r522s"
Sep 10 15:42:17.135670 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:42:17.135536    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/283d5a52-97d0-4c1a-8d4e-05c1c9aa592e-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-f89c4f3a-67df946f46-5nb6t\" (UID: \"283d5a52-97d0-4c1a-8d4e-05c1c9aa592e\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-f89c4f3a-67df946f46-5nb6t"
Sep 10 15:42:17.144126 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:42:17.144102    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s275d\" (UniqueName: \"kubernetes.io/projected/69ac2a3c-12b7-4e49-9ae7-11025fdc236c-kube-api-access-s275d\") pod \"payload-processing-e2e-migrate-f89c4f3a-56cdb7b74-r522s\" (UID: \"69ac2a3c-12b7-4e49-9ae7-11025fdc236c\") " pod="openshift-ingress/payload-processing-e2e-migrate-f89c4f3a-56cdb7b74-r522s"
Sep 10 15:42:17.144527 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:42:17.144507    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sdp47\" (UniqueName: \"kubernetes.io/projected/283d5a52-97d0-4c1a-8d4e-05c1c9aa592e-kube-api-access-sdp47\") pod \"payload-pre-processing-e2e-migrate-f89c4f3a-67df946f46-5nb6t\" (UID: \"283d5a52-97d0-4c1a-8d4e-05c1c9aa592e\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-f89c4f3a-67df946f46-5nb6t"
Sep 10 15:42:17.190564 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:42:17.190529    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-f89c4f3a-56cdb7b74-r522s"
Sep 10 15:42:17.229715 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:42:17.229674    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-f89c4f3a-67df946f46-5nb6t"
Sep 10 15:42:17.359100 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:42:17.359067    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:42:17.467910 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:42:17.467876    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-f89c4f3a-56cdb7b74-r522s"]
Sep 10 15:42:17.470923 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:42:17.470881    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod69ac2a3c_12b7_4e49_9ae7_11025fdc236c.slice/crio-21f8fd16cabc115098c89857498bff3d7ab43355a9a0f5056f267ebf6459ed77 WatchSource:0}: Error finding container 21f8fd16cabc115098c89857498bff3d7ab43355a9a0f5056f267ebf6459ed77: Status 404 returned error can't find the container with id 21f8fd16cabc115098c89857498bff3d7ab43355a9a0f5056f267ebf6459ed77
Sep 10 15:42:17.515149 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:42:17.515124    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-f89c4f3a-67df946f46-5nb6t"]
Sep 10 15:42:17.517080 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:42:17.517055    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod283d5a52_97d0_4c1a_8d4e_05c1c9aa592e.slice/crio-597da191abdf597a48e714380eb14d7c052f662d66184a8433fc5ae490064691 WatchSource:0}: Error finding container 597da191abdf597a48e714380eb14d7c052f662d66184a8433fc5ae490064691: Status 404 returned error can't find the container with id 597da191abdf597a48e714380eb14d7c052f662d66184a8433fc5ae490064691
Sep 10 15:42:17.848002 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:42:17.847907    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-f89c4f3a-67df946f46-5nb6t" event={"ID":"283d5a52-97d0-4c1a-8d4e-05c1c9aa592e","Type":"ContainerStarted","Data":"3f5b1bd966994170d53d1063e451ca0e021eb89393da2bd220fbc6da4c840445"}
Sep 10 15:42:17.848002 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:42:17.847967    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-f89c4f3a-67df946f46-5nb6t" event={"ID":"283d5a52-97d0-4c1a-8d4e-05c1c9aa592e","Type":"ContainerStarted","Data":"597da191abdf597a48e714380eb14d7c052f662d66184a8433fc5ae490064691"}
Sep 10 15:42:17.848002 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:42:17.847997    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-migrate-f89c4f3a-67df946f46-5nb6t"
Sep 10 15:42:17.849829 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:42:17.849804    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-f89c4f3a-56cdb7b74-r522s" event={"ID":"69ac2a3c-12b7-4e49-9ae7-11025fdc236c","Type":"ContainerStarted","Data":"065dd80bbef7e2b81292c6b3658e781d9e3a11cfec8d4df5c1863a971ced7642"}
Sep 10 15:42:17.849952 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:42:17.849851    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-migrate-f89c4f3a-56cdb7b74-r522s"
Sep 10 15:42:17.849952 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:42:17.849874    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-f89c4f3a-56cdb7b74-r522s" event={"ID":"69ac2a3c-12b7-4e49-9ae7-11025fdc236c","Type":"ContainerStarted","Data":"21f8fd16cabc115098c89857498bff3d7ab43355a9a0f5056f267ebf6459ed77"}
Sep 10 15:42:17.868281 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:42:17.868214    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-migrate-f89c4f3a-67df946f46-5nb6t" podStartSLOduration=1.868197373 podStartE2EDuration="1.868197373s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 15:42:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 15:42:17.866761888 +0000 UTC m=+923.434993621" watchObservedRunningTime="2026-09-10 15:42:17.868197373 +0000 UTC m=+923.436429102"
Sep 10 15:42:17.884807 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:42:17.884751    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-migrate-f89c4f3a-56cdb7b74-r522s" podStartSLOduration=1.884733558 podStartE2EDuration="1.884733558s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 15:42:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 15:42:17.882785235 +0000 UTC m=+923.451016966" watchObservedRunningTime="2026-09-10 15:42:17.884733558 +0000 UTC m=+923.452965287"
Sep 10 15:42:18.239620 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:42:18.239585    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:42:19.167386 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:42:19.167352    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:42:28.857190 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:42:28.857142    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-migrate-f89c4f3a-67df946f46-5nb6t"
Sep 10 15:42:28.857695 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:42:28.857254    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-migrate-f89c4f3a-56cdb7b74-r522s"
Sep 10 15:42:34.195998 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:42:34.195967    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:42:37.177518 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:42:37.177469    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:42:38.788121 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:42:38.788083    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:42:39.749489 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:42:39.749453    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:42:40.604483 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:42:40.604450    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:42:41.558630 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:42:41.558590    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:42:41.994611 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:42:41.994561    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:42:42.087424 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:42:42.087387    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:42:42.883224 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:42:42.883190    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:42:43.616742 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:42:43.616714    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:42:44.419909 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:42:44.419870    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:42:44.454432 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:42:44.454398    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-f89c4f3a-56cdb7b74-r522s"]
Sep 10 15:42:44.454720 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:42:44.454691    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-migrate-f89c4f3a-56cdb7b74-r522s" podUID="69ac2a3c-12b7-4e49-9ae7-11025fdc236c" containerName="payload-processing" containerID="cri-o://065dd80bbef7e2b81292c6b3658e781d9e3a11cfec8d4df5c1863a971ced7642" gracePeriod=30
Sep 10 15:42:44.463450 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:42:44.463400    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-f89c4f3a-67df946f46-5nb6t"]
Sep 10 15:42:44.463722 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:42:44.463665    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-migrate-f89c4f3a-67df946f46-5nb6t" podUID="283d5a52-97d0-4c1a-8d4e-05c1c9aa592e" containerName="payload-pre-processing" containerID="cri-o://3f5b1bd966994170d53d1063e451ca0e021eb89393da2bd220fbc6da4c840445" gracePeriod=30
Sep 10 15:42:44.499607 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:42:44.499574    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:42:44.569370 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:42:44.569337    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:42:44.676139 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:42:44.676048    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:42:44.737138 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:42:44.737102    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:42:44.738088 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:42:44.738069    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-f89c4f3a-56cdb7b74-r522s"
Sep 10 15:42:44.741084 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:42:44.741065    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-f89c4f3a-67df946f46-5nb6t"
Sep 10 15:42:44.766737 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:42:44.766145    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-s275d\" (UniqueName: \"kubernetes.io/projected/69ac2a3c-12b7-4e49-9ae7-11025fdc236c-kube-api-access-s275d\") pod \"69ac2a3c-12b7-4e49-9ae7-11025fdc236c\" (UID: \"69ac2a3c-12b7-4e49-9ae7-11025fdc236c\") "
Sep 10 15:42:44.766737 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:42:44.766312    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sdp47\" (UniqueName: \"kubernetes.io/projected/283d5a52-97d0-4c1a-8d4e-05c1c9aa592e-kube-api-access-sdp47\") pod \"283d5a52-97d0-4c1a-8d4e-05c1c9aa592e\" (UID: \"283d5a52-97d0-4c1a-8d4e-05c1c9aa592e\") "
Sep 10 15:42:44.766737 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:42:44.766345    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/69ac2a3c-12b7-4e49-9ae7-11025fdc236c-plugins-config-volume\") pod \"69ac2a3c-12b7-4e49-9ae7-11025fdc236c\" (UID: \"69ac2a3c-12b7-4e49-9ae7-11025fdc236c\") "
Sep 10 15:42:44.766737 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:42:44.766425    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/283d5a52-97d0-4c1a-8d4e-05c1c9aa592e-plugins-config-volume\") pod \"283d5a52-97d0-4c1a-8d4e-05c1c9aa592e\" (UID: \"283d5a52-97d0-4c1a-8d4e-05c1c9aa592e\") "
Sep 10 15:42:44.767382 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:42:44.767046    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/69ac2a3c-12b7-4e49-9ae7-11025fdc236c-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "69ac2a3c-12b7-4e49-9ae7-11025fdc236c" (UID: "69ac2a3c-12b7-4e49-9ae7-11025fdc236c"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 15:42:44.767382 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:42:44.767129    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/283d5a52-97d0-4c1a-8d4e-05c1c9aa592e-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "283d5a52-97d0-4c1a-8d4e-05c1c9aa592e" (UID: "283d5a52-97d0-4c1a-8d4e-05c1c9aa592e"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 15:42:44.768686 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:42:44.768651    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/283d5a52-97d0-4c1a-8d4e-05c1c9aa592e-kube-api-access-sdp47" (OuterVolumeSpecName: "kube-api-access-sdp47") pod "283d5a52-97d0-4c1a-8d4e-05c1c9aa592e" (UID: "283d5a52-97d0-4c1a-8d4e-05c1c9aa592e"). InnerVolumeSpecName "kube-api-access-sdp47". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 15:42:44.769050 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:42:44.769033    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/69ac2a3c-12b7-4e49-9ae7-11025fdc236c-kube-api-access-s275d" (OuterVolumeSpecName: "kube-api-access-s275d") pod "69ac2a3c-12b7-4e49-9ae7-11025fdc236c" (UID: "69ac2a3c-12b7-4e49-9ae7-11025fdc236c"). InnerVolumeSpecName "kube-api-access-s275d". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 15:42:44.830797 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:42:44.830765    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:42:44.867645 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:42:44.867609    2567 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/283d5a52-97d0-4c1a-8d4e-05c1c9aa592e-plugins-config-volume\") on node \"ip-10-0-128-241.ec2.internal\" DevicePath \"\""
Sep 10 15:42:44.867645 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:42:44.867639    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s275d\" (UniqueName: \"kubernetes.io/projected/69ac2a3c-12b7-4e49-9ae7-11025fdc236c-kube-api-access-s275d\") on node \"ip-10-0-128-241.ec2.internal\" DevicePath \"\""
Sep 10 15:42:44.867645 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:42:44.867650    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sdp47\" (UniqueName: \"kubernetes.io/projected/283d5a52-97d0-4c1a-8d4e-05c1c9aa592e-kube-api-access-sdp47\") on node \"ip-10-0-128-241.ec2.internal\" DevicePath \"\""
Sep 10 15:42:44.867875 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:42:44.867659    2567 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/69ac2a3c-12b7-4e49-9ae7-11025fdc236c-plugins-config-volume\") on node \"ip-10-0-128-241.ec2.internal\" DevicePath \"\""
Sep 10 15:42:44.983296 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:42:44.983197    2567 generic.go:356] "Generic (PLEG): container finished" podID="69ac2a3c-12b7-4e49-9ae7-11025fdc236c" containerID="065dd80bbef7e2b81292c6b3658e781d9e3a11cfec8d4df5c1863a971ced7642" exitCode=0
Sep 10 15:42:44.983296 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:42:44.983258    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-f89c4f3a-56cdb7b74-r522s"
Sep 10 15:42:44.983296 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:42:44.983266    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-f89c4f3a-56cdb7b74-r522s" event={"ID":"69ac2a3c-12b7-4e49-9ae7-11025fdc236c","Type":"ContainerDied","Data":"065dd80bbef7e2b81292c6b3658e781d9e3a11cfec8d4df5c1863a971ced7642"}
Sep 10 15:42:44.983548 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:42:44.983324    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-f89c4f3a-56cdb7b74-r522s" event={"ID":"69ac2a3c-12b7-4e49-9ae7-11025fdc236c","Type":"ContainerDied","Data":"21f8fd16cabc115098c89857498bff3d7ab43355a9a0f5056f267ebf6459ed77"}
Sep 10 15:42:44.983548 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:42:44.983355    2567 scope.go:117] "RemoveContainer" containerID="065dd80bbef7e2b81292c6b3658e781d9e3a11cfec8d4df5c1863a971ced7642"
Sep 10 15:42:44.984775 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:42:44.984749    2567 generic.go:356] "Generic (PLEG): container finished" podID="283d5a52-97d0-4c1a-8d4e-05c1c9aa592e" containerID="3f5b1bd966994170d53d1063e451ca0e021eb89393da2bd220fbc6da4c840445" exitCode=0
Sep 10 15:42:44.984901 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:42:44.984791    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-f89c4f3a-67df946f46-5nb6t"
Sep 10 15:42:44.984901 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:42:44.984806    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-f89c4f3a-67df946f46-5nb6t" event={"ID":"283d5a52-97d0-4c1a-8d4e-05c1c9aa592e","Type":"ContainerDied","Data":"3f5b1bd966994170d53d1063e451ca0e021eb89393da2bd220fbc6da4c840445"}
Sep 10 15:42:44.984901 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:42:44.984839    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-f89c4f3a-67df946f46-5nb6t" event={"ID":"283d5a52-97d0-4c1a-8d4e-05c1c9aa592e","Type":"ContainerDied","Data":"597da191abdf597a48e714380eb14d7c052f662d66184a8433fc5ae490064691"}
Sep 10 15:42:44.992541 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:42:44.992522    2567 scope.go:117] "RemoveContainer" containerID="065dd80bbef7e2b81292c6b3658e781d9e3a11cfec8d4df5c1863a971ced7642"
Sep 10 15:42:44.992864 ip-10-0-128-241 kubenswrapper[2567]: E0910 15:42:44.992844    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"065dd80bbef7e2b81292c6b3658e781d9e3a11cfec8d4df5c1863a971ced7642\": container with ID starting with 065dd80bbef7e2b81292c6b3658e781d9e3a11cfec8d4df5c1863a971ced7642 not found: ID does not exist" containerID="065dd80bbef7e2b81292c6b3658e781d9e3a11cfec8d4df5c1863a971ced7642"
Sep 10 15:42:44.992925 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:42:44.992874    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"065dd80bbef7e2b81292c6b3658e781d9e3a11cfec8d4df5c1863a971ced7642"} err="failed to get container status \"065dd80bbef7e2b81292c6b3658e781d9e3a11cfec8d4df5c1863a971ced7642\": rpc error: code = NotFound desc = could not find container \"065dd80bbef7e2b81292c6b3658e781d9e3a11cfec8d4df5c1863a971ced7642\": container with ID starting with 065dd80bbef7e2b81292c6b3658e781d9e3a11cfec8d4df5c1863a971ced7642 not found: ID does not exist"
Sep 10 15:42:44.992925 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:42:44.992894    2567 scope.go:117] "RemoveContainer" containerID="3f5b1bd966994170d53d1063e451ca0e021eb89393da2bd220fbc6da4c840445"
Sep 10 15:42:45.000905 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:42:45.000886    2567 scope.go:117] "RemoveContainer" containerID="3f5b1bd966994170d53d1063e451ca0e021eb89393da2bd220fbc6da4c840445"
Sep 10 15:42:45.001211 ip-10-0-128-241 kubenswrapper[2567]: E0910 15:42:45.001189    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3f5b1bd966994170d53d1063e451ca0e021eb89393da2bd220fbc6da4c840445\": container with ID starting with 3f5b1bd966994170d53d1063e451ca0e021eb89393da2bd220fbc6da4c840445 not found: ID does not exist" containerID="3f5b1bd966994170d53d1063e451ca0e021eb89393da2bd220fbc6da4c840445"
Sep 10 15:42:45.001304 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:42:45.001218    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3f5b1bd966994170d53d1063e451ca0e021eb89393da2bd220fbc6da4c840445"} err="failed to get container status \"3f5b1bd966994170d53d1063e451ca0e021eb89393da2bd220fbc6da4c840445\": rpc error: code = NotFound desc = could not find container \"3f5b1bd966994170d53d1063e451ca0e021eb89393da2bd220fbc6da4c840445\": container with ID starting with 3f5b1bd966994170d53d1063e451ca0e021eb89393da2bd220fbc6da4c840445 not found: ID does not exist"
Sep 10 15:42:45.004653 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:42:45.004630    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-f89c4f3a-56cdb7b74-r522s"]
Sep 10 15:42:45.009966 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:42:45.009944    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-f89c4f3a-56cdb7b74-r522s"]
Sep 10 15:42:45.022362 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:42:45.022336    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-f89c4f3a-67df946f46-5nb6t"]
Sep 10 15:42:45.032623 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:42:45.032592    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-f89c4f3a-67df946f46-5nb6t"]
Sep 10 15:42:45.148350 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:42:45.148319    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:42:45.296582 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:42:45.296501    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:42:45.638634 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:42:45.638553    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:42:46.040703 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:42:46.040669    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:42:46.364510 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:42:46.364423    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:42:46.903954 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:42:46.903918    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="283d5a52-97d0-4c1a-8d4e-05c1c9aa592e" path="/var/lib/kubelet/pods/283d5a52-97d0-4c1a-8d4e-05c1c9aa592e/volumes"
Sep 10 15:42:46.904310 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:42:46.904295    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="69ac2a3c-12b7-4e49-9ae7-11025fdc236c" path="/var/lib/kubelet/pods/69ac2a3c-12b7-4e49-9ae7-11025fdc236c/volumes"
Sep 10 15:42:47.429653 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:42:47.429614    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:42:47.938493 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:42:47.938452    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:42:48.050581 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:42:48.050524    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:42:48.197469 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:42:48.197373    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:42:49.166931 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:42:49.166895    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:42:50.396349 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:42:50.396314    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:42:50.922967 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:42:50.922935    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-bd8496a8-6b56dd7bfb-ndl8d"]
Sep 10 15:42:50.923699 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:42:50.923676    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="69ac2a3c-12b7-4e49-9ae7-11025fdc236c" containerName="payload-processing"
Sep 10 15:42:50.923699 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:42:50.923698    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="69ac2a3c-12b7-4e49-9ae7-11025fdc236c" containerName="payload-processing"
Sep 10 15:42:50.923851 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:42:50.923712    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="283d5a52-97d0-4c1a-8d4e-05c1c9aa592e" containerName="payload-pre-processing"
Sep 10 15:42:50.923851 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:42:50.923721    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="283d5a52-97d0-4c1a-8d4e-05c1c9aa592e" containerName="payload-pre-processing"
Sep 10 15:42:50.923851 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:42:50.923839    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="69ac2a3c-12b7-4e49-9ae7-11025fdc236c" containerName="payload-processing"
Sep 10 15:42:50.923973 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:42:50.923857    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="283d5a52-97d0-4c1a-8d4e-05c1c9aa592e" containerName="payload-pre-processing"
Sep 10 15:42:50.928816 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:42:50.928788    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-bd8496a8-6b56dd7bfb-ndl8d"
Sep 10 15:42:50.951684 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:42:50.950655    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-bd8496a8-serving-cert\""
Sep 10 15:42:50.954805 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:42:50.954766    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-bd8496a8-6b56dd7bfb-ndl8d"]
Sep 10 15:42:50.968688 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:42:50.968651    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:42:51.019742 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:42:51.019696    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c5c2b5fb-bdcb-4a72-9a59-015a9e94bd7e-maas-api-tls\") pod \"maas-api-e2e-ait-bd8496a8-6b56dd7bfb-ndl8d\" (UID: \"c5c2b5fb-bdcb-4a72-9a59-015a9e94bd7e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-bd8496a8-6b56dd7bfb-ndl8d"
Sep 10 15:42:51.019918 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:42:51.019761    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c5c2b5fb-bdcb-4a72-9a59-015a9e94bd7e-openshift-service-ca\") pod \"maas-api-e2e-ait-bd8496a8-6b56dd7bfb-ndl8d\" (UID: \"c5c2b5fb-bdcb-4a72-9a59-015a9e94bd7e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-bd8496a8-6b56dd7bfb-ndl8d"
Sep 10 15:42:51.019918 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:42:51.019818    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c5c2b5fb-bdcb-4a72-9a59-015a9e94bd7e-metrics-certs\") pod \"maas-api-e2e-ait-bd8496a8-6b56dd7bfb-ndl8d\" (UID: \"c5c2b5fb-bdcb-4a72-9a59-015a9e94bd7e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-bd8496a8-6b56dd7bfb-ndl8d"
Sep 10 15:42:51.019918 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:42:51.019856    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9gzg5\" (UniqueName: \"kubernetes.io/projected/c5c2b5fb-bdcb-4a72-9a59-015a9e94bd7e-kube-api-access-9gzg5\") pod \"maas-api-e2e-ait-bd8496a8-6b56dd7bfb-ndl8d\" (UID: \"c5c2b5fb-bdcb-4a72-9a59-015a9e94bd7e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-bd8496a8-6b56dd7bfb-ndl8d"
Sep 10 15:42:51.067684 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:42:51.067640    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:42:51.120374 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:42:51.120338    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c5c2b5fb-bdcb-4a72-9a59-015a9e94bd7e-maas-api-tls\") pod \"maas-api-e2e-ait-bd8496a8-6b56dd7bfb-ndl8d\" (UID: \"c5c2b5fb-bdcb-4a72-9a59-015a9e94bd7e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-bd8496a8-6b56dd7bfb-ndl8d"
Sep 10 15:42:51.120374 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:42:51.120385    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c5c2b5fb-bdcb-4a72-9a59-015a9e94bd7e-openshift-service-ca\") pod \"maas-api-e2e-ait-bd8496a8-6b56dd7bfb-ndl8d\" (UID: \"c5c2b5fb-bdcb-4a72-9a59-015a9e94bd7e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-bd8496a8-6b56dd7bfb-ndl8d"
Sep 10 15:42:51.120655 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:42:51.120425    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c5c2b5fb-bdcb-4a72-9a59-015a9e94bd7e-metrics-certs\") pod \"maas-api-e2e-ait-bd8496a8-6b56dd7bfb-ndl8d\" (UID: \"c5c2b5fb-bdcb-4a72-9a59-015a9e94bd7e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-bd8496a8-6b56dd7bfb-ndl8d"
Sep 10 15:42:51.120655 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:42:51.120443    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9gzg5\" (UniqueName: \"kubernetes.io/projected/c5c2b5fb-bdcb-4a72-9a59-015a9e94bd7e-kube-api-access-9gzg5\") pod \"maas-api-e2e-ait-bd8496a8-6b56dd7bfb-ndl8d\" (UID: \"c5c2b5fb-bdcb-4a72-9a59-015a9e94bd7e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-bd8496a8-6b56dd7bfb-ndl8d"
Sep 10 15:42:51.121172 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:42:51.121149    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c5c2b5fb-bdcb-4a72-9a59-015a9e94bd7e-openshift-service-ca\") pod \"maas-api-e2e-ait-bd8496a8-6b56dd7bfb-ndl8d\" (UID: \"c5c2b5fb-bdcb-4a72-9a59-015a9e94bd7e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-bd8496a8-6b56dd7bfb-ndl8d"
Sep 10 15:42:51.123032 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:42:51.123002    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c5c2b5fb-bdcb-4a72-9a59-015a9e94bd7e-metrics-certs\") pod \"maas-api-e2e-ait-bd8496a8-6b56dd7bfb-ndl8d\" (UID: \"c5c2b5fb-bdcb-4a72-9a59-015a9e94bd7e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-bd8496a8-6b56dd7bfb-ndl8d"
Sep 10 15:42:51.123162 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:42:51.123002    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c5c2b5fb-bdcb-4a72-9a59-015a9e94bd7e-maas-api-tls\") pod \"maas-api-e2e-ait-bd8496a8-6b56dd7bfb-ndl8d\" (UID: \"c5c2b5fb-bdcb-4a72-9a59-015a9e94bd7e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-bd8496a8-6b56dd7bfb-ndl8d"
Sep 10 15:42:51.129545 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:42:51.129518    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9gzg5\" (UniqueName: \"kubernetes.io/projected/c5c2b5fb-bdcb-4a72-9a59-015a9e94bd7e-kube-api-access-9gzg5\") pod \"maas-api-e2e-ait-bd8496a8-6b56dd7bfb-ndl8d\" (UID: \"c5c2b5fb-bdcb-4a72-9a59-015a9e94bd7e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-bd8496a8-6b56dd7bfb-ndl8d"
Sep 10 15:42:51.248784 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:42:51.248693    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-bd8496a8-6b56dd7bfb-ndl8d"
Sep 10 15:42:51.356558 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:42:51.356521    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:42:51.533213 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:42:51.533187    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-bd8496a8-6b56dd7bfb-ndl8d"]
Sep 10 15:42:51.535551 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:42:51.535518    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc5c2b5fb_bdcb_4a72_9a59_015a9e94bd7e.slice/crio-f099ac1c7426e4151fc2175ec1321a52ae71362a0880742b9259dcc5da9080ee WatchSource:0}: Error finding container f099ac1c7426e4151fc2175ec1321a52ae71362a0880742b9259dcc5da9080ee: Status 404 returned error can't find the container with id f099ac1c7426e4151fc2175ec1321a52ae71362a0880742b9259dcc5da9080ee
Sep 10 15:42:51.537177 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:42:51.537161    2567 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 10 15:42:51.681757 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:42:51.681721    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:42:51.872106 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:42:51.872027    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-bd8496a8-754fd849b6-5nmmg"]
Sep 10 15:42:51.875567 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:42:51.875548    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-bd8496a8-754fd849b6-5nmmg"
Sep 10 15:42:51.878241 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:42:51.878212    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-bd8496a8\""
Sep 10 15:42:51.878495 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:42:51.878480    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-bd8496a8-dockercfg-d7fmq\""
Sep 10 15:42:51.888354 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:42:51.888314    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-bd8496a8-754fd849b6-5nmmg"]
Sep 10 15:42:51.928354 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:42:51.928299    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c7203fc2-5472-4306-ba88-1946aa28c463-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-bd8496a8-754fd849b6-5nmmg\" (UID: \"c7203fc2-5472-4306-ba88-1946aa28c463\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-bd8496a8-754fd849b6-5nmmg"
Sep 10 15:42:51.928547 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:42:51.928398    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q9whw\" (UniqueName: \"kubernetes.io/projected/c7203fc2-5472-4306-ba88-1946aa28c463-kube-api-access-q9whw\") pod \"payload-pre-processing-e2e-ait-bd8496a8-754fd849b6-5nmmg\" (UID: \"c7203fc2-5472-4306-ba88-1946aa28c463\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-bd8496a8-754fd849b6-5nmmg"
Sep 10 15:42:52.009590 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:42:52.009550    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:42:52.022002 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:42:52.021976    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-bd8496a8-6b56dd7bfb-ndl8d" event={"ID":"c5c2b5fb-bdcb-4a72-9a59-015a9e94bd7e","Type":"ContainerStarted","Data":"f099ac1c7426e4151fc2175ec1321a52ae71362a0880742b9259dcc5da9080ee"}
Sep 10 15:42:52.029953 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:42:52.029848    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c7203fc2-5472-4306-ba88-1946aa28c463-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-bd8496a8-754fd849b6-5nmmg\" (UID: \"c7203fc2-5472-4306-ba88-1946aa28c463\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-bd8496a8-754fd849b6-5nmmg"
Sep 10 15:42:52.029953 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:42:52.029901    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q9whw\" (UniqueName: \"kubernetes.io/projected/c7203fc2-5472-4306-ba88-1946aa28c463-kube-api-access-q9whw\") pod \"payload-pre-processing-e2e-ait-bd8496a8-754fd849b6-5nmmg\" (UID: \"c7203fc2-5472-4306-ba88-1946aa28c463\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-bd8496a8-754fd849b6-5nmmg"
Sep 10 15:42:52.030480 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:42:52.030457    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c7203fc2-5472-4306-ba88-1946aa28c463-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-bd8496a8-754fd849b6-5nmmg\" (UID: \"c7203fc2-5472-4306-ba88-1946aa28c463\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-bd8496a8-754fd849b6-5nmmg"
Sep 10 15:42:52.039418 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:42:52.039387    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q9whw\" (UniqueName: \"kubernetes.io/projected/c7203fc2-5472-4306-ba88-1946aa28c463-kube-api-access-q9whw\") pod \"payload-pre-processing-e2e-ait-bd8496a8-754fd849b6-5nmmg\" (UID: \"c7203fc2-5472-4306-ba88-1946aa28c463\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-bd8496a8-754fd849b6-5nmmg"
Sep 10 15:42:52.186472 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:42:52.186442    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-bd8496a8-754fd849b6-5nmmg"
Sep 10 15:42:52.464889 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:42:52.464859    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-bd8496a8-754fd849b6-5nmmg"]
Sep 10 15:42:52.467049 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:42:52.467019    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc7203fc2_5472_4306_ba88_1946aa28c463.slice/crio-1117000c13563c2f18a7b584a0b1cbf6bc36b3b017c7201ac0a6aec9343de7c1 WatchSource:0}: Error finding container 1117000c13563c2f18a7b584a0b1cbf6bc36b3b017c7201ac0a6aec9343de7c1: Status 404 returned error can't find the container with id 1117000c13563c2f18a7b584a0b1cbf6bc36b3b017c7201ac0a6aec9343de7c1
Sep 10 15:42:53.034634 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:42:53.034597    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-bd8496a8-6b56dd7bfb-ndl8d" event={"ID":"c5c2b5fb-bdcb-4a72-9a59-015a9e94bd7e","Type":"ContainerStarted","Data":"7f64055a9c5f69464ed3f82715d5fce7fce9a595b63a92e733939de1e3023ac3"}
Sep 10 15:42:53.035098 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:42:53.034702    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-bd8496a8-6b56dd7bfb-ndl8d"
Sep 10 15:42:53.036206 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:42:53.036184    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-bd8496a8-754fd849b6-5nmmg" event={"ID":"c7203fc2-5472-4306-ba88-1946aa28c463","Type":"ContainerStarted","Data":"98957ca9041b71aed048695a1bf70299e669216e2ce887b70857ffb6be2b371d"}
Sep 10 15:42:53.036315 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:42:53.036221    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-bd8496a8-754fd849b6-5nmmg" event={"ID":"c7203fc2-5472-4306-ba88-1946aa28c463","Type":"ContainerStarted","Data":"1117000c13563c2f18a7b584a0b1cbf6bc36b3b017c7201ac0a6aec9343de7c1"}
Sep 10 15:42:53.036315 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:42:53.036280    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-bd8496a8-754fd849b6-5nmmg"
Sep 10 15:42:53.056466 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:42:53.056397    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-bd8496a8-6b56dd7bfb-ndl8d" podStartSLOduration=2.577990069 podStartE2EDuration="3.056376252s" totalImagesPullingTime="478.386183ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 15:42:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 15:42:53.056019461 +0000 UTC m=+958.624251193" watchObservedRunningTime="2026-09-10 15:42:53.056376252 +0000 UTC m=+958.624607983"
Sep 10 15:42:53.083741 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:42:53.083683    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ait-bd8496a8-754fd849b6-5nmmg" podStartSLOduration=2.083664608 podStartE2EDuration="2.083664608s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 15:42:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 15:42:53.082046365 +0000 UTC m=+958.650278096" watchObservedRunningTime="2026-09-10 15:42:53.083664608 +0000 UTC m=+958.651896337"
Sep 10 15:42:59.048609 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:42:59.048578    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-bd8496a8-6b56dd7bfb-ndl8d"
Sep 10 15:43:04.045912 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:43:04.045850    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-bd8496a8-754fd849b6-5nmmg"
Sep 10 15:43:07.198829 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:43:07.198795    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-autores-bd5806/autores-model-bd5806-kserve-654584c895-n4dxx"]
Sep 10 15:43:07.203680 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:43:07.203660    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-autores-bd5806/autores-model-bd5806-kserve-654584c895-n4dxx"
Sep 10 15:43:07.207725 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:43:07.207698    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-autores-bd5806\"/\"openshift-service-ca.crt\""
Sep 10 15:43:07.207996 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:43:07.207977    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-autores-bd5806\"/\"default-dockercfg-g2vjd\""
Sep 10 15:43:07.208089 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:43:07.208025    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-autores-bd5806\"/\"kube-root-ca.crt\""
Sep 10 15:43:07.208089 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:43:07.208033    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-autores-bd5806\"/\"autores-model-bd5806-kserve-self-signed-certs\""
Sep 10 15:43:07.217600 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:43:07.217564    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-autores-bd5806/autores-model-bd5806-kserve-654584c895-n4dxx"]
Sep 10 15:43:07.266675 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:43:07.266646    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4f7b2e31-7b8f-4ea5-8c84-49b0ba67d5c8-home\") pod \"autores-model-bd5806-kserve-654584c895-n4dxx\" (UID: \"4f7b2e31-7b8f-4ea5-8c84-49b0ba67d5c8\") " pod="ai-tenant-e2e-autores-bd5806/autores-model-bd5806-kserve-654584c895-n4dxx"
Sep 10 15:43:07.266675 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:43:07.266677    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4f7b2e31-7b8f-4ea5-8c84-49b0ba67d5c8-dshm\") pod \"autores-model-bd5806-kserve-654584c895-n4dxx\" (UID: \"4f7b2e31-7b8f-4ea5-8c84-49b0ba67d5c8\") " pod="ai-tenant-e2e-autores-bd5806/autores-model-bd5806-kserve-654584c895-n4dxx"
Sep 10 15:43:07.266874 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:43:07.266696    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4f7b2e31-7b8f-4ea5-8c84-49b0ba67d5c8-tls-certs\") pod \"autores-model-bd5806-kserve-654584c895-n4dxx\" (UID: \"4f7b2e31-7b8f-4ea5-8c84-49b0ba67d5c8\") " pod="ai-tenant-e2e-autores-bd5806/autores-model-bd5806-kserve-654584c895-n4dxx"
Sep 10 15:43:07.266874 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:43:07.266760    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4f7b2e31-7b8f-4ea5-8c84-49b0ba67d5c8-model-cache\") pod \"autores-model-bd5806-kserve-654584c895-n4dxx\" (UID: \"4f7b2e31-7b8f-4ea5-8c84-49b0ba67d5c8\") " pod="ai-tenant-e2e-autores-bd5806/autores-model-bd5806-kserve-654584c895-n4dxx"
Sep 10 15:43:07.266874 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:43:07.266815    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4f7b2e31-7b8f-4ea5-8c84-49b0ba67d5c8-tmp-dir\") pod \"autores-model-bd5806-kserve-654584c895-n4dxx\" (UID: \"4f7b2e31-7b8f-4ea5-8c84-49b0ba67d5c8\") " pod="ai-tenant-e2e-autores-bd5806/autores-model-bd5806-kserve-654584c895-n4dxx"
Sep 10 15:43:07.266874 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:43:07.266843    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5dmx5\" (UniqueName: \"kubernetes.io/projected/4f7b2e31-7b8f-4ea5-8c84-49b0ba67d5c8-kube-api-access-5dmx5\") pod \"autores-model-bd5806-kserve-654584c895-n4dxx\" (UID: \"4f7b2e31-7b8f-4ea5-8c84-49b0ba67d5c8\") " pod="ai-tenant-e2e-autores-bd5806/autores-model-bd5806-kserve-654584c895-n4dxx"
Sep 10 15:43:07.367878 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:43:07.367836    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4f7b2e31-7b8f-4ea5-8c84-49b0ba67d5c8-home\") pod \"autores-model-bd5806-kserve-654584c895-n4dxx\" (UID: \"4f7b2e31-7b8f-4ea5-8c84-49b0ba67d5c8\") " pod="ai-tenant-e2e-autores-bd5806/autores-model-bd5806-kserve-654584c895-n4dxx"
Sep 10 15:43:07.367878 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:43:07.367878    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4f7b2e31-7b8f-4ea5-8c84-49b0ba67d5c8-dshm\") pod \"autores-model-bd5806-kserve-654584c895-n4dxx\" (UID: \"4f7b2e31-7b8f-4ea5-8c84-49b0ba67d5c8\") " pod="ai-tenant-e2e-autores-bd5806/autores-model-bd5806-kserve-654584c895-n4dxx"
Sep 10 15:43:07.368135 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:43:07.367900    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4f7b2e31-7b8f-4ea5-8c84-49b0ba67d5c8-tls-certs\") pod \"autores-model-bd5806-kserve-654584c895-n4dxx\" (UID: \"4f7b2e31-7b8f-4ea5-8c84-49b0ba67d5c8\") " pod="ai-tenant-e2e-autores-bd5806/autores-model-bd5806-kserve-654584c895-n4dxx"
Sep 10 15:43:07.368135 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:43:07.368022    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4f7b2e31-7b8f-4ea5-8c84-49b0ba67d5c8-model-cache\") pod \"autores-model-bd5806-kserve-654584c895-n4dxx\" (UID: \"4f7b2e31-7b8f-4ea5-8c84-49b0ba67d5c8\") " pod="ai-tenant-e2e-autores-bd5806/autores-model-bd5806-kserve-654584c895-n4dxx"
Sep 10 15:43:07.368135 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:43:07.368074    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4f7b2e31-7b8f-4ea5-8c84-49b0ba67d5c8-tmp-dir\") pod \"autores-model-bd5806-kserve-654584c895-n4dxx\" (UID: \"4f7b2e31-7b8f-4ea5-8c84-49b0ba67d5c8\") " pod="ai-tenant-e2e-autores-bd5806/autores-model-bd5806-kserve-654584c895-n4dxx"
Sep 10 15:43:07.368135 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:43:07.368104    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5dmx5\" (UniqueName: \"kubernetes.io/projected/4f7b2e31-7b8f-4ea5-8c84-49b0ba67d5c8-kube-api-access-5dmx5\") pod \"autores-model-bd5806-kserve-654584c895-n4dxx\" (UID: \"4f7b2e31-7b8f-4ea5-8c84-49b0ba67d5c8\") " pod="ai-tenant-e2e-autores-bd5806/autores-model-bd5806-kserve-654584c895-n4dxx"
Sep 10 15:43:07.368386 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:43:07.368286    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4f7b2e31-7b8f-4ea5-8c84-49b0ba67d5c8-home\") pod \"autores-model-bd5806-kserve-654584c895-n4dxx\" (UID: \"4f7b2e31-7b8f-4ea5-8c84-49b0ba67d5c8\") " pod="ai-tenant-e2e-autores-bd5806/autores-model-bd5806-kserve-654584c895-n4dxx"
Sep 10 15:43:07.368425 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:43:07.368382    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4f7b2e31-7b8f-4ea5-8c84-49b0ba67d5c8-model-cache\") pod \"autores-model-bd5806-kserve-654584c895-n4dxx\" (UID: \"4f7b2e31-7b8f-4ea5-8c84-49b0ba67d5c8\") " pod="ai-tenant-e2e-autores-bd5806/autores-model-bd5806-kserve-654584c895-n4dxx"
Sep 10 15:43:07.368425 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:43:07.368416    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4f7b2e31-7b8f-4ea5-8c84-49b0ba67d5c8-tmp-dir\") pod \"autores-model-bd5806-kserve-654584c895-n4dxx\" (UID: \"4f7b2e31-7b8f-4ea5-8c84-49b0ba67d5c8\") " pod="ai-tenant-e2e-autores-bd5806/autores-model-bd5806-kserve-654584c895-n4dxx"
Sep 10 15:43:07.370189 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:43:07.370156    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4f7b2e31-7b8f-4ea5-8c84-49b0ba67d5c8-dshm\") pod \"autores-model-bd5806-kserve-654584c895-n4dxx\" (UID: \"4f7b2e31-7b8f-4ea5-8c84-49b0ba67d5c8\") " pod="ai-tenant-e2e-autores-bd5806/autores-model-bd5806-kserve-654584c895-n4dxx"
Sep 10 15:43:07.370452 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:43:07.370435    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4f7b2e31-7b8f-4ea5-8c84-49b0ba67d5c8-tls-certs\") pod \"autores-model-bd5806-kserve-654584c895-n4dxx\" (UID: \"4f7b2e31-7b8f-4ea5-8c84-49b0ba67d5c8\") " pod="ai-tenant-e2e-autores-bd5806/autores-model-bd5806-kserve-654584c895-n4dxx"
Sep 10 15:43:07.378694 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:43:07.378668    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5dmx5\" (UniqueName: \"kubernetes.io/projected/4f7b2e31-7b8f-4ea5-8c84-49b0ba67d5c8-kube-api-access-5dmx5\") pod \"autores-model-bd5806-kserve-654584c895-n4dxx\" (UID: \"4f7b2e31-7b8f-4ea5-8c84-49b0ba67d5c8\") " pod="ai-tenant-e2e-autores-bd5806/autores-model-bd5806-kserve-654584c895-n4dxx"
Sep 10 15:43:07.513026 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:43:07.512932    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-autores-bd5806/autores-model-bd5806-kserve-654584c895-n4dxx"
Sep 10 15:43:07.524419 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:43:07.524390    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:43:07.785792 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:43:07.785709    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-autores-bd5806/autores-model-bd5806-kserve-654584c895-n4dxx"]
Sep 10 15:43:07.788793 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:43:07.788767    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4f7b2e31_7b8f_4ea5_8c84_49b0ba67d5c8.slice/crio-4e05a6dd863917b56866fa060a5370b45da32c004165cdfd752d06f854891295 WatchSource:0}: Error finding container 4e05a6dd863917b56866fa060a5370b45da32c004165cdfd752d06f854891295: Status 404 returned error can't find the container with id 4e05a6dd863917b56866fa060a5370b45da32c004165cdfd752d06f854891295
Sep 10 15:43:08.112581 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:43:08.112494    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-autores-bd5806/autores-model-bd5806-kserve-654584c895-n4dxx" event={"ID":"4f7b2e31-7b8f-4ea5-8c84-49b0ba67d5c8","Type":"ContainerStarted","Data":"19afb0afa5b850c6dfb188bd66a46735378373271213e5f60cbfd290b1bba36c"}
Sep 10 15:43:08.112581 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:43:08.112534    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-autores-bd5806/autores-model-bd5806-kserve-654584c895-n4dxx" event={"ID":"4f7b2e31-7b8f-4ea5-8c84-49b0ba67d5c8","Type":"ContainerStarted","Data":"4e05a6dd863917b56866fa060a5370b45da32c004165cdfd752d06f854891295"}
Sep 10 15:43:08.142609 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:43:08.142554    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-autores-bd5806/autores-model-bd5806-kserve-654584c895-n4dxx" podStartSLOduration=1.142539116 podStartE2EDuration="1.142539116s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 15:43:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 15:43:08.140479522 +0000 UTC m=+973.708711263" watchObservedRunningTime="2026-09-10 15:43:08.142539116 +0000 UTC m=+973.710770843"
Sep 10 15:43:13.452707 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:43:13.452665    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:43:14.308123 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:43:14.308088    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:43:15.187051 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:43:15.187014    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:43:16.067941 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:43:16.067901    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:43:16.461296 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:43:16.460969    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:43:17.190933 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:43:17.190847    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:43:17.513823 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:43:17.513720    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-autores-bd5806/autores-model-bd5806-kserve-654584c895-n4dxx"
Sep 10 15:43:17.513823 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:43:17.513782    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-autores-bd5806/autores-model-bd5806-kserve-654584c895-n4dxx"
Sep 10 15:43:17.526150 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:43:17.526118    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-autores-bd5806/autores-model-bd5806-kserve-654584c895-n4dxx"
Sep 10 15:43:18.173544 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:43:18.173505    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-autores-bd5806/autores-model-bd5806-kserve-654584c895-n4dxx"
Sep 10 15:43:20.187860 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:43:20.187823    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:43:21.274022 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:43:21.273966    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:43:22.534407 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:43:22.534379    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:43:23.511974 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:43:23.511935    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:43:24.723928 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:43:24.723890    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:43:30.248479 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:43:30.248441    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-bd8496a8-6b56dd7bfb-ndl8d"]
Sep 10 15:43:30.248909 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:43:30.248649    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-bd8496a8-6b56dd7bfb-ndl8d" podUID="c5c2b5fb-bdcb-4a72-9a59-015a9e94bd7e" containerName="maas-api" containerID="cri-o://7f64055a9c5f69464ed3f82715d5fce7fce9a595b63a92e733939de1e3023ac3" gracePeriod=30
Sep 10 15:43:30.290114 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:43:30.290082    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:43:30.378424 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:43:30.378391    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-bd8496a8-754fd849b6-5nmmg"]
Sep 10 15:43:30.378643 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:43:30.378616    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ait-bd8496a8-754fd849b6-5nmmg" podUID="c7203fc2-5472-4306-ba88-1946aa28c463" containerName="payload-pre-processing" containerID="cri-o://98957ca9041b71aed048695a1bf70299e669216e2ce887b70857ffb6be2b371d" gracePeriod=30
Sep 10 15:43:30.432561 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:43:30.432521    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:43:30.514158 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:43:30.514087    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-bd8496a8-6b56dd7bfb-ndl8d"
Sep 10 15:43:30.589666 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:43:30.589626    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:43:30.636412 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:43:30.636386    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-bd8496a8-754fd849b6-5nmmg"
Sep 10 15:43:30.685687 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:43:30.685657    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:43:30.689144 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:43:30.689125    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c5c2b5fb-bdcb-4a72-9a59-015a9e94bd7e-metrics-certs\") pod \"c5c2b5fb-bdcb-4a72-9a59-015a9e94bd7e\" (UID: \"c5c2b5fb-bdcb-4a72-9a59-015a9e94bd7e\") "
Sep 10 15:43:30.689352 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:43:30.689334    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9gzg5\" (UniqueName: \"kubernetes.io/projected/c5c2b5fb-bdcb-4a72-9a59-015a9e94bd7e-kube-api-access-9gzg5\") pod \"c5c2b5fb-bdcb-4a72-9a59-015a9e94bd7e\" (UID: \"c5c2b5fb-bdcb-4a72-9a59-015a9e94bd7e\") "
Sep 10 15:43:30.689401 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:43:30.689376    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c5c2b5fb-bdcb-4a72-9a59-015a9e94bd7e-openshift-service-ca\") pod \"c5c2b5fb-bdcb-4a72-9a59-015a9e94bd7e\" (UID: \"c5c2b5fb-bdcb-4a72-9a59-015a9e94bd7e\") "
Sep 10 15:43:30.689452 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:43:30.689428    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c5c2b5fb-bdcb-4a72-9a59-015a9e94bd7e-maas-api-tls\") pod \"c5c2b5fb-bdcb-4a72-9a59-015a9e94bd7e\" (UID: \"c5c2b5fb-bdcb-4a72-9a59-015a9e94bd7e\") "
Sep 10 15:43:30.689883 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:43:30.689806    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c5c2b5fb-bdcb-4a72-9a59-015a9e94bd7e-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "c5c2b5fb-bdcb-4a72-9a59-015a9e94bd7e" (UID: "c5c2b5fb-bdcb-4a72-9a59-015a9e94bd7e"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 15:43:30.691913 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:43:30.691885    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c5c2b5fb-bdcb-4a72-9a59-015a9e94bd7e-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "c5c2b5fb-bdcb-4a72-9a59-015a9e94bd7e" (UID: "c5c2b5fb-bdcb-4a72-9a59-015a9e94bd7e"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 15:43:30.692009 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:43:30.691909    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c5c2b5fb-bdcb-4a72-9a59-015a9e94bd7e-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "c5c2b5fb-bdcb-4a72-9a59-015a9e94bd7e" (UID: "c5c2b5fb-bdcb-4a72-9a59-015a9e94bd7e"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 15:43:30.692009 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:43:30.691995    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c5c2b5fb-bdcb-4a72-9a59-015a9e94bd7e-kube-api-access-9gzg5" (OuterVolumeSpecName: "kube-api-access-9gzg5") pod "c5c2b5fb-bdcb-4a72-9a59-015a9e94bd7e" (UID: "c5c2b5fb-bdcb-4a72-9a59-015a9e94bd7e"). InnerVolumeSpecName "kube-api-access-9gzg5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 15:43:30.790353 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:43:30.790248    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c7203fc2-5472-4306-ba88-1946aa28c463-plugins-config-volume\") pod \"c7203fc2-5472-4306-ba88-1946aa28c463\" (UID: \"c7203fc2-5472-4306-ba88-1946aa28c463\") "
Sep 10 15:43:30.790353 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:43:30.790363    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q9whw\" (UniqueName: \"kubernetes.io/projected/c7203fc2-5472-4306-ba88-1946aa28c463-kube-api-access-q9whw\") pod \"c7203fc2-5472-4306-ba88-1946aa28c463\" (UID: \"c7203fc2-5472-4306-ba88-1946aa28c463\") "
Sep 10 15:43:30.790608 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:43:30.790552    2567 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c5c2b5fb-bdcb-4a72-9a59-015a9e94bd7e-metrics-certs\") on node \"ip-10-0-128-241.ec2.internal\" DevicePath \"\""
Sep 10 15:43:30.790608 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:43:30.790565    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9gzg5\" (UniqueName: \"kubernetes.io/projected/c5c2b5fb-bdcb-4a72-9a59-015a9e94bd7e-kube-api-access-9gzg5\") on node \"ip-10-0-128-241.ec2.internal\" DevicePath \"\""
Sep 10 15:43:30.790608 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:43:30.790576    2567 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c5c2b5fb-bdcb-4a72-9a59-015a9e94bd7e-openshift-service-ca\") on node \"ip-10-0-128-241.ec2.internal\" DevicePath \"\""
Sep 10 15:43:30.790608 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:43:30.790584    2567 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c5c2b5fb-bdcb-4a72-9a59-015a9e94bd7e-maas-api-tls\") on node \"ip-10-0-128-241.ec2.internal\" DevicePath \"\""
Sep 10 15:43:30.790793 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:43:30.790612    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c7203fc2-5472-4306-ba88-1946aa28c463-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "c7203fc2-5472-4306-ba88-1946aa28c463" (UID: "c7203fc2-5472-4306-ba88-1946aa28c463"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 15:43:30.792459 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:43:30.792431    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c7203fc2-5472-4306-ba88-1946aa28c463-kube-api-access-q9whw" (OuterVolumeSpecName: "kube-api-access-q9whw") pod "c7203fc2-5472-4306-ba88-1946aa28c463" (UID: "c7203fc2-5472-4306-ba88-1946aa28c463"). InnerVolumeSpecName "kube-api-access-q9whw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 15:43:30.793870 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:43:30.793849    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:43:30.891933 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:43:30.891885    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q9whw\" (UniqueName: \"kubernetes.io/projected/c7203fc2-5472-4306-ba88-1946aa28c463-kube-api-access-q9whw\") on node \"ip-10-0-128-241.ec2.internal\" DevicePath \"\""
Sep 10 15:43:30.891933 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:43:30.891925    2567 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c7203fc2-5472-4306-ba88-1946aa28c463-plugins-config-volume\") on node \"ip-10-0-128-241.ec2.internal\" DevicePath \"\""
Sep 10 15:43:31.185430 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:43:31.185400    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:43:31.229290 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:43:31.229257    2567 generic.go:356] "Generic (PLEG): container finished" podID="c5c2b5fb-bdcb-4a72-9a59-015a9e94bd7e" containerID="7f64055a9c5f69464ed3f82715d5fce7fce9a595b63a92e733939de1e3023ac3" exitCode=0
Sep 10 15:43:31.229476 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:43:31.229296    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-bd8496a8-6b56dd7bfb-ndl8d" event={"ID":"c5c2b5fb-bdcb-4a72-9a59-015a9e94bd7e","Type":"ContainerDied","Data":"7f64055a9c5f69464ed3f82715d5fce7fce9a595b63a92e733939de1e3023ac3"}
Sep 10 15:43:31.229476 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:43:31.229309    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-bd8496a8-6b56dd7bfb-ndl8d"
Sep 10 15:43:31.229476 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:43:31.229363    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-bd8496a8-6b56dd7bfb-ndl8d" event={"ID":"c5c2b5fb-bdcb-4a72-9a59-015a9e94bd7e","Type":"ContainerDied","Data":"f099ac1c7426e4151fc2175ec1321a52ae71362a0880742b9259dcc5da9080ee"}
Sep 10 15:43:31.229476 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:43:31.229392    2567 scope.go:117] "RemoveContainer" containerID="7f64055a9c5f69464ed3f82715d5fce7fce9a595b63a92e733939de1e3023ac3"
Sep 10 15:43:31.230805 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:43:31.230784    2567 generic.go:356] "Generic (PLEG): container finished" podID="c7203fc2-5472-4306-ba88-1946aa28c463" containerID="98957ca9041b71aed048695a1bf70299e669216e2ce887b70857ffb6be2b371d" exitCode=0
Sep 10 15:43:31.230928 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:43:31.230829    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-bd8496a8-754fd849b6-5nmmg"
Sep 10 15:43:31.230928 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:43:31.230876    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-bd8496a8-754fd849b6-5nmmg" event={"ID":"c7203fc2-5472-4306-ba88-1946aa28c463","Type":"ContainerDied","Data":"98957ca9041b71aed048695a1bf70299e669216e2ce887b70857ffb6be2b371d"}
Sep 10 15:43:31.231024 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:43:31.230934    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-bd8496a8-754fd849b6-5nmmg" event={"ID":"c7203fc2-5472-4306-ba88-1946aa28c463","Type":"ContainerDied","Data":"1117000c13563c2f18a7b584a0b1cbf6bc36b3b017c7201ac0a6aec9343de7c1"}
Sep 10 15:43:31.238283 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:43:31.238265    2567 scope.go:117] "RemoveContainer" containerID="7f64055a9c5f69464ed3f82715d5fce7fce9a595b63a92e733939de1e3023ac3"
Sep 10 15:43:31.238569 ip-10-0-128-241 kubenswrapper[2567]: E0910 15:43:31.238549    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7f64055a9c5f69464ed3f82715d5fce7fce9a595b63a92e733939de1e3023ac3\": container with ID starting with 7f64055a9c5f69464ed3f82715d5fce7fce9a595b63a92e733939de1e3023ac3 not found: ID does not exist" containerID="7f64055a9c5f69464ed3f82715d5fce7fce9a595b63a92e733939de1e3023ac3"
Sep 10 15:43:31.238614 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:43:31.238578    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7f64055a9c5f69464ed3f82715d5fce7fce9a595b63a92e733939de1e3023ac3"} err="failed to get container status \"7f64055a9c5f69464ed3f82715d5fce7fce9a595b63a92e733939de1e3023ac3\": rpc error: code = NotFound desc = could not find container \"7f64055a9c5f69464ed3f82715d5fce7fce9a595b63a92e733939de1e3023ac3\": container with ID starting with 7f64055a9c5f69464ed3f82715d5fce7fce9a595b63a92e733939de1e3023ac3 not found: ID does not exist"
Sep 10 15:43:31.238614 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:43:31.238595    2567 scope.go:117] "RemoveContainer" containerID="98957ca9041b71aed048695a1bf70299e669216e2ce887b70857ffb6be2b371d"
Sep 10 15:43:31.246629 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:43:31.246607    2567 scope.go:117] "RemoveContainer" containerID="98957ca9041b71aed048695a1bf70299e669216e2ce887b70857ffb6be2b371d"
Sep 10 15:43:31.246920 ip-10-0-128-241 kubenswrapper[2567]: E0910 15:43:31.246900    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"98957ca9041b71aed048695a1bf70299e669216e2ce887b70857ffb6be2b371d\": container with ID starting with 98957ca9041b71aed048695a1bf70299e669216e2ce887b70857ffb6be2b371d not found: ID does not exist" containerID="98957ca9041b71aed048695a1bf70299e669216e2ce887b70857ffb6be2b371d"
Sep 10 15:43:31.246981 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:43:31.246932    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"98957ca9041b71aed048695a1bf70299e669216e2ce887b70857ffb6be2b371d"} err="failed to get container status \"98957ca9041b71aed048695a1bf70299e669216e2ce887b70857ffb6be2b371d\": rpc error: code = NotFound desc = could not find container \"98957ca9041b71aed048695a1bf70299e669216e2ce887b70857ffb6be2b371d\": container with ID starting with 98957ca9041b71aed048695a1bf70299e669216e2ce887b70857ffb6be2b371d not found: ID does not exist"
Sep 10 15:43:31.264093 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:43:31.264068    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-bd8496a8-754fd849b6-5nmmg"]
Sep 10 15:43:31.276664 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:43:31.276636    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-bd8496a8-754fd849b6-5nmmg"]
Sep 10 15:43:31.298073 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:43:31.298039    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-bd8496a8-6b56dd7bfb-ndl8d"]
Sep 10 15:43:31.301259 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:43:31.301214    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-bd8496a8-6b56dd7bfb-ndl8d"]
Sep 10 15:43:31.978160 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:43:31.978129    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:43:32.347139 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:43:32.347054    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:43:32.448622 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:43:32.448591    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:43:32.851099 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:43:32.851061    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:43:32.903902 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:43:32.903862    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c5c2b5fb-bdcb-4a72-9a59-015a9e94bd7e" path="/var/lib/kubelet/pods/c5c2b5fb-bdcb-4a72-9a59-015a9e94bd7e/volumes"
Sep 10 15:43:32.904304 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:43:32.904288    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c7203fc2-5472-4306-ba88-1946aa28c463" path="/var/lib/kubelet/pods/c7203fc2-5472-4306-ba88-1946aa28c463/volumes"
Sep 10 15:43:33.072192 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:43:33.072159    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:43:33.745831 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:43:33.745795    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:43:34.851702 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:43:34.851665    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:43:35.404018 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:43:35.403987    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:43:35.525465 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:43:35.525411    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:43:36.142924 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:43:36.142893    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:43:37.504919 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:43:37.504881    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-conflict-b-fa9f2a54-kserve-86d68b877-rq7rv"]
Sep 10 15:43:37.505314 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:43:37.505268    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c5c2b5fb-bdcb-4a72-9a59-015a9e94bd7e" containerName="maas-api"
Sep 10 15:43:37.505314 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:43:37.505282    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="c5c2b5fb-bdcb-4a72-9a59-015a9e94bd7e" containerName="maas-api"
Sep 10 15:43:37.505314 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:43:37.505300    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c7203fc2-5472-4306-ba88-1946aa28c463" containerName="payload-pre-processing"
Sep 10 15:43:37.505314 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:43:37.505306    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="c7203fc2-5472-4306-ba88-1946aa28c463" containerName="payload-pre-processing"
Sep 10 15:43:37.505454 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:43:37.505369    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="c7203fc2-5472-4306-ba88-1946aa28c463" containerName="payload-pre-processing"
Sep 10 15:43:37.505454 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:43:37.505378    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="c5c2b5fb-bdcb-4a72-9a59-015a9e94bd7e" containerName="maas-api"
Sep 10 15:43:37.509987 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:43:37.509968    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-conflict-b-fa9f2a54-kserve-86d68b877-rq7rv"
Sep 10 15:43:37.514175 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:43:37.514155    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-conflict-b-fa9f2a54-kserve-self-signed-certs\""
Sep 10 15:43:37.523915 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:43:37.523889    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-conflict-b-fa9f2a54-kserve-86d68b877-rq7rv"]
Sep 10 15:43:37.526594 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:43:37.526571    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:43:37.551343 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:43:37.551312    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a1381c6f-a19e-49cd-86b9-a159effffce2-home\") pod \"e2e-conflict-b-fa9f2a54-kserve-86d68b877-rq7rv\" (UID: \"a1381c6f-a19e-49cd-86b9-a159effffce2\") " pod="llm/e2e-conflict-b-fa9f2a54-kserve-86d68b877-rq7rv"
Sep 10 15:43:37.551504 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:43:37.551405    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a1381c6f-a19e-49cd-86b9-a159effffce2-tmp-dir\") pod \"e2e-conflict-b-fa9f2a54-kserve-86d68b877-rq7rv\" (UID: \"a1381c6f-a19e-49cd-86b9-a159effffce2\") " pod="llm/e2e-conflict-b-fa9f2a54-kserve-86d68b877-rq7rv"
Sep 10 15:43:37.551504 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:43:37.551432    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wsvtg\" (UniqueName: \"kubernetes.io/projected/a1381c6f-a19e-49cd-86b9-a159effffce2-kube-api-access-wsvtg\") pod \"e2e-conflict-b-fa9f2a54-kserve-86d68b877-rq7rv\" (UID: \"a1381c6f-a19e-49cd-86b9-a159effffce2\") " pod="llm/e2e-conflict-b-fa9f2a54-kserve-86d68b877-rq7rv"
Sep 10 15:43:37.551623 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:43:37.551517    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a1381c6f-a19e-49cd-86b9-a159effffce2-dshm\") pod \"e2e-conflict-b-fa9f2a54-kserve-86d68b877-rq7rv\" (UID: \"a1381c6f-a19e-49cd-86b9-a159effffce2\") " pod="llm/e2e-conflict-b-fa9f2a54-kserve-86d68b877-rq7rv"
Sep 10 15:43:37.551623 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:43:37.551556    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a1381c6f-a19e-49cd-86b9-a159effffce2-model-cache\") pod \"e2e-conflict-b-fa9f2a54-kserve-86d68b877-rq7rv\" (UID: \"a1381c6f-a19e-49cd-86b9-a159effffce2\") " pod="llm/e2e-conflict-b-fa9f2a54-kserve-86d68b877-rq7rv"
Sep 10 15:43:37.551623 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:43:37.551589    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a1381c6f-a19e-49cd-86b9-a159effffce2-tls-certs\") pod \"e2e-conflict-b-fa9f2a54-kserve-86d68b877-rq7rv\" (UID: \"a1381c6f-a19e-49cd-86b9-a159effffce2\") " pod="llm/e2e-conflict-b-fa9f2a54-kserve-86d68b877-rq7rv"
Sep 10 15:43:37.652024 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:43:37.651985    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a1381c6f-a19e-49cd-86b9-a159effffce2-tls-certs\") pod \"e2e-conflict-b-fa9f2a54-kserve-86d68b877-rq7rv\" (UID: \"a1381c6f-a19e-49cd-86b9-a159effffce2\") " pod="llm/e2e-conflict-b-fa9f2a54-kserve-86d68b877-rq7rv"
Sep 10 15:43:37.652024 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:43:37.652027    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a1381c6f-a19e-49cd-86b9-a159effffce2-home\") pod \"e2e-conflict-b-fa9f2a54-kserve-86d68b877-rq7rv\" (UID: \"a1381c6f-a19e-49cd-86b9-a159effffce2\") " pod="llm/e2e-conflict-b-fa9f2a54-kserve-86d68b877-rq7rv"
Sep 10 15:43:37.652291 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:43:37.652076    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a1381c6f-a19e-49cd-86b9-a159effffce2-tmp-dir\") pod \"e2e-conflict-b-fa9f2a54-kserve-86d68b877-rq7rv\" (UID: \"a1381c6f-a19e-49cd-86b9-a159effffce2\") " pod="llm/e2e-conflict-b-fa9f2a54-kserve-86d68b877-rq7rv"
Sep 10 15:43:37.652291 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:43:37.652099    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wsvtg\" (UniqueName: \"kubernetes.io/projected/a1381c6f-a19e-49cd-86b9-a159effffce2-kube-api-access-wsvtg\") pod \"e2e-conflict-b-fa9f2a54-kserve-86d68b877-rq7rv\" (UID: \"a1381c6f-a19e-49cd-86b9-a159effffce2\") " pod="llm/e2e-conflict-b-fa9f2a54-kserve-86d68b877-rq7rv"
Sep 10 15:43:37.652291 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:43:37.652221    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a1381c6f-a19e-49cd-86b9-a159effffce2-dshm\") pod \"e2e-conflict-b-fa9f2a54-kserve-86d68b877-rq7rv\" (UID: \"a1381c6f-a19e-49cd-86b9-a159effffce2\") " pod="llm/e2e-conflict-b-fa9f2a54-kserve-86d68b877-rq7rv"
Sep 10 15:43:37.652445 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:43:37.652310    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a1381c6f-a19e-49cd-86b9-a159effffce2-model-cache\") pod \"e2e-conflict-b-fa9f2a54-kserve-86d68b877-rq7rv\" (UID: \"a1381c6f-a19e-49cd-86b9-a159effffce2\") " pod="llm/e2e-conflict-b-fa9f2a54-kserve-86d68b877-rq7rv"
Sep 10 15:43:37.652595 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:43:37.652572    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a1381c6f-a19e-49cd-86b9-a159effffce2-home\") pod \"e2e-conflict-b-fa9f2a54-kserve-86d68b877-rq7rv\" (UID: \"a1381c6f-a19e-49cd-86b9-a159effffce2\") " pod="llm/e2e-conflict-b-fa9f2a54-kserve-86d68b877-rq7rv"
Sep 10 15:43:37.652675 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:43:37.652600    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a1381c6f-a19e-49cd-86b9-a159effffce2-tmp-dir\") pod \"e2e-conflict-b-fa9f2a54-kserve-86d68b877-rq7rv\" (UID: \"a1381c6f-a19e-49cd-86b9-a159effffce2\") " pod="llm/e2e-conflict-b-fa9f2a54-kserve-86d68b877-rq7rv"
Sep 10 15:43:37.652675 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:43:37.652639    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a1381c6f-a19e-49cd-86b9-a159effffce2-model-cache\") pod \"e2e-conflict-b-fa9f2a54-kserve-86d68b877-rq7rv\" (UID: \"a1381c6f-a19e-49cd-86b9-a159effffce2\") " pod="llm/e2e-conflict-b-fa9f2a54-kserve-86d68b877-rq7rv"
Sep 10 15:43:37.654427 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:43:37.654409    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a1381c6f-a19e-49cd-86b9-a159effffce2-dshm\") pod \"e2e-conflict-b-fa9f2a54-kserve-86d68b877-rq7rv\" (UID: \"a1381c6f-a19e-49cd-86b9-a159effffce2\") " pod="llm/e2e-conflict-b-fa9f2a54-kserve-86d68b877-rq7rv"
Sep 10 15:43:37.654560 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:43:37.654543    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a1381c6f-a19e-49cd-86b9-a159effffce2-tls-certs\") pod \"e2e-conflict-b-fa9f2a54-kserve-86d68b877-rq7rv\" (UID: \"a1381c6f-a19e-49cd-86b9-a159effffce2\") " pod="llm/e2e-conflict-b-fa9f2a54-kserve-86d68b877-rq7rv"
Sep 10 15:43:37.667419 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:43:37.667393    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wsvtg\" (UniqueName: \"kubernetes.io/projected/a1381c6f-a19e-49cd-86b9-a159effffce2-kube-api-access-wsvtg\") pod \"e2e-conflict-b-fa9f2a54-kserve-86d68b877-rq7rv\" (UID: \"a1381c6f-a19e-49cd-86b9-a159effffce2\") " pod="llm/e2e-conflict-b-fa9f2a54-kserve-86d68b877-rq7rv"
Sep 10 15:43:37.819103 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:43:37.819014    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-conflict-b-fa9f2a54-kserve-86d68b877-rq7rv"
Sep 10 15:43:37.821865 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:43:37.821836    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:43:38.114066 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:43:38.114039    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-conflict-b-fa9f2a54-kserve-86d68b877-rq7rv"]
Sep 10 15:43:38.116487 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:43:38.116460    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda1381c6f_a19e_49cd_86b9_a159effffce2.slice/crio-1d5d90573cb2d21477e776a085dd6037b8ab20d69e8aa4c6c42f49f0acfee728 WatchSource:0}: Error finding container 1d5d90573cb2d21477e776a085dd6037b8ab20d69e8aa4c6c42f49f0acfee728: Status 404 returned error can't find the container with id 1d5d90573cb2d21477e776a085dd6037b8ab20d69e8aa4c6c42f49f0acfee728
Sep 10 15:43:38.268129 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:43:38.268093    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-conflict-b-fa9f2a54-kserve-86d68b877-rq7rv" event={"ID":"a1381c6f-a19e-49cd-86b9-a159effffce2","Type":"ContainerStarted","Data":"379450663d742fe0fb31f88452cc060ad56983310b12fd682afcec36e45b4ed8"}
Sep 10 15:43:38.268314 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:43:38.268137    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-conflict-b-fa9f2a54-kserve-86d68b877-rq7rv" event={"ID":"a1381c6f-a19e-49cd-86b9-a159effffce2","Type":"ContainerStarted","Data":"1d5d90573cb2d21477e776a085dd6037b8ab20d69e8aa4c6c42f49f0acfee728"}
Sep 10 15:43:38.291801 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:43:38.291734    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-conflict-b-fa9f2a54-kserve-86d68b877-rq7rv" podStartSLOduration=1.291710838 podStartE2EDuration="1.291710838s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 15:43:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 15:43:38.291015302 +0000 UTC m=+1003.859247034" watchObservedRunningTime="2026-09-10 15:43:38.291710838 +0000 UTC m=+1003.859942568"
Sep 10 15:43:38.699713 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:43:38.699681    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:43:39.926807 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:43:39.926774    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-680721f9-54cdfb9966-h85l2"]
Sep 10 15:43:39.931616 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:43:39.931591    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-680721f9-54cdfb9966-h85l2"
Sep 10 15:43:39.935285 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:43:39.935256    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-derive-680721f9-serving-cert\""
Sep 10 15:43:39.944914 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:43:39.944888    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-680721f9-54cdfb9966-h85l2"]
Sep 10 15:43:39.970984 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:43:39.970948    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:43:39.973683 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:43:39.973659    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/926cc7ae-07fd-4f38-b6c9-feea815e13b3-metrics-certs\") pod \"maas-api-e2e-derive-680721f9-54cdfb9966-h85l2\" (UID: \"926cc7ae-07fd-4f38-b6c9-feea815e13b3\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-680721f9-54cdfb9966-h85l2"
Sep 10 15:43:39.973792 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:43:39.973709    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/926cc7ae-07fd-4f38-b6c9-feea815e13b3-maas-api-tls\") pod \"maas-api-e2e-derive-680721f9-54cdfb9966-h85l2\" (UID: \"926cc7ae-07fd-4f38-b6c9-feea815e13b3\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-680721f9-54cdfb9966-h85l2"
Sep 10 15:43:39.973792 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:43:39.973758    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rfrjb\" (UniqueName: \"kubernetes.io/projected/926cc7ae-07fd-4f38-b6c9-feea815e13b3-kube-api-access-rfrjb\") pod \"maas-api-e2e-derive-680721f9-54cdfb9966-h85l2\" (UID: \"926cc7ae-07fd-4f38-b6c9-feea815e13b3\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-680721f9-54cdfb9966-h85l2"
Sep 10 15:43:39.973875 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:43:39.973864    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/926cc7ae-07fd-4f38-b6c9-feea815e13b3-openshift-service-ca\") pod \"maas-api-e2e-derive-680721f9-54cdfb9966-h85l2\" (UID: \"926cc7ae-07fd-4f38-b6c9-feea815e13b3\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-680721f9-54cdfb9966-h85l2"
Sep 10 15:43:40.068682 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:43:40.068651    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:43:40.074844 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:43:40.074818    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/926cc7ae-07fd-4f38-b6c9-feea815e13b3-openshift-service-ca\") pod \"maas-api-e2e-derive-680721f9-54cdfb9966-h85l2\" (UID: \"926cc7ae-07fd-4f38-b6c9-feea815e13b3\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-680721f9-54cdfb9966-h85l2"
Sep 10 15:43:40.074917 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:43:40.074862    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/926cc7ae-07fd-4f38-b6c9-feea815e13b3-metrics-certs\") pod \"maas-api-e2e-derive-680721f9-54cdfb9966-h85l2\" (UID: \"926cc7ae-07fd-4f38-b6c9-feea815e13b3\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-680721f9-54cdfb9966-h85l2"
Sep 10 15:43:40.074917 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:43:40.074889    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/926cc7ae-07fd-4f38-b6c9-feea815e13b3-maas-api-tls\") pod \"maas-api-e2e-derive-680721f9-54cdfb9966-h85l2\" (UID: \"926cc7ae-07fd-4f38-b6c9-feea815e13b3\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-680721f9-54cdfb9966-h85l2"
Sep 10 15:43:40.075010 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:43:40.074920    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rfrjb\" (UniqueName: \"kubernetes.io/projected/926cc7ae-07fd-4f38-b6c9-feea815e13b3-kube-api-access-rfrjb\") pod \"maas-api-e2e-derive-680721f9-54cdfb9966-h85l2\" (UID: \"926cc7ae-07fd-4f38-b6c9-feea815e13b3\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-680721f9-54cdfb9966-h85l2"
Sep 10 15:43:40.075617 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:43:40.075588    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/926cc7ae-07fd-4f38-b6c9-feea815e13b3-openshift-service-ca\") pod \"maas-api-e2e-derive-680721f9-54cdfb9966-h85l2\" (UID: \"926cc7ae-07fd-4f38-b6c9-feea815e13b3\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-680721f9-54cdfb9966-h85l2"
Sep 10 15:43:40.077438 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:43:40.077419    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/926cc7ae-07fd-4f38-b6c9-feea815e13b3-metrics-certs\") pod \"maas-api-e2e-derive-680721f9-54cdfb9966-h85l2\" (UID: \"926cc7ae-07fd-4f38-b6c9-feea815e13b3\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-680721f9-54cdfb9966-h85l2"
Sep 10 15:43:40.077508 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:43:40.077443    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/926cc7ae-07fd-4f38-b6c9-feea815e13b3-maas-api-tls\") pod \"maas-api-e2e-derive-680721f9-54cdfb9966-h85l2\" (UID: \"926cc7ae-07fd-4f38-b6c9-feea815e13b3\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-680721f9-54cdfb9966-h85l2"
Sep 10 15:43:40.085143 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:43:40.085120    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rfrjb\" (UniqueName: \"kubernetes.io/projected/926cc7ae-07fd-4f38-b6c9-feea815e13b3-kube-api-access-rfrjb\") pod \"maas-api-e2e-derive-680721f9-54cdfb9966-h85l2\" (UID: \"926cc7ae-07fd-4f38-b6c9-feea815e13b3\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-680721f9-54cdfb9966-h85l2"
Sep 10 15:43:40.168510 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:43:40.168477    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:43:40.246390 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:43:40.246305    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-680721f9-54cdfb9966-h85l2"
Sep 10 15:43:40.526830 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:43:40.526805    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-680721f9-54cdfb9966-h85l2"]
Sep 10 15:43:40.529702 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:43:40.529669    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod926cc7ae_07fd_4f38_b6c9_feea815e13b3.slice/crio-edfe1790d1c70a526b916f46e44184a66ad432328ed0e69bf444714f351505c3 WatchSource:0}: Error finding container edfe1790d1c70a526b916f46e44184a66ad432328ed0e69bf444714f351505c3: Status 404 returned error can't find the container with id edfe1790d1c70a526b916f46e44184a66ad432328ed0e69bf444714f351505c3
Sep 10 15:43:40.566855 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:43:40.566809    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:43:40.787056 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:43:40.786975    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:43:41.283685 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:43:41.283648    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-680721f9-54cdfb9966-h85l2" event={"ID":"926cc7ae-07fd-4f38-b6c9-feea815e13b3","Type":"ContainerStarted","Data":"fc4e14cd3a1c4c72a298a6666ce28388c389da3537085fdd449fd5e87955dca0"}
Sep 10 15:43:41.284150 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:43:41.283697    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-680721f9-54cdfb9966-h85l2" event={"ID":"926cc7ae-07fd-4f38-b6c9-feea815e13b3","Type":"ContainerStarted","Data":"edfe1790d1c70a526b916f46e44184a66ad432328ed0e69bf444714f351505c3"}
Sep 10 15:43:41.284150 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:43:41.283718    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-680721f9-54cdfb9966-h85l2"
Sep 10 15:43:41.305759 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:43:41.305689    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-derive-680721f9-54cdfb9966-h85l2" podStartSLOduration=1.7314743049999999 podStartE2EDuration="2.305666784s" totalImagesPullingTime="574.192479ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 15:43:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 15:43:41.304570851 +0000 UTC m=+1006.872802582" watchObservedRunningTime="2026-09-10 15:43:41.305666784 +0000 UTC m=+1006.873898516"
Sep 10 15:43:43.759267 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:43:43.759216    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:43:44.615444 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:43:44.615414    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:43:45.451034 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:43:45.451001    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:43:46.363599 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:43:46.363560    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:43:47.199122 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:43:47.199088    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:43:47.294985 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:43:47.294954    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-680721f9-54cdfb9966-h85l2"
Sep 10 15:43:47.819176 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:43:47.819134    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-conflict-b-fa9f2a54-kserve-86d68b877-rq7rv"
Sep 10 15:43:47.819176 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:43:47.819183    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-conflict-b-fa9f2a54-kserve-86d68b877-rq7rv"
Sep 10 15:43:47.831895 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:43:47.831865    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-conflict-b-fa9f2a54-kserve-86d68b877-rq7rv"
Sep 10 15:43:48.105610 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:43:48.105485    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:43:48.174035 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:43:48.173992    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-autores-bd5806/autores-model-bd5806-kserve-654584c895-n4dxx"]
Sep 10 15:43:48.200095 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:43:48.200055    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:43:48.327838 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:43:48.327806    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-conflict-b-fa9f2a54-kserve-86d68b877-rq7rv"
Sep 10 15:43:54.181874 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:43:54.181822    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:43:54.220030 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:43:54.219987    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-bd5806-647db94487-xv97q"]
Sep 10 15:43:54.220375 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:43:54.220322    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-autores-bd5806-647db94487-xv97q" podUID="a8268b2f-1ba6-42c8-bb45-bcc2c389a765" containerName="payload-processing" containerID="cri-o://b07b4ed8fb8b3e24bda01e2e9cc6f3307b41c0c0267f2e1b56d7bce4f3eb19c9" gracePeriod=30
Sep 10 15:43:54.271508 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:43:54.271470    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:43:54.351577 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:43:54.351539    2567 generic.go:356] "Generic (PLEG): container finished" podID="a8268b2f-1ba6-42c8-bb45-bcc2c389a765" containerID="b07b4ed8fb8b3e24bda01e2e9cc6f3307b41c0c0267f2e1b56d7bce4f3eb19c9" exitCode=0
Sep 10 15:43:54.351754 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:43:54.351603    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-autores-bd5806-647db94487-xv97q" event={"ID":"a8268b2f-1ba6-42c8-bb45-bcc2c389a765","Type":"ContainerDied","Data":"b07b4ed8fb8b3e24bda01e2e9cc6f3307b41c0c0267f2e1b56d7bce4f3eb19c9"}
Sep 10 15:43:54.368678 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:43:54.368646    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:43:54.465617 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:43:54.465502    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:43:54.490114 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:43:54.490081    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-autores-bd5806-647db94487-xv97q"
Sep 10 15:43:54.612607 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:43:54.612572    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vv9pt\" (UniqueName: \"kubernetes.io/projected/a8268b2f-1ba6-42c8-bb45-bcc2c389a765-kube-api-access-vv9pt\") pod \"a8268b2f-1ba6-42c8-bb45-bcc2c389a765\" (UID: \"a8268b2f-1ba6-42c8-bb45-bcc2c389a765\") "
Sep 10 15:43:54.612777 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:43:54.612766    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a8268b2f-1ba6-42c8-bb45-bcc2c389a765-plugins-config-volume\") pod \"a8268b2f-1ba6-42c8-bb45-bcc2c389a765\" (UID: \"a8268b2f-1ba6-42c8-bb45-bcc2c389a765\") "
Sep 10 15:43:54.613166 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:43:54.613141    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a8268b2f-1ba6-42c8-bb45-bcc2c389a765-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "a8268b2f-1ba6-42c8-bb45-bcc2c389a765" (UID: "a8268b2f-1ba6-42c8-bb45-bcc2c389a765"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 15:43:54.614682 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:43:54.614654    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a8268b2f-1ba6-42c8-bb45-bcc2c389a765-kube-api-access-vv9pt" (OuterVolumeSpecName: "kube-api-access-vv9pt") pod "a8268b2f-1ba6-42c8-bb45-bcc2c389a765" (UID: "a8268b2f-1ba6-42c8-bb45-bcc2c389a765"). InnerVolumeSpecName "kube-api-access-vv9pt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 15:43:54.720276 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:43:54.714914    2567 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a8268b2f-1ba6-42c8-bb45-bcc2c389a765-plugins-config-volume\") on node \"ip-10-0-128-241.ec2.internal\" DevicePath \"\""
Sep 10 15:43:54.720276 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:43:54.714963    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vv9pt\" (UniqueName: \"kubernetes.io/projected/a8268b2f-1ba6-42c8-bb45-bcc2c389a765-kube-api-access-vv9pt\") on node \"ip-10-0-128-241.ec2.internal\" DevicePath \"\""
Sep 10 15:43:54.799533 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:43:54.799497    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:43:54.873565 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:43:54.873533    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:43:55.357639 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:43:55.357607    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-autores-bd5806-647db94487-xv97q" event={"ID":"a8268b2f-1ba6-42c8-bb45-bcc2c389a765","Type":"ContainerDied","Data":"828de96c78bc54ef5f937ab6124c6b3282b148b956667262e9999970f6c12fdc"}
Sep 10 15:43:55.358114 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:43:55.357678    2567 scope.go:117] "RemoveContainer" containerID="b07b4ed8fb8b3e24bda01e2e9cc6f3307b41c0c0267f2e1b56d7bce4f3eb19c9"
Sep 10 15:43:55.358114 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:43:55.357690    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-autores-bd5806-647db94487-xv97q"
Sep 10 15:43:55.375371 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:43:55.375341    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-bd5806-647db94487-xv97q"]
Sep 10 15:43:55.383607 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:43:55.383579    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-bd5806-647db94487-xv97q"]
Sep 10 15:43:55.699787 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:43:55.699754    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:43:56.567821 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:43:56.567784    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:43:56.903777 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:43:56.903689    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a8268b2f-1ba6-42c8-bb45-bcc2c389a765" path="/var/lib/kubelet/pods/a8268b2f-1ba6-42c8-bb45-bcc2c389a765/volumes"
Sep 10 15:43:57.569485 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:43:57.569447    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:43:59.053597 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:43:59.053540    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:43:59.868276 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:43:59.868220    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:44:00.073692 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:44:00.073658    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:44:01.342571 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:44:01.342536    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:44:02.173574 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:44:02.173530    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:44:03.179594 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:44:03.179553    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:44:03.193916 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:44:03.193868    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-autores-bd5806/autores-model-bd5806-kserve-654584c895-n4dxx" podUID="4f7b2e31-7b8f-4ea5-8c84-49b0ba67d5c8" containerName="main" containerID="cri-o://19afb0afa5b850c6dfb188bd66a46735378373271213e5f60cbfd290b1bba36c" gracePeriod=45
Sep 10 15:44:03.399730 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:44:03.399699    2567 generic.go:356] "Generic (PLEG): container finished" podID="4f7b2e31-7b8f-4ea5-8c84-49b0ba67d5c8" containerID="19afb0afa5b850c6dfb188bd66a46735378373271213e5f60cbfd290b1bba36c" exitCode=0
Sep 10 15:44:03.399907 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:44:03.399779    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-autores-bd5806/autores-model-bd5806-kserve-654584c895-n4dxx" event={"ID":"4f7b2e31-7b8f-4ea5-8c84-49b0ba67d5c8","Type":"ContainerDied","Data":"19afb0afa5b850c6dfb188bd66a46735378373271213e5f60cbfd290b1bba36c"}
Sep 10 15:44:03.449758 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:44:03.449731    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-autores-bd5806/autores-model-bd5806-kserve-654584c895-n4dxx"
Sep 10 15:44:03.596086 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:44:03.596044    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4f7b2e31-7b8f-4ea5-8c84-49b0ba67d5c8-dshm\") pod \"4f7b2e31-7b8f-4ea5-8c84-49b0ba67d5c8\" (UID: \"4f7b2e31-7b8f-4ea5-8c84-49b0ba67d5c8\") "
Sep 10 15:44:03.596086 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:44:03.596084    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4f7b2e31-7b8f-4ea5-8c84-49b0ba67d5c8-model-cache\") pod \"4f7b2e31-7b8f-4ea5-8c84-49b0ba67d5c8\" (UID: \"4f7b2e31-7b8f-4ea5-8c84-49b0ba67d5c8\") "
Sep 10 15:44:03.596374 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:44:03.596126    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4f7b2e31-7b8f-4ea5-8c84-49b0ba67d5c8-tmp-dir\") pod \"4f7b2e31-7b8f-4ea5-8c84-49b0ba67d5c8\" (UID: \"4f7b2e31-7b8f-4ea5-8c84-49b0ba67d5c8\") "
Sep 10 15:44:03.596374 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:44:03.596158    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4f7b2e31-7b8f-4ea5-8c84-49b0ba67d5c8-tls-certs\") pod \"4f7b2e31-7b8f-4ea5-8c84-49b0ba67d5c8\" (UID: \"4f7b2e31-7b8f-4ea5-8c84-49b0ba67d5c8\") "
Sep 10 15:44:03.596374 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:44:03.596177    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4f7b2e31-7b8f-4ea5-8c84-49b0ba67d5c8-home\") pod \"4f7b2e31-7b8f-4ea5-8c84-49b0ba67d5c8\" (UID: \"4f7b2e31-7b8f-4ea5-8c84-49b0ba67d5c8\") "
Sep 10 15:44:03.596374 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:44:03.596206    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5dmx5\" (UniqueName: \"kubernetes.io/projected/4f7b2e31-7b8f-4ea5-8c84-49b0ba67d5c8-kube-api-access-5dmx5\") pod \"4f7b2e31-7b8f-4ea5-8c84-49b0ba67d5c8\" (UID: \"4f7b2e31-7b8f-4ea5-8c84-49b0ba67d5c8\") "
Sep 10 15:44:03.596584 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:44:03.596482    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4f7b2e31-7b8f-4ea5-8c84-49b0ba67d5c8-model-cache" (OuterVolumeSpecName: "model-cache") pod "4f7b2e31-7b8f-4ea5-8c84-49b0ba67d5c8" (UID: "4f7b2e31-7b8f-4ea5-8c84-49b0ba67d5c8"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 15:44:03.596584 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:44:03.596493    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4f7b2e31-7b8f-4ea5-8c84-49b0ba67d5c8-home" (OuterVolumeSpecName: "home") pod "4f7b2e31-7b8f-4ea5-8c84-49b0ba67d5c8" (UID: "4f7b2e31-7b8f-4ea5-8c84-49b0ba67d5c8"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 15:44:03.596584 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:44:03.596499    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4f7b2e31-7b8f-4ea5-8c84-49b0ba67d5c8-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "4f7b2e31-7b8f-4ea5-8c84-49b0ba67d5c8" (UID: "4f7b2e31-7b8f-4ea5-8c84-49b0ba67d5c8"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 15:44:03.596731 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:44:03.596588    2567 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4f7b2e31-7b8f-4ea5-8c84-49b0ba67d5c8-home\") on node \"ip-10-0-128-241.ec2.internal\" DevicePath \"\""
Sep 10 15:44:03.596731 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:44:03.596607    2567 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4f7b2e31-7b8f-4ea5-8c84-49b0ba67d5c8-model-cache\") on node \"ip-10-0-128-241.ec2.internal\" DevicePath \"\""
Sep 10 15:44:03.596731 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:44:03.596621    2567 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4f7b2e31-7b8f-4ea5-8c84-49b0ba67d5c8-tmp-dir\") on node \"ip-10-0-128-241.ec2.internal\" DevicePath \"\""
Sep 10 15:44:03.598496 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:44:03.598471    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4f7b2e31-7b8f-4ea5-8c84-49b0ba67d5c8-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "4f7b2e31-7b8f-4ea5-8c84-49b0ba67d5c8" (UID: "4f7b2e31-7b8f-4ea5-8c84-49b0ba67d5c8"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 15:44:03.598496 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:44:03.598483    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4f7b2e31-7b8f-4ea5-8c84-49b0ba67d5c8-kube-api-access-5dmx5" (OuterVolumeSpecName: "kube-api-access-5dmx5") pod "4f7b2e31-7b8f-4ea5-8c84-49b0ba67d5c8" (UID: "4f7b2e31-7b8f-4ea5-8c84-49b0ba67d5c8"). InnerVolumeSpecName "kube-api-access-5dmx5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 15:44:03.598629 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:44:03.598547    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4f7b2e31-7b8f-4ea5-8c84-49b0ba67d5c8-dshm" (OuterVolumeSpecName: "dshm") pod "4f7b2e31-7b8f-4ea5-8c84-49b0ba67d5c8" (UID: "4f7b2e31-7b8f-4ea5-8c84-49b0ba67d5c8"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 15:44:03.697221 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:44:03.697117    2567 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4f7b2e31-7b8f-4ea5-8c84-49b0ba67d5c8-dshm\") on node \"ip-10-0-128-241.ec2.internal\" DevicePath \"\""
Sep 10 15:44:03.697221 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:44:03.697183    2567 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4f7b2e31-7b8f-4ea5-8c84-49b0ba67d5c8-tls-certs\") on node \"ip-10-0-128-241.ec2.internal\" DevicePath \"\""
Sep 10 15:44:03.697221 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:44:03.697201    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5dmx5\" (UniqueName: \"kubernetes.io/projected/4f7b2e31-7b8f-4ea5-8c84-49b0ba67d5c8-kube-api-access-5dmx5\") on node \"ip-10-0-128-241.ec2.internal\" DevicePath \"\""
Sep 10 15:44:04.267708 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:44:04.267669    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:44:04.407183 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:44:04.407143    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-autores-bd5806/autores-model-bd5806-kserve-654584c895-n4dxx" event={"ID":"4f7b2e31-7b8f-4ea5-8c84-49b0ba67d5c8","Type":"ContainerDied","Data":"4e05a6dd863917b56866fa060a5370b45da32c004165cdfd752d06f854891295"}
Sep 10 15:44:04.407183 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:44:04.407165    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-autores-bd5806/autores-model-bd5806-kserve-654584c895-n4dxx"
Sep 10 15:44:04.407443 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:44:04.407200    2567 scope.go:117] "RemoveContainer" containerID="19afb0afa5b850c6dfb188bd66a46735378373271213e5f60cbfd290b1bba36c"
Sep 10 15:44:04.432881 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:44:04.432851    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-autores-bd5806/autores-model-bd5806-kserve-654584c895-n4dxx"]
Sep 10 15:44:04.437747 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:44:04.437718    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-autores-bd5806/autores-model-bd5806-kserve-654584c895-n4dxx"]
Sep 10 15:44:04.905211 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:44:04.905177    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4f7b2e31-7b8f-4ea5-8c84-49b0ba67d5c8" path="/var/lib/kubelet/pods/4f7b2e31-7b8f-4ea5-8c84-49b0ba67d5c8/volumes"
Sep 10 15:44:05.120497 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:44:05.120467    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:44:05.449398 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:44:05.449357    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-conflict-b-fa9f2a54-kserve-86d68b877-rq7rv"]
Sep 10 15:44:05.477521 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:44:05.477487    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:44:06.811532 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:44:06.811485    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:44:07.335421 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:44:07.335387    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:44:07.468022 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:44:07.467990    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-680721f9-54cdfb9966-h85l2"]
Sep 10 15:44:07.468279 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:44:07.468250    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-derive-680721f9-54cdfb9966-h85l2" podUID="926cc7ae-07fd-4f38-b6c9-feea815e13b3" containerName="maas-api" containerID="cri-o://fc4e14cd3a1c4c72a298a6666ce28388c389da3537085fdd449fd5e87955dca0" gracePeriod=30
Sep 10 15:44:07.513423 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:44:07.513380    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:44:07.632327 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:44:07.632207    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:44:07.732355 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:44:07.732331    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-680721f9-54cdfb9966-h85l2"
Sep 10 15:44:07.734545 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:44:07.734513    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:44:07.836881 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:44:07.836847    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/926cc7ae-07fd-4f38-b6c9-feea815e13b3-maas-api-tls\") pod \"926cc7ae-07fd-4f38-b6c9-feea815e13b3\" (UID: \"926cc7ae-07fd-4f38-b6c9-feea815e13b3\") "
Sep 10 15:44:07.837403 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:44:07.836921    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/926cc7ae-07fd-4f38-b6c9-feea815e13b3-metrics-certs\") pod \"926cc7ae-07fd-4f38-b6c9-feea815e13b3\" (UID: \"926cc7ae-07fd-4f38-b6c9-feea815e13b3\") "
Sep 10 15:44:07.837403 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:44:07.837015    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rfrjb\" (UniqueName: \"kubernetes.io/projected/926cc7ae-07fd-4f38-b6c9-feea815e13b3-kube-api-access-rfrjb\") pod \"926cc7ae-07fd-4f38-b6c9-feea815e13b3\" (UID: \"926cc7ae-07fd-4f38-b6c9-feea815e13b3\") "
Sep 10 15:44:07.837403 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:44:07.837070    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/926cc7ae-07fd-4f38-b6c9-feea815e13b3-openshift-service-ca\") pod \"926cc7ae-07fd-4f38-b6c9-feea815e13b3\" (UID: \"926cc7ae-07fd-4f38-b6c9-feea815e13b3\") "
Sep 10 15:44:07.837538 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:44:07.837501    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/926cc7ae-07fd-4f38-b6c9-feea815e13b3-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "926cc7ae-07fd-4f38-b6c9-feea815e13b3" (UID: "926cc7ae-07fd-4f38-b6c9-feea815e13b3"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 15:44:07.839079 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:44:07.839054    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/926cc7ae-07fd-4f38-b6c9-feea815e13b3-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "926cc7ae-07fd-4f38-b6c9-feea815e13b3" (UID: "926cc7ae-07fd-4f38-b6c9-feea815e13b3"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 15:44:07.839509 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:44:07.839491    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/926cc7ae-07fd-4f38-b6c9-feea815e13b3-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "926cc7ae-07fd-4f38-b6c9-feea815e13b3" (UID: "926cc7ae-07fd-4f38-b6c9-feea815e13b3"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 15:44:07.839596 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:44:07.839513    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/926cc7ae-07fd-4f38-b6c9-feea815e13b3-kube-api-access-rfrjb" (OuterVolumeSpecName: "kube-api-access-rfrjb") pod "926cc7ae-07fd-4f38-b6c9-feea815e13b3" (UID: "926cc7ae-07fd-4f38-b6c9-feea815e13b3"). InnerVolumeSpecName "kube-api-access-rfrjb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 15:44:07.851140 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:44:07.851114    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:44:07.938425 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:44:07.938393    2567 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/926cc7ae-07fd-4f38-b6c9-feea815e13b3-maas-api-tls\") on node \"ip-10-0-128-241.ec2.internal\" DevicePath \"\""
Sep 10 15:44:07.938425 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:44:07.938425    2567 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/926cc7ae-07fd-4f38-b6c9-feea815e13b3-metrics-certs\") on node \"ip-10-0-128-241.ec2.internal\" DevicePath \"\""
Sep 10 15:44:07.938658 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:44:07.938439    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rfrjb\" (UniqueName: \"kubernetes.io/projected/926cc7ae-07fd-4f38-b6c9-feea815e13b3-kube-api-access-rfrjb\") on node \"ip-10-0-128-241.ec2.internal\" DevicePath \"\""
Sep 10 15:44:07.938658 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:44:07.938453    2567 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/926cc7ae-07fd-4f38-b6c9-feea815e13b3-openshift-service-ca\") on node \"ip-10-0-128-241.ec2.internal\" DevicePath \"\""
Sep 10 15:44:07.978799 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:44:07.978760    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:44:08.076881 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:44:08.076847    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:44:08.399951 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:44:08.399869    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:44:08.428361 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:44:08.428327    2567 generic.go:356] "Generic (PLEG): container finished" podID="926cc7ae-07fd-4f38-b6c9-feea815e13b3" containerID="fc4e14cd3a1c4c72a298a6666ce28388c389da3537085fdd449fd5e87955dca0" exitCode=0
Sep 10 15:44:08.428547 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:44:08.428372    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-680721f9-54cdfb9966-h85l2"
Sep 10 15:44:08.428547 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:44:08.428411    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-680721f9-54cdfb9966-h85l2" event={"ID":"926cc7ae-07fd-4f38-b6c9-feea815e13b3","Type":"ContainerDied","Data":"fc4e14cd3a1c4c72a298a6666ce28388c389da3537085fdd449fd5e87955dca0"}
Sep 10 15:44:08.428547 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:44:08.428461    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-680721f9-54cdfb9966-h85l2" event={"ID":"926cc7ae-07fd-4f38-b6c9-feea815e13b3","Type":"ContainerDied","Data":"edfe1790d1c70a526b916f46e44184a66ad432328ed0e69bf444714f351505c3"}
Sep 10 15:44:08.428547 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:44:08.428483    2567 scope.go:117] "RemoveContainer" containerID="fc4e14cd3a1c4c72a298a6666ce28388c389da3537085fdd449fd5e87955dca0"
Sep 10 15:44:08.437936 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:44:08.437912    2567 scope.go:117] "RemoveContainer" containerID="fc4e14cd3a1c4c72a298a6666ce28388c389da3537085fdd449fd5e87955dca0"
Sep 10 15:44:08.438245 ip-10-0-128-241 kubenswrapper[2567]: E0910 15:44:08.438207    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fc4e14cd3a1c4c72a298a6666ce28388c389da3537085fdd449fd5e87955dca0\": container with ID starting with fc4e14cd3a1c4c72a298a6666ce28388c389da3537085fdd449fd5e87955dca0 not found: ID does not exist" containerID="fc4e14cd3a1c4c72a298a6666ce28388c389da3537085fdd449fd5e87955dca0"
Sep 10 15:44:08.438326 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:44:08.438258    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fc4e14cd3a1c4c72a298a6666ce28388c389da3537085fdd449fd5e87955dca0"} err="failed to get container status \"fc4e14cd3a1c4c72a298a6666ce28388c389da3537085fdd449fd5e87955dca0\": rpc error: code = NotFound desc = could not find container \"fc4e14cd3a1c4c72a298a6666ce28388c389da3537085fdd449fd5e87955dca0\": container with ID starting with fc4e14cd3a1c4c72a298a6666ce28388c389da3537085fdd449fd5e87955dca0 not found: ID does not exist"
Sep 10 15:44:08.457936 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:44:08.457901    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-680721f9-54cdfb9966-h85l2"]
Sep 10 15:44:08.465921 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:44:08.465890    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-680721f9-54cdfb9966-h85l2"]
Sep 10 15:44:08.684788 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:44:08.684710    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:44:08.867434 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:44:08.867399    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:44:08.904148 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:44:08.904106    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="926cc7ae-07fd-4f38-b6c9-feea815e13b3" path="/var/lib/kubelet/pods/926cc7ae-07fd-4f38-b6c9-feea815e13b3/volumes"
Sep 10 15:44:08.976502 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:44:08.976409    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:44:09.498335 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:44:09.498298    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:44:10.424678 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:44:10.424647    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:44:10.675936 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:44:10.675816    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:44:11.660004 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:44:11.659969    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:44:11.975753 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:44:11.975705    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:44:12.075209 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:44:12.075173    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:44:12.581051 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:44:12.581013    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:44:13.439550 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:44:13.439505    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:44:14.213864 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:44:14.213829    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-fc495e-76f6c89b8b-rw8hz"]
Sep 10 15:44:14.214362 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:44:14.214347    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4f7b2e31-7b8f-4ea5-8c84-49b0ba67d5c8" containerName="main"
Sep 10 15:44:14.214431 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:44:14.214364    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="4f7b2e31-7b8f-4ea5-8c84-49b0ba67d5c8" containerName="main"
Sep 10 15:44:14.214431 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:44:14.214374    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a8268b2f-1ba6-42c8-bb45-bcc2c389a765" containerName="payload-processing"
Sep 10 15:44:14.214431 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:44:14.214380    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="a8268b2f-1ba6-42c8-bb45-bcc2c389a765" containerName="payload-processing"
Sep 10 15:44:14.214431 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:44:14.214390    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="926cc7ae-07fd-4f38-b6c9-feea815e13b3" containerName="maas-api"
Sep 10 15:44:14.214431 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:44:14.214396    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="926cc7ae-07fd-4f38-b6c9-feea815e13b3" containerName="maas-api"
Sep 10 15:44:14.214592 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:44:14.214456    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="4f7b2e31-7b8f-4ea5-8c84-49b0ba67d5c8" containerName="main"
Sep 10 15:44:14.214592 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:44:14.214463    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="926cc7ae-07fd-4f38-b6c9-feea815e13b3" containerName="maas-api"
Sep 10 15:44:14.214592 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:44:14.214470    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="a8268b2f-1ba6-42c8-bb45-bcc2c389a765" containerName="payload-processing"
Sep 10 15:44:14.224265 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:44:14.224122    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-fc495e-76f6c89b8b-rw8hz"
Sep 10 15:44:14.227314 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:44:14.227284    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-explicit-fc495e-serving-cert\""
Sep 10 15:44:14.234948 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:44:14.234447    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-fc495e-76f6c89b8b-rw8hz"]
Sep 10 15:44:14.257857 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:44:14.257809    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:44:14.295394 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:44:14.295350    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a7ca8d8b-db0b-47af-b685-75c3c446912d-metrics-certs\") pod \"maas-api-e2e-explicit-fc495e-76f6c89b8b-rw8hz\" (UID: \"a7ca8d8b-db0b-47af-b685-75c3c446912d\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-fc495e-76f6c89b8b-rw8hz"
Sep 10 15:44:14.295558 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:44:14.295420    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a7ca8d8b-db0b-47af-b685-75c3c446912d-openshift-service-ca\") pod \"maas-api-e2e-explicit-fc495e-76f6c89b8b-rw8hz\" (UID: \"a7ca8d8b-db0b-47af-b685-75c3c446912d\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-fc495e-76f6c89b8b-rw8hz"
Sep 10 15:44:14.295558 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:44:14.295452    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a7ca8d8b-db0b-47af-b685-75c3c446912d-maas-api-tls\") pod \"maas-api-e2e-explicit-fc495e-76f6c89b8b-rw8hz\" (UID: \"a7ca8d8b-db0b-47af-b685-75c3c446912d\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-fc495e-76f6c89b8b-rw8hz"
Sep 10 15:44:14.295558 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:44:14.295503    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lmbdw\" (UniqueName: \"kubernetes.io/projected/a7ca8d8b-db0b-47af-b685-75c3c446912d-kube-api-access-lmbdw\") pod \"maas-api-e2e-explicit-fc495e-76f6c89b8b-rw8hz\" (UID: \"a7ca8d8b-db0b-47af-b685-75c3c446912d\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-fc495e-76f6c89b8b-rw8hz"
Sep 10 15:44:14.358688 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:44:14.358657    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:44:14.395921 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:44:14.395888    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a7ca8d8b-db0b-47af-b685-75c3c446912d-maas-api-tls\") pod \"maas-api-e2e-explicit-fc495e-76f6c89b8b-rw8hz\" (UID: \"a7ca8d8b-db0b-47af-b685-75c3c446912d\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-fc495e-76f6c89b8b-rw8hz"
Sep 10 15:44:14.396099 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:44:14.395958    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lmbdw\" (UniqueName: \"kubernetes.io/projected/a7ca8d8b-db0b-47af-b685-75c3c446912d-kube-api-access-lmbdw\") pod \"maas-api-e2e-explicit-fc495e-76f6c89b8b-rw8hz\" (UID: \"a7ca8d8b-db0b-47af-b685-75c3c446912d\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-fc495e-76f6c89b8b-rw8hz"
Sep 10 15:44:14.396099 ip-10-0-128-241 kubenswrapper[2567]: E0910 15:44:14.396039    2567 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-explicit-fc495e-serving-cert: secret "maas-api-e2e-explicit-fc495e-serving-cert" not found
Sep 10 15:44:14.396206 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:44:14.396102    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a7ca8d8b-db0b-47af-b685-75c3c446912d-metrics-certs\") pod \"maas-api-e2e-explicit-fc495e-76f6c89b8b-rw8hz\" (UID: \"a7ca8d8b-db0b-47af-b685-75c3c446912d\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-fc495e-76f6c89b8b-rw8hz"
Sep 10 15:44:14.396206 ip-10-0-128-241 kubenswrapper[2567]: E0910 15:44:14.396120    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a7ca8d8b-db0b-47af-b685-75c3c446912d-maas-api-tls podName:a7ca8d8b-db0b-47af-b685-75c3c446912d nodeName:}" failed. No retries permitted until 2026-09-10 15:44:14.896097364 +0000 UTC m=+1040.464329078 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/a7ca8d8b-db0b-47af-b685-75c3c446912d-maas-api-tls") pod "maas-api-e2e-explicit-fc495e-76f6c89b8b-rw8hz" (UID: "a7ca8d8b-db0b-47af-b685-75c3c446912d") : secret "maas-api-e2e-explicit-fc495e-serving-cert" not found
Sep 10 15:44:14.396422 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:44:14.396204    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a7ca8d8b-db0b-47af-b685-75c3c446912d-openshift-service-ca\") pod \"maas-api-e2e-explicit-fc495e-76f6c89b8b-rw8hz\" (UID: \"a7ca8d8b-db0b-47af-b685-75c3c446912d\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-fc495e-76f6c89b8b-rw8hz"
Sep 10 15:44:14.396844 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:44:14.396818    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a7ca8d8b-db0b-47af-b685-75c3c446912d-openshift-service-ca\") pod \"maas-api-e2e-explicit-fc495e-76f6c89b8b-rw8hz\" (UID: \"a7ca8d8b-db0b-47af-b685-75c3c446912d\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-fc495e-76f6c89b8b-rw8hz"
Sep 10 15:44:14.398629 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:44:14.398608    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a7ca8d8b-db0b-47af-b685-75c3c446912d-metrics-certs\") pod \"maas-api-e2e-explicit-fc495e-76f6c89b8b-rw8hz\" (UID: \"a7ca8d8b-db0b-47af-b685-75c3c446912d\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-fc495e-76f6c89b8b-rw8hz"
Sep 10 15:44:14.405372 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:44:14.405350    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lmbdw\" (UniqueName: \"kubernetes.io/projected/a7ca8d8b-db0b-47af-b685-75c3c446912d-kube-api-access-lmbdw\") pod \"maas-api-e2e-explicit-fc495e-76f6c89b8b-rw8hz\" (UID: \"a7ca8d8b-db0b-47af-b685-75c3c446912d\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-fc495e-76f6c89b8b-rw8hz"
Sep 10 15:44:14.708131 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:44:14.708098    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:44:14.806573 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:44:14.806537    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:44:14.897180 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:44:14.897145    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:44:14.900427 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:44:14.900399    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a7ca8d8b-db0b-47af-b685-75c3c446912d-maas-api-tls\") pod \"maas-api-e2e-explicit-fc495e-76f6c89b8b-rw8hz\" (UID: \"a7ca8d8b-db0b-47af-b685-75c3c446912d\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-fc495e-76f6c89b8b-rw8hz"
Sep 10 15:44:14.902984 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:44:14.902963    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a7ca8d8b-db0b-47af-b685-75c3c446912d-maas-api-tls\") pod \"maas-api-e2e-explicit-fc495e-76f6c89b8b-rw8hz\" (UID: \"a7ca8d8b-db0b-47af-b685-75c3c446912d\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-fc495e-76f6c89b8b-rw8hz"
Sep 10 15:44:15.140370 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:44:15.140282    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-fc495e-76f6c89b8b-rw8hz"
Sep 10 15:44:15.415545 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:44:15.415519    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-fc495e-76f6c89b8b-rw8hz"]
Sep 10 15:44:15.418037 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:44:15.418002    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda7ca8d8b_db0b_47af_b685_75c3c446912d.slice/crio-01c4fd8c7df9bf6bb8fd1c89d0789efd55f3b022473199131e30d1ab41ec437c WatchSource:0}: Error finding container 01c4fd8c7df9bf6bb8fd1c89d0789efd55f3b022473199131e30d1ab41ec437c: Status 404 returned error can't find the container with id 01c4fd8c7df9bf6bb8fd1c89d0789efd55f3b022473199131e30d1ab41ec437c
Sep 10 15:44:15.469196 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:44:15.469160    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-fc495e-76f6c89b8b-rw8hz" event={"ID":"a7ca8d8b-db0b-47af-b685-75c3c446912d","Type":"ContainerStarted","Data":"01c4fd8c7df9bf6bb8fd1c89d0789efd55f3b022473199131e30d1ab41ec437c"}
Sep 10 15:44:16.475744 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:44:16.475707    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-fc495e-76f6c89b8b-rw8hz" event={"ID":"a7ca8d8b-db0b-47af-b685-75c3c446912d","Type":"ContainerStarted","Data":"7e8baf5fbd22c4fbadb43df2277ee15c21f6b5824391a76ca7127a8a1e0f8787"}
Sep 10 15:44:16.476127 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:44:16.475837    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-fc495e-76f6c89b8b-rw8hz"
Sep 10 15:44:16.495870 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:44:16.495813    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-fc495e-76f6c89b8b-rw8hz" podStartSLOduration=2.052816294 podStartE2EDuration="2.495794195s" totalImagesPullingTime="442.977901ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 15:44:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 15:44:16.494363179 +0000 UTC m=+1042.062594913" watchObservedRunningTime="2026-09-10 15:44:16.495794195 +0000 UTC m=+1042.064025954"
Sep 10 15:44:17.122525 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:44:17.122492    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:44:17.230681 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:44:17.230645    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:44:17.326169 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:44:17.326140    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:44:17.428391 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:44:17.428361    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:44:17.833614 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:44:17.833526    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:44:20.469510 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:44:20.469460    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-conflict-b-fa9f2a54-kserve-86d68b877-rq7rv" podUID="a1381c6f-a19e-49cd-86b9-a159effffce2" containerName="main" containerID="cri-o://379450663d742fe0fb31f88452cc060ad56983310b12fd682afcec36e45b4ed8" gracePeriod=45
Sep 10 15:44:20.725070 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:44:20.724999    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-conflict-b-fa9f2a54-kserve-86d68b877-rq7rv"
Sep 10 15:44:20.753367 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:44:20.753334    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a1381c6f-a19e-49cd-86b9-a159effffce2-home\") pod \"a1381c6f-a19e-49cd-86b9-a159effffce2\" (UID: \"a1381c6f-a19e-49cd-86b9-a159effffce2\") "
Sep 10 15:44:20.753532 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:44:20.753384    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a1381c6f-a19e-49cd-86b9-a159effffce2-tls-certs\") pod \"a1381c6f-a19e-49cd-86b9-a159effffce2\" (UID: \"a1381c6f-a19e-49cd-86b9-a159effffce2\") "
Sep 10 15:44:20.753532 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:44:20.753448    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wsvtg\" (UniqueName: \"kubernetes.io/projected/a1381c6f-a19e-49cd-86b9-a159effffce2-kube-api-access-wsvtg\") pod \"a1381c6f-a19e-49cd-86b9-a159effffce2\" (UID: \"a1381c6f-a19e-49cd-86b9-a159effffce2\") "
Sep 10 15:44:20.753532 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:44:20.753466    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a1381c6f-a19e-49cd-86b9-a159effffce2-dshm\") pod \"a1381c6f-a19e-49cd-86b9-a159effffce2\" (UID: \"a1381c6f-a19e-49cd-86b9-a159effffce2\") "
Sep 10 15:44:20.753532 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:44:20.753497    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a1381c6f-a19e-49cd-86b9-a159effffce2-tmp-dir\") pod \"a1381c6f-a19e-49cd-86b9-a159effffce2\" (UID: \"a1381c6f-a19e-49cd-86b9-a159effffce2\") "
Sep 10 15:44:20.753754 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:44:20.753570    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a1381c6f-a19e-49cd-86b9-a159effffce2-model-cache\") pod \"a1381c6f-a19e-49cd-86b9-a159effffce2\" (UID: \"a1381c6f-a19e-49cd-86b9-a159effffce2\") "
Sep 10 15:44:20.753754 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:44:20.753670    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a1381c6f-a19e-49cd-86b9-a159effffce2-home" (OuterVolumeSpecName: "home") pod "a1381c6f-a19e-49cd-86b9-a159effffce2" (UID: "a1381c6f-a19e-49cd-86b9-a159effffce2"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 15:44:20.753860 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:44:20.753799    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a1381c6f-a19e-49cd-86b9-a159effffce2-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "a1381c6f-a19e-49cd-86b9-a159effffce2" (UID: "a1381c6f-a19e-49cd-86b9-a159effffce2"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 15:44:20.753915 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:44:20.753857    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a1381c6f-a19e-49cd-86b9-a159effffce2-model-cache" (OuterVolumeSpecName: "model-cache") pod "a1381c6f-a19e-49cd-86b9-a159effffce2" (UID: "a1381c6f-a19e-49cd-86b9-a159effffce2"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 15:44:20.753969 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:44:20.753925    2567 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a1381c6f-a19e-49cd-86b9-a159effffce2-home\") on node \"ip-10-0-128-241.ec2.internal\" DevicePath \"\""
Sep 10 15:44:20.753969 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:44:20.753946    2567 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a1381c6f-a19e-49cd-86b9-a159effffce2-tmp-dir\") on node \"ip-10-0-128-241.ec2.internal\" DevicePath \"\""
Sep 10 15:44:20.756208 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:44:20.756182    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a1381c6f-a19e-49cd-86b9-a159effffce2-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "a1381c6f-a19e-49cd-86b9-a159effffce2" (UID: "a1381c6f-a19e-49cd-86b9-a159effffce2"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 15:44:20.756344 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:44:20.756182    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a1381c6f-a19e-49cd-86b9-a159effffce2-dshm" (OuterVolumeSpecName: "dshm") pod "a1381c6f-a19e-49cd-86b9-a159effffce2" (UID: "a1381c6f-a19e-49cd-86b9-a159effffce2"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 15:44:20.756344 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:44:20.756314    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a1381c6f-a19e-49cd-86b9-a159effffce2-kube-api-access-wsvtg" (OuterVolumeSpecName: "kube-api-access-wsvtg") pod "a1381c6f-a19e-49cd-86b9-a159effffce2" (UID: "a1381c6f-a19e-49cd-86b9-a159effffce2"). InnerVolumeSpecName "kube-api-access-wsvtg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 15:44:20.854446 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:44:20.854411    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wsvtg\" (UniqueName: \"kubernetes.io/projected/a1381c6f-a19e-49cd-86b9-a159effffce2-kube-api-access-wsvtg\") on node \"ip-10-0-128-241.ec2.internal\" DevicePath \"\""
Sep 10 15:44:20.854446 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:44:20.854440    2567 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a1381c6f-a19e-49cd-86b9-a159effffce2-dshm\") on node \"ip-10-0-128-241.ec2.internal\" DevicePath \"\""
Sep 10 15:44:20.854446 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:44:20.854450    2567 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a1381c6f-a19e-49cd-86b9-a159effffce2-model-cache\") on node \"ip-10-0-128-241.ec2.internal\" DevicePath \"\""
Sep 10 15:44:20.854690 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:44:20.854459    2567 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a1381c6f-a19e-49cd-86b9-a159effffce2-tls-certs\") on node \"ip-10-0-128-241.ec2.internal\" DevicePath \"\""
Sep 10 15:44:21.092188 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:44:21.092094    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:44:21.193687 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:44:21.193652    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:44:21.331017 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:44:21.330979    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:44:21.502077 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:44:21.502035    2567 generic.go:356] "Generic (PLEG): container finished" podID="a1381c6f-a19e-49cd-86b9-a159effffce2" containerID="379450663d742fe0fb31f88452cc060ad56983310b12fd682afcec36e45b4ed8" exitCode=0
Sep 10 15:44:21.502587 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:44:21.502082    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-conflict-b-fa9f2a54-kserve-86d68b877-rq7rv"
Sep 10 15:44:21.502587 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:44:21.502123    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-conflict-b-fa9f2a54-kserve-86d68b877-rq7rv" event={"ID":"a1381c6f-a19e-49cd-86b9-a159effffce2","Type":"ContainerDied","Data":"379450663d742fe0fb31f88452cc060ad56983310b12fd682afcec36e45b4ed8"}
Sep 10 15:44:21.502587 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:44:21.502164    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-conflict-b-fa9f2a54-kserve-86d68b877-rq7rv" event={"ID":"a1381c6f-a19e-49cd-86b9-a159effffce2","Type":"ContainerDied","Data":"1d5d90573cb2d21477e776a085dd6037b8ab20d69e8aa4c6c42f49f0acfee728"}
Sep 10 15:44:21.502587 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:44:21.502187    2567 scope.go:117] "RemoveContainer" containerID="379450663d742fe0fb31f88452cc060ad56983310b12fd682afcec36e45b4ed8"
Sep 10 15:44:21.510572 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:44:21.510555    2567 scope.go:117] "RemoveContainer" containerID="379450663d742fe0fb31f88452cc060ad56983310b12fd682afcec36e45b4ed8"
Sep 10 15:44:21.510823 ip-10-0-128-241 kubenswrapper[2567]: E0910 15:44:21.510807    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"379450663d742fe0fb31f88452cc060ad56983310b12fd682afcec36e45b4ed8\": container with ID starting with 379450663d742fe0fb31f88452cc060ad56983310b12fd682afcec36e45b4ed8 not found: ID does not exist" containerID="379450663d742fe0fb31f88452cc060ad56983310b12fd682afcec36e45b4ed8"
Sep 10 15:44:21.510874 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:44:21.510831    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"379450663d742fe0fb31f88452cc060ad56983310b12fd682afcec36e45b4ed8"} err="failed to get container status \"379450663d742fe0fb31f88452cc060ad56983310b12fd682afcec36e45b4ed8\": rpc error: code = NotFound desc = could not find container \"379450663d742fe0fb31f88452cc060ad56983310b12fd682afcec36e45b4ed8\": container with ID starting with 379450663d742fe0fb31f88452cc060ad56983310b12fd682afcec36e45b4ed8 not found: ID does not exist"
Sep 10 15:44:21.520080 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:44:21.520053    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-conflict-b-fa9f2a54-kserve-86d68b877-rq7rv"]
Sep 10 15:44:21.527179 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:44:21.527153    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-conflict-b-fa9f2a54-kserve-86d68b877-rq7rv"]
Sep 10 15:44:22.486587 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:44:22.486556    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-fc495e-76f6c89b8b-rw8hz"
Sep 10 15:44:22.904224 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:44:22.904143    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a1381c6f-a19e-49cd-86b9-a159effffce2" path="/var/lib/kubelet/pods/a1381c6f-a19e-49cd-86b9-a159effffce2/volumes"
Sep 10 15:44:22.910101 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:44:22.910072    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-b-90ca97/test-model-90ca97-kserve-6597c9f47f-g6cdl"]
Sep 10 15:44:22.910490 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:44:22.910475    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a1381c6f-a19e-49cd-86b9-a159effffce2" containerName="main"
Sep 10 15:44:22.910550 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:44:22.910494    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="a1381c6f-a19e-49cd-86b9-a159effffce2" containerName="main"
Sep 10 15:44:22.910595 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:44:22.910575    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="a1381c6f-a19e-49cd-86b9-a159effffce2" containerName="main"
Sep 10 15:44:22.913455 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:44:22.913438    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-90ca97/test-model-90ca97-kserve-6597c9f47f-g6cdl"
Sep 10 15:44:22.915999 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:44:22.915977    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-90ca97\"/\"kube-root-ca.crt\""
Sep 10 15:44:22.916112 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:44:22.916028    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-90ca97\"/\"openshift-service-ca.crt\""
Sep 10 15:44:22.916742 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:44:22.916726    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-90ca97\"/\"test-model-90ca97-kserve-self-signed-certs\""
Sep 10 15:44:22.916831 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:44:22.916730    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-90ca97\"/\"default-dockercfg-kdh49\""
Sep 10 15:44:22.922859 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:44:22.922835    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-90ca97/test-model-90ca97-kserve-6597c9f47f-g6cdl"]
Sep 10 15:44:22.974943 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:44:22.974903    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/30ae33ce-692b-4493-bfc4-e7b45c56dfb2-home\") pod \"test-model-90ca97-kserve-6597c9f47f-g6cdl\" (UID: \"30ae33ce-692b-4493-bfc4-e7b45c56dfb2\") " pod="ai-tenant-e2e-inf-b-90ca97/test-model-90ca97-kserve-6597c9f47f-g6cdl"
Sep 10 15:44:22.974943 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:44:22.974936    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/30ae33ce-692b-4493-bfc4-e7b45c56dfb2-model-cache\") pod \"test-model-90ca97-kserve-6597c9f47f-g6cdl\" (UID: \"30ae33ce-692b-4493-bfc4-e7b45c56dfb2\") " pod="ai-tenant-e2e-inf-b-90ca97/test-model-90ca97-kserve-6597c9f47f-g6cdl"
Sep 10 15:44:22.975198 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:44:22.974963    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/30ae33ce-692b-4493-bfc4-e7b45c56dfb2-dshm\") pod \"test-model-90ca97-kserve-6597c9f47f-g6cdl\" (UID: \"30ae33ce-692b-4493-bfc4-e7b45c56dfb2\") " pod="ai-tenant-e2e-inf-b-90ca97/test-model-90ca97-kserve-6597c9f47f-g6cdl"
Sep 10 15:44:22.975198 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:44:22.975120    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gxj4z\" (UniqueName: \"kubernetes.io/projected/30ae33ce-692b-4493-bfc4-e7b45c56dfb2-kube-api-access-gxj4z\") pod \"test-model-90ca97-kserve-6597c9f47f-g6cdl\" (UID: \"30ae33ce-692b-4493-bfc4-e7b45c56dfb2\") " pod="ai-tenant-e2e-inf-b-90ca97/test-model-90ca97-kserve-6597c9f47f-g6cdl"
Sep 10 15:44:22.975198 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:44:22.975160    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/30ae33ce-692b-4493-bfc4-e7b45c56dfb2-tmp-dir\") pod \"test-model-90ca97-kserve-6597c9f47f-g6cdl\" (UID: \"30ae33ce-692b-4493-bfc4-e7b45c56dfb2\") " pod="ai-tenant-e2e-inf-b-90ca97/test-model-90ca97-kserve-6597c9f47f-g6cdl"
Sep 10 15:44:22.975335 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:44:22.975257    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/30ae33ce-692b-4493-bfc4-e7b45c56dfb2-tls-certs\") pod \"test-model-90ca97-kserve-6597c9f47f-g6cdl\" (UID: \"30ae33ce-692b-4493-bfc4-e7b45c56dfb2\") " pod="ai-tenant-e2e-inf-b-90ca97/test-model-90ca97-kserve-6597c9f47f-g6cdl"
Sep 10 15:44:23.076061 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:44:23.076020    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gxj4z\" (UniqueName: \"kubernetes.io/projected/30ae33ce-692b-4493-bfc4-e7b45c56dfb2-kube-api-access-gxj4z\") pod \"test-model-90ca97-kserve-6597c9f47f-g6cdl\" (UID: \"30ae33ce-692b-4493-bfc4-e7b45c56dfb2\") " pod="ai-tenant-e2e-inf-b-90ca97/test-model-90ca97-kserve-6597c9f47f-g6cdl"
Sep 10 15:44:23.076061 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:44:23.076067    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/30ae33ce-692b-4493-bfc4-e7b45c56dfb2-tmp-dir\") pod \"test-model-90ca97-kserve-6597c9f47f-g6cdl\" (UID: \"30ae33ce-692b-4493-bfc4-e7b45c56dfb2\") " pod="ai-tenant-e2e-inf-b-90ca97/test-model-90ca97-kserve-6597c9f47f-g6cdl"
Sep 10 15:44:23.076366 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:44:23.076113    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/30ae33ce-692b-4493-bfc4-e7b45c56dfb2-tls-certs\") pod \"test-model-90ca97-kserve-6597c9f47f-g6cdl\" (UID: \"30ae33ce-692b-4493-bfc4-e7b45c56dfb2\") " pod="ai-tenant-e2e-inf-b-90ca97/test-model-90ca97-kserve-6597c9f47f-g6cdl"
Sep 10 15:44:23.076366 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:44:23.076146    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/30ae33ce-692b-4493-bfc4-e7b45c56dfb2-home\") pod \"test-model-90ca97-kserve-6597c9f47f-g6cdl\" (UID: \"30ae33ce-692b-4493-bfc4-e7b45c56dfb2\") " pod="ai-tenant-e2e-inf-b-90ca97/test-model-90ca97-kserve-6597c9f47f-g6cdl"
Sep 10 15:44:23.076366 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:44:23.076168    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/30ae33ce-692b-4493-bfc4-e7b45c56dfb2-model-cache\") pod \"test-model-90ca97-kserve-6597c9f47f-g6cdl\" (UID: \"30ae33ce-692b-4493-bfc4-e7b45c56dfb2\") " pod="ai-tenant-e2e-inf-b-90ca97/test-model-90ca97-kserve-6597c9f47f-g6cdl"
Sep 10 15:44:23.076366 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:44:23.076202    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/30ae33ce-692b-4493-bfc4-e7b45c56dfb2-dshm\") pod \"test-model-90ca97-kserve-6597c9f47f-g6cdl\" (UID: \"30ae33ce-692b-4493-bfc4-e7b45c56dfb2\") " pod="ai-tenant-e2e-inf-b-90ca97/test-model-90ca97-kserve-6597c9f47f-g6cdl"
Sep 10 15:44:23.076610 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:44:23.076586    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/30ae33ce-692b-4493-bfc4-e7b45c56dfb2-home\") pod \"test-model-90ca97-kserve-6597c9f47f-g6cdl\" (UID: \"30ae33ce-692b-4493-bfc4-e7b45c56dfb2\") " pod="ai-tenant-e2e-inf-b-90ca97/test-model-90ca97-kserve-6597c9f47f-g6cdl"
Sep 10 15:44:23.076677 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:44:23.076590    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/30ae33ce-692b-4493-bfc4-e7b45c56dfb2-tmp-dir\") pod \"test-model-90ca97-kserve-6597c9f47f-g6cdl\" (UID: \"30ae33ce-692b-4493-bfc4-e7b45c56dfb2\") " pod="ai-tenant-e2e-inf-b-90ca97/test-model-90ca97-kserve-6597c9f47f-g6cdl"
Sep 10 15:44:23.076726 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:44:23.076676    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/30ae33ce-692b-4493-bfc4-e7b45c56dfb2-model-cache\") pod \"test-model-90ca97-kserve-6597c9f47f-g6cdl\" (UID: \"30ae33ce-692b-4493-bfc4-e7b45c56dfb2\") " pod="ai-tenant-e2e-inf-b-90ca97/test-model-90ca97-kserve-6597c9f47f-g6cdl"
Sep 10 15:44:23.078482 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:44:23.078460    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/30ae33ce-692b-4493-bfc4-e7b45c56dfb2-dshm\") pod \"test-model-90ca97-kserve-6597c9f47f-g6cdl\" (UID: \"30ae33ce-692b-4493-bfc4-e7b45c56dfb2\") " pod="ai-tenant-e2e-inf-b-90ca97/test-model-90ca97-kserve-6597c9f47f-g6cdl"
Sep 10 15:44:23.078861 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:44:23.078841    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/30ae33ce-692b-4493-bfc4-e7b45c56dfb2-tls-certs\") pod \"test-model-90ca97-kserve-6597c9f47f-g6cdl\" (UID: \"30ae33ce-692b-4493-bfc4-e7b45c56dfb2\") " pod="ai-tenant-e2e-inf-b-90ca97/test-model-90ca97-kserve-6597c9f47f-g6cdl"
Sep 10 15:44:23.085835 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:44:23.085803    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gxj4z\" (UniqueName: \"kubernetes.io/projected/30ae33ce-692b-4493-bfc4-e7b45c56dfb2-kube-api-access-gxj4z\") pod \"test-model-90ca97-kserve-6597c9f47f-g6cdl\" (UID: \"30ae33ce-692b-4493-bfc4-e7b45c56dfb2\") " pod="ai-tenant-e2e-inf-b-90ca97/test-model-90ca97-kserve-6597c9f47f-g6cdl"
Sep 10 15:44:23.211699 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:44:23.211663    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:44:23.222712 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:44:23.222683    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-90ca97/test-model-90ca97-kserve-6597c9f47f-g6cdl"
Sep 10 15:44:23.504194 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:44:23.504127    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-90ca97/test-model-90ca97-kserve-6597c9f47f-g6cdl"]
Sep 10 15:44:23.506968 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:44:23.506943    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod30ae33ce_692b_4493_bfc4_e7b45c56dfb2.slice/crio-10665ac9584538ba75e3f1407613ad768f0d66b036c781eb96944c83b0fabd27 WatchSource:0}: Error finding container 10665ac9584538ba75e3f1407613ad768f0d66b036c781eb96944c83b0fabd27: Status 404 returned error can't find the container with id 10665ac9584538ba75e3f1407613ad768f0d66b036c781eb96944c83b0fabd27
Sep 10 15:44:23.514443 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:44:23.514417    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-90ca97/test-model-90ca97-kserve-6597c9f47f-g6cdl" event={"ID":"30ae33ce-692b-4493-bfc4-e7b45c56dfb2","Type":"ContainerStarted","Data":"10665ac9584538ba75e3f1407613ad768f0d66b036c781eb96944c83b0fabd27"}
Sep 10 15:44:24.520558 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:44:24.520519    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-90ca97/test-model-90ca97-kserve-6597c9f47f-g6cdl" event={"ID":"30ae33ce-692b-4493-bfc4-e7b45c56dfb2","Type":"ContainerStarted","Data":"68087a091f4d633270242b5a35653e69eb749857e8cd573fa67068c3ccb1d41e"}
Sep 10 15:44:24.542328 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:44:24.542281    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-b-90ca97/test-model-90ca97-kserve-6597c9f47f-g6cdl" podStartSLOduration=2.54226533 podStartE2EDuration="2.54226533s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 15:44:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 15:44:24.540932815 +0000 UTC m=+1050.109164546" watchObservedRunningTime="2026-09-10 15:44:24.54226533 +0000 UTC m=+1050.110497119"
Sep 10 15:44:28.401641 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:44:28.401575    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:44:33.223944 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:44:33.223879    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-b-90ca97/test-model-90ca97-kserve-6597c9f47f-g6cdl"
Sep 10 15:44:33.223944 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:44:33.223949    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-b-90ca97/test-model-90ca97-kserve-6597c9f47f-g6cdl"
Sep 10 15:44:33.236688 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:44:33.236658    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-b-90ca97/test-model-90ca97-kserve-6597c9f47f-g6cdl"
Sep 10 15:44:33.574274 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:44:33.574180    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-b-90ca97/test-model-90ca97-kserve-6597c9f47f-g6cdl"
Sep 10 15:44:36.530523 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:44:36.530487    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:44:36.842605 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:44:36.842521    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:44:37.537407 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:44:37.537374    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:44:38.354002 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:44:38.353970    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:44:39.280756 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:44:39.280728    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:44:43.752114 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:44:43.752080    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:44:50.131220 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:44:50.131140    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:44:50.234921 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:44:50.234880    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:44:50.363160 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:44:50.363127    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:45:06.233899 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:45:06.233853    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:45:06.302435 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:45:06.302397    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:45:08.907872 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:45:08.907831    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:45:10.058810 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:45:10.058768    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:45:10.841726 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:45:10.841686    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:45:11.733507 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:45:11.733463    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:45:11.834378 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:45:11.834345    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:45:12.531464 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:45:12.531434    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:45:13.580469 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:45:13.580430    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:45:13.650517 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:45:13.650479    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:45:13.922402 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:45:13.922370    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:45:14.029539 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:45:14.029504    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:45:19.686594 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:45:19.686559    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-fc495e-76f6c89b8b-rw8hz"]
Sep 10 15:45:19.687055 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:45:19.686809    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-fc495e-76f6c89b8b-rw8hz" podUID="a7ca8d8b-db0b-47af-b685-75c3c446912d" containerName="maas-api" containerID="cri-o://7e8baf5fbd22c4fbadb43df2277ee15c21f6b5824391a76ca7127a8a1e0f8787" gracePeriod=30
Sep 10 15:45:19.720614 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:45:19.720584    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:45:19.797727 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:45:19.797197    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:45:19.938539 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:45:19.938460    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-fc495e-76f6c89b8b-rw8hz"
Sep 10 15:45:19.994342 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:45:19.994307    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a7ca8d8b-db0b-47af-b685-75c3c446912d-metrics-certs\") pod \"a7ca8d8b-db0b-47af-b685-75c3c446912d\" (UID: \"a7ca8d8b-db0b-47af-b685-75c3c446912d\") "
Sep 10 15:45:19.994512 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:45:19.994430    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a7ca8d8b-db0b-47af-b685-75c3c446912d-openshift-service-ca\") pod \"a7ca8d8b-db0b-47af-b685-75c3c446912d\" (UID: \"a7ca8d8b-db0b-47af-b685-75c3c446912d\") "
Sep 10 15:45:19.994512 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:45:19.994449    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lmbdw\" (UniqueName: \"kubernetes.io/projected/a7ca8d8b-db0b-47af-b685-75c3c446912d-kube-api-access-lmbdw\") pod \"a7ca8d8b-db0b-47af-b685-75c3c446912d\" (UID: \"a7ca8d8b-db0b-47af-b685-75c3c446912d\") "
Sep 10 15:45:19.994512 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:45:19.994488    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a7ca8d8b-db0b-47af-b685-75c3c446912d-maas-api-tls\") pod \"a7ca8d8b-db0b-47af-b685-75c3c446912d\" (UID: \"a7ca8d8b-db0b-47af-b685-75c3c446912d\") "
Sep 10 15:45:19.994833 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:45:19.994797    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a7ca8d8b-db0b-47af-b685-75c3c446912d-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "a7ca8d8b-db0b-47af-b685-75c3c446912d" (UID: "a7ca8d8b-db0b-47af-b685-75c3c446912d"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 15:45:19.995102 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:45:19.995057    2567 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a7ca8d8b-db0b-47af-b685-75c3c446912d-openshift-service-ca\") on node \"ip-10-0-128-241.ec2.internal\" DevicePath \"\""
Sep 10 15:45:19.996701 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:45:19.996678    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a7ca8d8b-db0b-47af-b685-75c3c446912d-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "a7ca8d8b-db0b-47af-b685-75c3c446912d" (UID: "a7ca8d8b-db0b-47af-b685-75c3c446912d"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 15:45:19.997077 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:45:19.997048    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a7ca8d8b-db0b-47af-b685-75c3c446912d-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "a7ca8d8b-db0b-47af-b685-75c3c446912d" (UID: "a7ca8d8b-db0b-47af-b685-75c3c446912d"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 15:45:19.997077 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:45:19.997061    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a7ca8d8b-db0b-47af-b685-75c3c446912d-kube-api-access-lmbdw" (OuterVolumeSpecName: "kube-api-access-lmbdw") pod "a7ca8d8b-db0b-47af-b685-75c3c446912d" (UID: "a7ca8d8b-db0b-47af-b685-75c3c446912d"). InnerVolumeSpecName "kube-api-access-lmbdw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 15:45:20.095761 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:45:20.095728    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lmbdw\" (UniqueName: \"kubernetes.io/projected/a7ca8d8b-db0b-47af-b685-75c3c446912d-kube-api-access-lmbdw\") on node \"ip-10-0-128-241.ec2.internal\" DevicePath \"\""
Sep 10 15:45:20.095761 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:45:20.095760    2567 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a7ca8d8b-db0b-47af-b685-75c3c446912d-maas-api-tls\") on node \"ip-10-0-128-241.ec2.internal\" DevicePath \"\""
Sep 10 15:45:20.095956 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:45:20.095772    2567 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a7ca8d8b-db0b-47af-b685-75c3c446912d-metrics-certs\") on node \"ip-10-0-128-241.ec2.internal\" DevicePath \"\""
Sep 10 15:45:20.795384 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:45:20.795348    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:45:20.798322 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:45:20.798289    2567 generic.go:356] "Generic (PLEG): container finished" podID="a7ca8d8b-db0b-47af-b685-75c3c446912d" containerID="7e8baf5fbd22c4fbadb43df2277ee15c21f6b5824391a76ca7127a8a1e0f8787" exitCode=0
Sep 10 15:45:20.798696 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:45:20.798353    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-fc495e-76f6c89b8b-rw8hz"
Sep 10 15:45:20.798696 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:45:20.798350    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-fc495e-76f6c89b8b-rw8hz" event={"ID":"a7ca8d8b-db0b-47af-b685-75c3c446912d","Type":"ContainerDied","Data":"7e8baf5fbd22c4fbadb43df2277ee15c21f6b5824391a76ca7127a8a1e0f8787"}
Sep 10 15:45:20.798696 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:45:20.798509    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-fc495e-76f6c89b8b-rw8hz" event={"ID":"a7ca8d8b-db0b-47af-b685-75c3c446912d","Type":"ContainerDied","Data":"01c4fd8c7df9bf6bb8fd1c89d0789efd55f3b022473199131e30d1ab41ec437c"}
Sep 10 15:45:20.798696 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:45:20.798543    2567 scope.go:117] "RemoveContainer" containerID="7e8baf5fbd22c4fbadb43df2277ee15c21f6b5824391a76ca7127a8a1e0f8787"
Sep 10 15:45:20.808961 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:45:20.808940    2567 scope.go:117] "RemoveContainer" containerID="7e8baf5fbd22c4fbadb43df2277ee15c21f6b5824391a76ca7127a8a1e0f8787"
Sep 10 15:45:20.809296 ip-10-0-128-241 kubenswrapper[2567]: E0910 15:45:20.809275    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7e8baf5fbd22c4fbadb43df2277ee15c21f6b5824391a76ca7127a8a1e0f8787\": container with ID starting with 7e8baf5fbd22c4fbadb43df2277ee15c21f6b5824391a76ca7127a8a1e0f8787 not found: ID does not exist" containerID="7e8baf5fbd22c4fbadb43df2277ee15c21f6b5824391a76ca7127a8a1e0f8787"
Sep 10 15:45:20.809358 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:45:20.809306    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7e8baf5fbd22c4fbadb43df2277ee15c21f6b5824391a76ca7127a8a1e0f8787"} err="failed to get container status \"7e8baf5fbd22c4fbadb43df2277ee15c21f6b5824391a76ca7127a8a1e0f8787\": rpc error: code = NotFound desc = could not find container \"7e8baf5fbd22c4fbadb43df2277ee15c21f6b5824391a76ca7127a8a1e0f8787\": container with ID starting with 7e8baf5fbd22c4fbadb43df2277ee15c21f6b5824391a76ca7127a8a1e0f8787 not found: ID does not exist"
Sep 10 15:45:20.822798 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:45:20.822766    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-fc495e-76f6c89b8b-rw8hz"]
Sep 10 15:45:20.825052 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:45:20.825021    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-fc495e-76f6c89b8b-rw8hz"]
Sep 10 15:45:20.885625 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:45:20.885591    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:45:20.905099 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:45:20.905065    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a7ca8d8b-db0b-47af-b685-75c3c446912d" path="/var/lib/kubelet/pods/a7ca8d8b-db0b-47af-b685-75c3c446912d/volumes"
Sep 10 15:45:21.091585 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:45:21.091500    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:45:21.860153 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:45:21.860119    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:45:24.292081 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:45:24.292049    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:45:25.101728 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:45:25.101685    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:45:25.400521 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:45:25.400438    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:45:26.913299 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:45:26.913263    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:45:26.987633 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:45:26.987600    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:45:27.179202 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:45:27.179174    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:45:27.609655 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:45:27.609566    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:45:27.929287 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:45:27.929226    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:45:28.054260 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:45:28.054196    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:45:28.877459 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:45:28.877421    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:45:30.035730 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:45:30.035697    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:45:30.154106 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:45:30.154074    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:45:31.545876 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:45:31.545833    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:45:31.644260 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:45:31.644202    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:45:31.744848 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:45:31.744803    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:45:31.846807 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:45:31.846708    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:45:32.523071 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:45:32.523033    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:45:32.623056 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:45:32.623016    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:45:32.723435 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:45:32.723397    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:45:34.237222 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:45:34.237188    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:45:40.383254 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:45:40.383185    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:45:40.460021 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:45:40.459987    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:45:41.049659 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:45:41.049626    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:45:43.987590 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:45:43.987550    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:45:44.045348 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:45:44.045299    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:45:47.405300 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:45:47.405261    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:45:47.507633 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:45:47.507597    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:45:47.645821 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:45:47.645790    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:45:48.007744 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:45:48.007705    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:45:48.314945 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:45:48.314857    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:45:51.151392 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:45:51.151350    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:45:51.357549 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:45:51.357514    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:45:51.455675 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:45:51.455640    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:45:52.166938 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:45:52.166902    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:45:52.944773 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:45:52.944738    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-f4a9663c-674bc67747-8vltc"]
Sep 10 15:45:52.945360 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:45:52.945341    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a7ca8d8b-db0b-47af-b685-75c3c446912d" containerName="maas-api"
Sep 10 15:45:52.945423 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:45:52.945365    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="a7ca8d8b-db0b-47af-b685-75c3c446912d" containerName="maas-api"
Sep 10 15:45:52.945498 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:45:52.945485    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="a7ca8d8b-db0b-47af-b685-75c3c446912d" containerName="maas-api"
Sep 10 15:45:52.949000 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:45:52.948977    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-f4a9663c-674bc67747-8vltc"
Sep 10 15:45:52.951632 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:45:52.951608    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-f4a9663c-serving-cert\""
Sep 10 15:45:52.959509 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:45:52.959485    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-f4a9663c-674bc67747-8vltc"]
Sep 10 15:45:52.980145 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:45:52.980110    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:45:53.043134 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:45:53.043105    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:45:53.092489 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:45:53.092454    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/74336f8d-ef31-4383-970f-cf65586ed643-maas-api-tls\") pod \"maas-api-e2e-mt-f4a9663c-674bc67747-8vltc\" (UID: \"74336f8d-ef31-4383-970f-cf65586ed643\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-f4a9663c-674bc67747-8vltc"
Sep 10 15:45:53.092489 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:45:53.092487    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/74336f8d-ef31-4383-970f-cf65586ed643-openshift-service-ca\") pod \"maas-api-e2e-mt-f4a9663c-674bc67747-8vltc\" (UID: \"74336f8d-ef31-4383-970f-cf65586ed643\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-f4a9663c-674bc67747-8vltc"
Sep 10 15:45:53.092716 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:45:53.092695    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/74336f8d-ef31-4383-970f-cf65586ed643-metrics-certs\") pod \"maas-api-e2e-mt-f4a9663c-674bc67747-8vltc\" (UID: \"74336f8d-ef31-4383-970f-cf65586ed643\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-f4a9663c-674bc67747-8vltc"
Sep 10 15:45:53.092765 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:45:53.092745    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zlwp2\" (UniqueName: \"kubernetes.io/projected/74336f8d-ef31-4383-970f-cf65586ed643-kube-api-access-zlwp2\") pod \"maas-api-e2e-mt-f4a9663c-674bc67747-8vltc\" (UID: \"74336f8d-ef31-4383-970f-cf65586ed643\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-f4a9663c-674bc67747-8vltc"
Sep 10 15:45:53.143910 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:45:53.143868    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:45:53.193429 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:45:53.193390    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/74336f8d-ef31-4383-970f-cf65586ed643-maas-api-tls\") pod \"maas-api-e2e-mt-f4a9663c-674bc67747-8vltc\" (UID: \"74336f8d-ef31-4383-970f-cf65586ed643\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-f4a9663c-674bc67747-8vltc"
Sep 10 15:45:53.193429 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:45:53.193430    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/74336f8d-ef31-4383-970f-cf65586ed643-openshift-service-ca\") pod \"maas-api-e2e-mt-f4a9663c-674bc67747-8vltc\" (UID: \"74336f8d-ef31-4383-970f-cf65586ed643\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-f4a9663c-674bc67747-8vltc"
Sep 10 15:45:53.194002 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:45:53.193479    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/74336f8d-ef31-4383-970f-cf65586ed643-metrics-certs\") pod \"maas-api-e2e-mt-f4a9663c-674bc67747-8vltc\" (UID: \"74336f8d-ef31-4383-970f-cf65586ed643\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-f4a9663c-674bc67747-8vltc"
Sep 10 15:45:53.194002 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:45:53.193508    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zlwp2\" (UniqueName: \"kubernetes.io/projected/74336f8d-ef31-4383-970f-cf65586ed643-kube-api-access-zlwp2\") pod \"maas-api-e2e-mt-f4a9663c-674bc67747-8vltc\" (UID: \"74336f8d-ef31-4383-970f-cf65586ed643\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-f4a9663c-674bc67747-8vltc"
Sep 10 15:45:53.194279 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:45:53.194225    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/74336f8d-ef31-4383-970f-cf65586ed643-openshift-service-ca\") pod \"maas-api-e2e-mt-f4a9663c-674bc67747-8vltc\" (UID: \"74336f8d-ef31-4383-970f-cf65586ed643\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-f4a9663c-674bc67747-8vltc"
Sep 10 15:45:53.196063 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:45:53.196003    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/74336f8d-ef31-4383-970f-cf65586ed643-maas-api-tls\") pod \"maas-api-e2e-mt-f4a9663c-674bc67747-8vltc\" (UID: \"74336f8d-ef31-4383-970f-cf65586ed643\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-f4a9663c-674bc67747-8vltc"
Sep 10 15:45:53.196063 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:45:53.196054    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/74336f8d-ef31-4383-970f-cf65586ed643-metrics-certs\") pod \"maas-api-e2e-mt-f4a9663c-674bc67747-8vltc\" (UID: \"74336f8d-ef31-4383-970f-cf65586ed643\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-f4a9663c-674bc67747-8vltc"
Sep 10 15:45:53.203037 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:45:53.203012    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zlwp2\" (UniqueName: \"kubernetes.io/projected/74336f8d-ef31-4383-970f-cf65586ed643-kube-api-access-zlwp2\") pod \"maas-api-e2e-mt-f4a9663c-674bc67747-8vltc\" (UID: \"74336f8d-ef31-4383-970f-cf65586ed643\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-f4a9663c-674bc67747-8vltc"
Sep 10 15:45:53.262892 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:45:53.262856    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-f4a9663c-674bc67747-8vltc"
Sep 10 15:45:53.494105 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:45:53.494023    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:45:53.539726 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:45:53.539699    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-f4a9663c-674bc67747-8vltc"]
Sep 10 15:45:53.541998 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:45:53.541970    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74336f8d_ef31_4383_970f_cf65586ed643.slice/crio-0f51179e3001824a4b933554f25c6ff9bd4d7f2e4d962efe842454ea6e98d422 WatchSource:0}: Error finding container 0f51179e3001824a4b933554f25c6ff9bd4d7f2e4d962efe842454ea6e98d422: Status 404 returned error can't find the container with id 0f51179e3001824a4b933554f25c6ff9bd4d7f2e4d962efe842454ea6e98d422
Sep 10 15:45:53.603188 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:45:53.603145    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:45:53.604140 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:45:53.604121    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-f4a9663c-797b686986-mfgsw"]
Sep 10 15:45:53.608820 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:45:53.608797    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-f4a9663c-797b686986-mfgsw"
Sep 10 15:45:53.613652 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:45:53.613622    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-f4a9663c\""
Sep 10 15:45:53.613781 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:45:53.613627    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-f4a9663c-dockercfg-mxtwk\""
Sep 10 15:45:53.628873 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:45:53.627943    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-f4a9663c-797b686986-mfgsw"]
Sep 10 15:45:53.799535 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:45:53.799445    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ts8pl\" (UniqueName: \"kubernetes.io/projected/53520aaf-fc71-4178-8b2e-1c244f904eb8-kube-api-access-ts8pl\") pod \"payload-pre-processing-e2e-mt-f4a9663c-797b686986-mfgsw\" (UID: \"53520aaf-fc71-4178-8b2e-1c244f904eb8\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-f4a9663c-797b686986-mfgsw"
Sep 10 15:45:53.799712 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:45:53.799597    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/53520aaf-fc71-4178-8b2e-1c244f904eb8-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-f4a9663c-797b686986-mfgsw\" (UID: \"53520aaf-fc71-4178-8b2e-1c244f904eb8\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-f4a9663c-797b686986-mfgsw"
Sep 10 15:45:53.900804 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:45:53.900760    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/53520aaf-fc71-4178-8b2e-1c244f904eb8-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-f4a9663c-797b686986-mfgsw\" (UID: \"53520aaf-fc71-4178-8b2e-1c244f904eb8\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-f4a9663c-797b686986-mfgsw"
Sep 10 15:45:53.900993 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:45:53.900816    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ts8pl\" (UniqueName: \"kubernetes.io/projected/53520aaf-fc71-4178-8b2e-1c244f904eb8-kube-api-access-ts8pl\") pod \"payload-pre-processing-e2e-mt-f4a9663c-797b686986-mfgsw\" (UID: \"53520aaf-fc71-4178-8b2e-1c244f904eb8\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-f4a9663c-797b686986-mfgsw"
Sep 10 15:45:53.901568 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:45:53.901546    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/53520aaf-fc71-4178-8b2e-1c244f904eb8-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-f4a9663c-797b686986-mfgsw\" (UID: \"53520aaf-fc71-4178-8b2e-1c244f904eb8\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-f4a9663c-797b686986-mfgsw"
Sep 10 15:45:53.909988 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:45:53.909966    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ts8pl\" (UniqueName: \"kubernetes.io/projected/53520aaf-fc71-4178-8b2e-1c244f904eb8-kube-api-access-ts8pl\") pod \"payload-pre-processing-e2e-mt-f4a9663c-797b686986-mfgsw\" (UID: \"53520aaf-fc71-4178-8b2e-1c244f904eb8\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-f4a9663c-797b686986-mfgsw"
Sep 10 15:45:53.920831 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:45:53.920803    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-f4a9663c-797b686986-mfgsw"
Sep 10 15:45:53.953715 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:45:53.953680    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-f4a9663c-674bc67747-8vltc" event={"ID":"74336f8d-ef31-4383-970f-cf65586ed643","Type":"ContainerStarted","Data":"0f51179e3001824a4b933554f25c6ff9bd4d7f2e4d962efe842454ea6e98d422"}
Sep 10 15:45:54.192558 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:45:54.192528    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-f4a9663c-797b686986-mfgsw"]
Sep 10 15:45:54.194292 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:45:54.194264    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod53520aaf_fc71_4178_8b2e_1c244f904eb8.slice/crio-458687f665918c3212a630ed06e3468ffd9e662a7419b297ace9e9f87fe5997f WatchSource:0}: Error finding container 458687f665918c3212a630ed06e3468ffd9e662a7419b297ace9e9f87fe5997f: Status 404 returned error can't find the container with id 458687f665918c3212a630ed06e3468ffd9e662a7419b297ace9e9f87fe5997f
Sep 10 15:45:54.959810 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:45:54.959775    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-f4a9663c-674bc67747-8vltc" event={"ID":"74336f8d-ef31-4383-970f-cf65586ed643","Type":"ContainerStarted","Data":"e6a264dd521bf12b87af9d62bf7c525d7d4988fe9d08e6789b9ccbee9effd7d0"}
Sep 10 15:45:54.960005 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:45:54.959843    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-f4a9663c-674bc67747-8vltc"
Sep 10 15:45:54.961344 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:45:54.961322    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-f4a9663c-797b686986-mfgsw" event={"ID":"53520aaf-fc71-4178-8b2e-1c244f904eb8","Type":"ContainerStarted","Data":"f97d49caa7126fff04e74b7fc2d6fdc9ad8eb3349ee8270d348f08a55b8644a4"}
Sep 10 15:45:54.961483 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:45:54.961362    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-f4a9663c-797b686986-mfgsw" event={"ID":"53520aaf-fc71-4178-8b2e-1c244f904eb8","Type":"ContainerStarted","Data":"458687f665918c3212a630ed06e3468ffd9e662a7419b297ace9e9f87fe5997f"}
Sep 10 15:45:54.961483 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:45:54.961383    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-f4a9663c-797b686986-mfgsw"
Sep 10 15:45:54.978972 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:45:54.978912    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-f4a9663c-674bc67747-8vltc" podStartSLOduration=2.139507354 podStartE2EDuration="2.978896711s" totalImagesPullingTime="839.389357ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 15:45:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 15:45:54.976190624 +0000 UTC m=+1140.544422357" watchObservedRunningTime="2026-09-10 15:45:54.978896711 +0000 UTC m=+1140.547128440"
Sep 10 15:45:54.994159 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:45:54.994113    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-mt-f4a9663c-797b686986-mfgsw" podStartSLOduration=1.994097201 podStartE2EDuration="1.994097201s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 15:45:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 15:45:54.99187587 +0000 UTC m=+1140.560107609" watchObservedRunningTime="2026-09-10 15:45:54.994097201 +0000 UTC m=+1140.562328930"
Sep 10 15:45:57.418540 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:45:57.418500    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:46:00.955767 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:46:00.955727    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:46:00.975573 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:46:00.975542    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-f4a9663c-674bc67747-8vltc"
Sep 10 15:46:01.585905 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:46:01.585866    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-90ca97/test-model-90ca97-kserve-6597c9f47f-g6cdl"]
Sep 10 15:46:01.605542 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:46:01.605511    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:46:01.866220 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:46:01.866136    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:46:02.751869 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:46:02.751828    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:46:03.511828 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:46:03.511793    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:46:04.439669 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:46:04.439639    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:46:04.537832 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:46:04.537802    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:46:04.612962 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:46:04.612929    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:46:04.780202 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:46:04.780116    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:46:04.899569 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:46:04.899531    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:46:05.412897 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:46:05.412860    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:46:05.972578 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:46:05.972548    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-f4a9663c-797b686986-mfgsw"
Sep 10 15:46:06.059977 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:46:06.059947    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:46:06.369492 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:46:06.369398    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:46:06.660075 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:46:06.659981    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:46:06.757438 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:46:06.757405    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:46:08.382507 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:46:08.382468    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:46:08.482375 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:46:08.482344    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:46:09.900675 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:46:09.900638    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:46:11.351925 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:46:11.351889    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-139155-584bcf7687-nhzlf"]
Sep 10 15:46:11.352720 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:46:11.352689    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-139155-584bcf7687-nhzlf" podUID="53bb17de-fec9-4485-9850-f2d9a1c33d9e" containerName="maas-api" containerID="cri-o://ae4f548bcbd1f79998a621d3f767055a44e7c19006169d656c242c99da895606" gracePeriod=30
Sep 10 15:46:11.442759 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:46:11.442726    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-139155-7d477d9786-vhwg2"]
Sep 10 15:46:11.442981 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:46:11.442957    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-139155-7d477d9786-vhwg2" podUID="8ad3cae7-ba50-4435-8ee1-7494b6899ee1" containerName="payload-pre-processing" containerID="cri-o://fca38a64649665697f165114d6e23973e3e8a95b6e56fe10e66802c353fee1fe" gracePeriod=30
Sep 10 15:46:11.689689 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:46:11.689663    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-139155-584bcf7687-nhzlf"
Sep 10 15:46:11.721637 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:46:11.721610    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-139155-7d477d9786-vhwg2"
Sep 10 15:46:11.770251 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:46:11.770206    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/53bb17de-fec9-4485-9850-f2d9a1c33d9e-openshift-service-ca\") pod \"53bb17de-fec9-4485-9850-f2d9a1c33d9e\" (UID: \"53bb17de-fec9-4485-9850-f2d9a1c33d9e\") "
Sep 10 15:46:11.770407 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:46:11.770279    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2r2cl\" (UniqueName: \"kubernetes.io/projected/8ad3cae7-ba50-4435-8ee1-7494b6899ee1-kube-api-access-2r2cl\") pod \"8ad3cae7-ba50-4435-8ee1-7494b6899ee1\" (UID: \"8ad3cae7-ba50-4435-8ee1-7494b6899ee1\") "
Sep 10 15:46:11.770407 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:46:11.770343    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zfldt\" (UniqueName: \"kubernetes.io/projected/53bb17de-fec9-4485-9850-f2d9a1c33d9e-kube-api-access-zfldt\") pod \"53bb17de-fec9-4485-9850-f2d9a1c33d9e\" (UID: \"53bb17de-fec9-4485-9850-f2d9a1c33d9e\") "
Sep 10 15:46:11.770407 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:46:11.770366    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/53bb17de-fec9-4485-9850-f2d9a1c33d9e-maas-api-tls\") pod \"53bb17de-fec9-4485-9850-f2d9a1c33d9e\" (UID: \"53bb17de-fec9-4485-9850-f2d9a1c33d9e\") "
Sep 10 15:46:11.770407 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:46:11.770384    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8ad3cae7-ba50-4435-8ee1-7494b6899ee1-plugins-config-volume\") pod \"8ad3cae7-ba50-4435-8ee1-7494b6899ee1\" (UID: \"8ad3cae7-ba50-4435-8ee1-7494b6899ee1\") "
Sep 10 15:46:11.770620 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:46:11.770469    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/53bb17de-fec9-4485-9850-f2d9a1c33d9e-metrics-certs\") pod \"53bb17de-fec9-4485-9850-f2d9a1c33d9e\" (UID: \"53bb17de-fec9-4485-9850-f2d9a1c33d9e\") "
Sep 10 15:46:11.770620 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:46:11.770599    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/53bb17de-fec9-4485-9850-f2d9a1c33d9e-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "53bb17de-fec9-4485-9850-f2d9a1c33d9e" (UID: "53bb17de-fec9-4485-9850-f2d9a1c33d9e"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 15:46:11.770907 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:46:11.770874    2567 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/53bb17de-fec9-4485-9850-f2d9a1c33d9e-openshift-service-ca\") on node \"ip-10-0-128-241.ec2.internal\" DevicePath \"\""
Sep 10 15:46:11.771056 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:46:11.771016    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8ad3cae7-ba50-4435-8ee1-7494b6899ee1-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "8ad3cae7-ba50-4435-8ee1-7494b6899ee1" (UID: "8ad3cae7-ba50-4435-8ee1-7494b6899ee1"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 15:46:11.773177 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:46:11.773138    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8ad3cae7-ba50-4435-8ee1-7494b6899ee1-kube-api-access-2r2cl" (OuterVolumeSpecName: "kube-api-access-2r2cl") pod "8ad3cae7-ba50-4435-8ee1-7494b6899ee1" (UID: "8ad3cae7-ba50-4435-8ee1-7494b6899ee1"). InnerVolumeSpecName "kube-api-access-2r2cl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 15:46:11.773316 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:46:11.773191    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/53bb17de-fec9-4485-9850-f2d9a1c33d9e-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "53bb17de-fec9-4485-9850-f2d9a1c33d9e" (UID: "53bb17de-fec9-4485-9850-f2d9a1c33d9e"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 15:46:11.773316 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:46:11.773208    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/53bb17de-fec9-4485-9850-f2d9a1c33d9e-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "53bb17de-fec9-4485-9850-f2d9a1c33d9e" (UID: "53bb17de-fec9-4485-9850-f2d9a1c33d9e"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 15:46:11.773416 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:46:11.773321    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/53bb17de-fec9-4485-9850-f2d9a1c33d9e-kube-api-access-zfldt" (OuterVolumeSpecName: "kube-api-access-zfldt") pod "53bb17de-fec9-4485-9850-f2d9a1c33d9e" (UID: "53bb17de-fec9-4485-9850-f2d9a1c33d9e"). InnerVolumeSpecName "kube-api-access-zfldt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 15:46:11.872164 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:46:11.872069    2567 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/53bb17de-fec9-4485-9850-f2d9a1c33d9e-metrics-certs\") on node \"ip-10-0-128-241.ec2.internal\" DevicePath \"\""
Sep 10 15:46:11.872164 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:46:11.872104    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2r2cl\" (UniqueName: \"kubernetes.io/projected/8ad3cae7-ba50-4435-8ee1-7494b6899ee1-kube-api-access-2r2cl\") on node \"ip-10-0-128-241.ec2.internal\" DevicePath \"\""
Sep 10 15:46:11.872164 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:46:11.872119    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zfldt\" (UniqueName: \"kubernetes.io/projected/53bb17de-fec9-4485-9850-f2d9a1c33d9e-kube-api-access-zfldt\") on node \"ip-10-0-128-241.ec2.internal\" DevicePath \"\""
Sep 10 15:46:11.872164 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:46:11.872131    2567 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/53bb17de-fec9-4485-9850-f2d9a1c33d9e-maas-api-tls\") on node \"ip-10-0-128-241.ec2.internal\" DevicePath \"\""
Sep 10 15:46:11.872164 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:46:11.872143    2567 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8ad3cae7-ba50-4435-8ee1-7494b6899ee1-plugins-config-volume\") on node \"ip-10-0-128-241.ec2.internal\" DevicePath \"\""
Sep 10 15:46:12.050457 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:46:12.050426    2567 generic.go:356] "Generic (PLEG): container finished" podID="8ad3cae7-ba50-4435-8ee1-7494b6899ee1" containerID="fca38a64649665697f165114d6e23973e3e8a95b6e56fe10e66802c353fee1fe" exitCode=0
Sep 10 15:46:12.050663 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:46:12.050475    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-139155-7d477d9786-vhwg2"
Sep 10 15:46:12.050663 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:46:12.050515    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-139155-7d477d9786-vhwg2" event={"ID":"8ad3cae7-ba50-4435-8ee1-7494b6899ee1","Type":"ContainerDied","Data":"fca38a64649665697f165114d6e23973e3e8a95b6e56fe10e66802c353fee1fe"}
Sep 10 15:46:12.050663 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:46:12.050576    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-139155-7d477d9786-vhwg2" event={"ID":"8ad3cae7-ba50-4435-8ee1-7494b6899ee1","Type":"ContainerDied","Data":"7c830ef6a39478005a2e2a069697e5928b093afae4e4d462febfde85551b4ac5"}
Sep 10 15:46:12.050663 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:46:12.050611    2567 scope.go:117] "RemoveContainer" containerID="fca38a64649665697f165114d6e23973e3e8a95b6e56fe10e66802c353fee1fe"
Sep 10 15:46:12.051851 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:46:12.051830    2567 generic.go:356] "Generic (PLEG): container finished" podID="53bb17de-fec9-4485-9850-f2d9a1c33d9e" containerID="ae4f548bcbd1f79998a621d3f767055a44e7c19006169d656c242c99da895606" exitCode=0
Sep 10 15:46:12.051956 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:46:12.051928    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-139155-584bcf7687-nhzlf"
Sep 10 15:46:12.052064 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:46:12.051904    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-139155-584bcf7687-nhzlf" event={"ID":"53bb17de-fec9-4485-9850-f2d9a1c33d9e","Type":"ContainerDied","Data":"ae4f548bcbd1f79998a621d3f767055a44e7c19006169d656c242c99da895606"}
Sep 10 15:46:12.052064 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:46:12.052027    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-139155-584bcf7687-nhzlf" event={"ID":"53bb17de-fec9-4485-9850-f2d9a1c33d9e","Type":"ContainerDied","Data":"6a58eea1ae8306e1d4b59f722368517aa35f9261b0fb8836429cf39ad381e6aa"}
Sep 10 15:46:12.060441 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:46:12.060276    2567 scope.go:117] "RemoveContainer" containerID="fca38a64649665697f165114d6e23973e3e8a95b6e56fe10e66802c353fee1fe"
Sep 10 15:46:12.060741 ip-10-0-128-241 kubenswrapper[2567]: E0910 15:46:12.060708    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fca38a64649665697f165114d6e23973e3e8a95b6e56fe10e66802c353fee1fe\": container with ID starting with fca38a64649665697f165114d6e23973e3e8a95b6e56fe10e66802c353fee1fe not found: ID does not exist" containerID="fca38a64649665697f165114d6e23973e3e8a95b6e56fe10e66802c353fee1fe"
Sep 10 15:46:12.060841 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:46:12.060749    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fca38a64649665697f165114d6e23973e3e8a95b6e56fe10e66802c353fee1fe"} err="failed to get container status \"fca38a64649665697f165114d6e23973e3e8a95b6e56fe10e66802c353fee1fe\": rpc error: code = NotFound desc = could not find container \"fca38a64649665697f165114d6e23973e3e8a95b6e56fe10e66802c353fee1fe\": container with ID starting with fca38a64649665697f165114d6e23973e3e8a95b6e56fe10e66802c353fee1fe not found: ID does not exist"
Sep 10 15:46:12.060841 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:46:12.060768    2567 scope.go:117] "RemoveContainer" containerID="ae4f548bcbd1f79998a621d3f767055a44e7c19006169d656c242c99da895606"
Sep 10 15:46:12.069064 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:46:12.069036    2567 scope.go:117] "RemoveContainer" containerID="ae4f548bcbd1f79998a621d3f767055a44e7c19006169d656c242c99da895606"
Sep 10 15:46:12.069373 ip-10-0-128-241 kubenswrapper[2567]: E0910 15:46:12.069352    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ae4f548bcbd1f79998a621d3f767055a44e7c19006169d656c242c99da895606\": container with ID starting with ae4f548bcbd1f79998a621d3f767055a44e7c19006169d656c242c99da895606 not found: ID does not exist" containerID="ae4f548bcbd1f79998a621d3f767055a44e7c19006169d656c242c99da895606"
Sep 10 15:46:12.069467 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:46:12.069380    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ae4f548bcbd1f79998a621d3f767055a44e7c19006169d656c242c99da895606"} err="failed to get container status \"ae4f548bcbd1f79998a621d3f767055a44e7c19006169d656c242c99da895606\": rpc error: code = NotFound desc = could not find container \"ae4f548bcbd1f79998a621d3f767055a44e7c19006169d656c242c99da895606\": container with ID starting with ae4f548bcbd1f79998a621d3f767055a44e7c19006169d656c242c99da895606 not found: ID does not exist"
Sep 10 15:46:12.077520 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:46:12.077494    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-139155-7d477d9786-vhwg2"]
Sep 10 15:46:12.082316 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:46:12.082295    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-139155-7d477d9786-vhwg2"]
Sep 10 15:46:12.095995 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:46:12.095969    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-139155-584bcf7687-nhzlf"]
Sep 10 15:46:12.102934 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:46:12.102910    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-139155-584bcf7687-nhzlf"]
Sep 10 15:46:12.566379 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:46:12.566346    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:46:12.666018 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:46:12.665932    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:46:12.883365 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:46:12.883327    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:46:12.904072 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:46:12.904034    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="53bb17de-fec9-4485-9850-f2d9a1c33d9e" path="/var/lib/kubelet/pods/53bb17de-fec9-4485-9850-f2d9a1c33d9e/volumes"
Sep 10 15:46:12.904478 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:46:12.904460    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8ad3cae7-ba50-4435-8ee1-7494b6899ee1" path="/var/lib/kubelet/pods/8ad3cae7-ba50-4435-8ee1-7494b6899ee1/volumes"
Sep 10 15:46:13.856134 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:46:13.856095    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:46:14.041254 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:46:14.041194    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:46:14.139666 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:46:14.139579    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:46:14.607911 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:46:14.607858    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-139155-584bcf7687-nhzlf" podUID="53bb17de-fec9-4485-9850-f2d9a1c33d9e" containerName="maas-api" probeResult="failure" output="Get \"https://10.132.0.37:8443/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Sep 10 15:46:14.761191 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:46:14.761150    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:46:15.675853 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:46:15.675819    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:46:16.487026 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:46:16.486988    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:46:16.606405 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:46:16.606363    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-b-90ca97/test-model-90ca97-kserve-6597c9f47f-g6cdl" podUID="30ae33ce-692b-4493-bfc4-e7b45c56dfb2" containerName="main" containerID="cri-o://68087a091f4d633270242b5a35653e69eb749857e8cd573fa67068c3ccb1d41e" gracePeriod=45
Sep 10 15:46:16.851374 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:46:16.851347    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-90ca97/test-model-90ca97-kserve-6597c9f47f-g6cdl"
Sep 10 15:46:16.921491 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:46:16.921463    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gxj4z\" (UniqueName: \"kubernetes.io/projected/30ae33ce-692b-4493-bfc4-e7b45c56dfb2-kube-api-access-gxj4z\") pod \"30ae33ce-692b-4493-bfc4-e7b45c56dfb2\" (UID: \"30ae33ce-692b-4493-bfc4-e7b45c56dfb2\") "
Sep 10 15:46:16.921677 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:46:16.921504    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/30ae33ce-692b-4493-bfc4-e7b45c56dfb2-dshm\") pod \"30ae33ce-692b-4493-bfc4-e7b45c56dfb2\" (UID: \"30ae33ce-692b-4493-bfc4-e7b45c56dfb2\") "
Sep 10 15:46:16.921677 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:46:16.921537    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/30ae33ce-692b-4493-bfc4-e7b45c56dfb2-home\") pod \"30ae33ce-692b-4493-bfc4-e7b45c56dfb2\" (UID: \"30ae33ce-692b-4493-bfc4-e7b45c56dfb2\") "
Sep 10 15:46:16.921677 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:46:16.921584    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/30ae33ce-692b-4493-bfc4-e7b45c56dfb2-tls-certs\") pod \"30ae33ce-692b-4493-bfc4-e7b45c56dfb2\" (UID: \"30ae33ce-692b-4493-bfc4-e7b45c56dfb2\") "
Sep 10 15:46:16.921677 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:46:16.921604    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/30ae33ce-692b-4493-bfc4-e7b45c56dfb2-tmp-dir\") pod \"30ae33ce-692b-4493-bfc4-e7b45c56dfb2\" (UID: \"30ae33ce-692b-4493-bfc4-e7b45c56dfb2\") "
Sep 10 15:46:16.921677 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:46:16.921665    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/30ae33ce-692b-4493-bfc4-e7b45c56dfb2-model-cache\") pod \"30ae33ce-692b-4493-bfc4-e7b45c56dfb2\" (UID: \"30ae33ce-692b-4493-bfc4-e7b45c56dfb2\") "
Sep 10 15:46:16.922028 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:46:16.921773    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/30ae33ce-692b-4493-bfc4-e7b45c56dfb2-home" (OuterVolumeSpecName: "home") pod "30ae33ce-692b-4493-bfc4-e7b45c56dfb2" (UID: "30ae33ce-692b-4493-bfc4-e7b45c56dfb2"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 15:46:16.922028 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:46:16.921925    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/30ae33ce-692b-4493-bfc4-e7b45c56dfb2-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "30ae33ce-692b-4493-bfc4-e7b45c56dfb2" (UID: "30ae33ce-692b-4493-bfc4-e7b45c56dfb2"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 15:46:16.922028 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:46:16.921957    2567 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/30ae33ce-692b-4493-bfc4-e7b45c56dfb2-home\") on node \"ip-10-0-128-241.ec2.internal\" DevicePath \"\""
Sep 10 15:46:16.922028 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:46:16.921959    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/30ae33ce-692b-4493-bfc4-e7b45c56dfb2-model-cache" (OuterVolumeSpecName: "model-cache") pod "30ae33ce-692b-4493-bfc4-e7b45c56dfb2" (UID: "30ae33ce-692b-4493-bfc4-e7b45c56dfb2"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 15:46:16.923816 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:46:16.923793    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/30ae33ce-692b-4493-bfc4-e7b45c56dfb2-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "30ae33ce-692b-4493-bfc4-e7b45c56dfb2" (UID: "30ae33ce-692b-4493-bfc4-e7b45c56dfb2"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 15:46:16.923918 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:46:16.923832    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/30ae33ce-692b-4493-bfc4-e7b45c56dfb2-dshm" (OuterVolumeSpecName: "dshm") pod "30ae33ce-692b-4493-bfc4-e7b45c56dfb2" (UID: "30ae33ce-692b-4493-bfc4-e7b45c56dfb2"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 15:46:16.923918 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:46:16.923848    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/30ae33ce-692b-4493-bfc4-e7b45c56dfb2-kube-api-access-gxj4z" (OuterVolumeSpecName: "kube-api-access-gxj4z") pod "30ae33ce-692b-4493-bfc4-e7b45c56dfb2" (UID: "30ae33ce-692b-4493-bfc4-e7b45c56dfb2"). InnerVolumeSpecName "kube-api-access-gxj4z". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 15:46:17.022503 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:46:17.022416    2567 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/30ae33ce-692b-4493-bfc4-e7b45c56dfb2-tls-certs\") on node \"ip-10-0-128-241.ec2.internal\" DevicePath \"\""
Sep 10 15:46:17.022503 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:46:17.022449    2567 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/30ae33ce-692b-4493-bfc4-e7b45c56dfb2-tmp-dir\") on node \"ip-10-0-128-241.ec2.internal\" DevicePath \"\""
Sep 10 15:46:17.022503 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:46:17.022462    2567 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/30ae33ce-692b-4493-bfc4-e7b45c56dfb2-model-cache\") on node \"ip-10-0-128-241.ec2.internal\" DevicePath \"\""
Sep 10 15:46:17.022503 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:46:17.022476    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gxj4z\" (UniqueName: \"kubernetes.io/projected/30ae33ce-692b-4493-bfc4-e7b45c56dfb2-kube-api-access-gxj4z\") on node \"ip-10-0-128-241.ec2.internal\" DevicePath \"\""
Sep 10 15:46:17.022503 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:46:17.022487    2567 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/30ae33ce-692b-4493-bfc4-e7b45c56dfb2-dshm\") on node \"ip-10-0-128-241.ec2.internal\" DevicePath \"\""
Sep 10 15:46:17.086043 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:46:17.085955    2567 generic.go:356] "Generic (PLEG): container finished" podID="30ae33ce-692b-4493-bfc4-e7b45c56dfb2" containerID="68087a091f4d633270242b5a35653e69eb749857e8cd573fa67068c3ccb1d41e" exitCode=0
Sep 10 15:46:17.086043 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:46:17.086031    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-90ca97/test-model-90ca97-kserve-6597c9f47f-g6cdl"
Sep 10 15:46:17.086272 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:46:17.086039    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-90ca97/test-model-90ca97-kserve-6597c9f47f-g6cdl" event={"ID":"30ae33ce-692b-4493-bfc4-e7b45c56dfb2","Type":"ContainerDied","Data":"68087a091f4d633270242b5a35653e69eb749857e8cd573fa67068c3ccb1d41e"}
Sep 10 15:46:17.086272 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:46:17.086085    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-90ca97/test-model-90ca97-kserve-6597c9f47f-g6cdl" event={"ID":"30ae33ce-692b-4493-bfc4-e7b45c56dfb2","Type":"ContainerDied","Data":"10665ac9584538ba75e3f1407613ad768f0d66b036c781eb96944c83b0fabd27"}
Sep 10 15:46:17.086272 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:46:17.086102    2567 scope.go:117] "RemoveContainer" containerID="68087a091f4d633270242b5a35653e69eb749857e8cd573fa67068c3ccb1d41e"
Sep 10 15:46:17.095007 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:46:17.094985    2567 scope.go:117] "RemoveContainer" containerID="68087a091f4d633270242b5a35653e69eb749857e8cd573fa67068c3ccb1d41e"
Sep 10 15:46:17.095348 ip-10-0-128-241 kubenswrapper[2567]: E0910 15:46:17.095322    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"68087a091f4d633270242b5a35653e69eb749857e8cd573fa67068c3ccb1d41e\": container with ID starting with 68087a091f4d633270242b5a35653e69eb749857e8cd573fa67068c3ccb1d41e not found: ID does not exist" containerID="68087a091f4d633270242b5a35653e69eb749857e8cd573fa67068c3ccb1d41e"
Sep 10 15:46:17.095421 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:46:17.095357    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"68087a091f4d633270242b5a35653e69eb749857e8cd573fa67068c3ccb1d41e"} err="failed to get container status \"68087a091f4d633270242b5a35653e69eb749857e8cd573fa67068c3ccb1d41e\": rpc error: code = NotFound desc = could not find container \"68087a091f4d633270242b5a35653e69eb749857e8cd573fa67068c3ccb1d41e\": container with ID starting with 68087a091f4d633270242b5a35653e69eb749857e8cd573fa67068c3ccb1d41e not found: ID does not exist"
Sep 10 15:46:17.112930 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:46:17.112902    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-90ca97/test-model-90ca97-kserve-6597c9f47f-g6cdl"]
Sep 10 15:46:17.120513 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:46:17.120491    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-b-90ca97/test-model-90ca97-kserve-6597c9f47f-g6cdl"]
Sep 10 15:46:18.452611 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:46:18.452575    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:46:18.512520 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:46:18.512482    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:46:18.706187 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:46:18.706109    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:46:18.904428 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:46:18.904393    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="30ae33ce-692b-4493-bfc4-e7b45c56dfb2" path="/var/lib/kubelet/pods/30ae33ce-692b-4493-bfc4-e7b45c56dfb2/volumes"
Sep 10 15:46:19.837217 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:46:19.837182    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:46:20.654707 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:46:20.654672    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:46:21.380818 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:46:21.380778    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:46:22.709218 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:46:22.709186    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:46:23.356766 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:46:23.356730    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-f4a9663c-674bc67747-8vltc"]
Sep 10 15:46:23.356961 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:46:23.356939    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-f4a9663c-674bc67747-8vltc" podUID="74336f8d-ef31-4383-970f-cf65586ed643" containerName="maas-api" containerID="cri-o://e6a264dd521bf12b87af9d62bf7c525d7d4988fe9d08e6789b9ccbee9effd7d0" gracePeriod=30
Sep 10 15:46:23.374903 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:46:23.374861    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:46:23.442007 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:46:23.441923    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-f4a9663c-797b686986-mfgsw"]
Sep 10 15:46:23.442210 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:46:23.442169    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-mt-f4a9663c-797b686986-mfgsw" podUID="53520aaf-fc71-4178-8b2e-1c244f904eb8" containerName="payload-pre-processing" containerID="cri-o://f97d49caa7126fff04e74b7fc2d6fdc9ad8eb3349ee8270d348f08a55b8644a4" gracePeriod=30
Sep 10 15:46:23.509119 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:46:23.509086    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:46:23.602944 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:46:23.602907    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:46:23.690891 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:46:23.690862    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-f4a9663c-674bc67747-8vltc"
Sep 10 15:46:23.707451 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:46:23.707414    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:46:23.722219 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:46:23.722195    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-f4a9663c-797b686986-mfgsw"
Sep 10 15:46:23.789658 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:46:23.789625    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zlwp2\" (UniqueName: \"kubernetes.io/projected/74336f8d-ef31-4383-970f-cf65586ed643-kube-api-access-zlwp2\") pod \"74336f8d-ef31-4383-970f-cf65586ed643\" (UID: \"74336f8d-ef31-4383-970f-cf65586ed643\") "
Sep 10 15:46:23.789824 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:46:23.789666    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/74336f8d-ef31-4383-970f-cf65586ed643-maas-api-tls\") pod \"74336f8d-ef31-4383-970f-cf65586ed643\" (UID: \"74336f8d-ef31-4383-970f-cf65586ed643\") "
Sep 10 15:46:23.789824 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:46:23.789688    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/74336f8d-ef31-4383-970f-cf65586ed643-metrics-certs\") pod \"74336f8d-ef31-4383-970f-cf65586ed643\" (UID: \"74336f8d-ef31-4383-970f-cf65586ed643\") "
Sep 10 15:46:23.789824 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:46:23.789757    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/74336f8d-ef31-4383-970f-cf65586ed643-openshift-service-ca\") pod \"74336f8d-ef31-4383-970f-cf65586ed643\" (UID: \"74336f8d-ef31-4383-970f-cf65586ed643\") "
Sep 10 15:46:23.790120 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:46:23.790090    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/74336f8d-ef31-4383-970f-cf65586ed643-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "74336f8d-ef31-4383-970f-cf65586ed643" (UID: "74336f8d-ef31-4383-970f-cf65586ed643"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 15:46:23.791817 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:46:23.791793    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/74336f8d-ef31-4383-970f-cf65586ed643-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "74336f8d-ef31-4383-970f-cf65586ed643" (UID: "74336f8d-ef31-4383-970f-cf65586ed643"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 15:46:23.791888 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:46:23.791807    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/74336f8d-ef31-4383-970f-cf65586ed643-kube-api-access-zlwp2" (OuterVolumeSpecName: "kube-api-access-zlwp2") pod "74336f8d-ef31-4383-970f-cf65586ed643" (UID: "74336f8d-ef31-4383-970f-cf65586ed643"). InnerVolumeSpecName "kube-api-access-zlwp2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 15:46:23.791888 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:46:23.791812    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/74336f8d-ef31-4383-970f-cf65586ed643-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "74336f8d-ef31-4383-970f-cf65586ed643" (UID: "74336f8d-ef31-4383-970f-cf65586ed643"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 15:46:23.891118 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:46:23.891035    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ts8pl\" (UniqueName: \"kubernetes.io/projected/53520aaf-fc71-4178-8b2e-1c244f904eb8-kube-api-access-ts8pl\") pod \"53520aaf-fc71-4178-8b2e-1c244f904eb8\" (UID: \"53520aaf-fc71-4178-8b2e-1c244f904eb8\") "
Sep 10 15:46:23.891302 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:46:23.891137    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/53520aaf-fc71-4178-8b2e-1c244f904eb8-plugins-config-volume\") pod \"53520aaf-fc71-4178-8b2e-1c244f904eb8\" (UID: \"53520aaf-fc71-4178-8b2e-1c244f904eb8\") "
Sep 10 15:46:23.891437 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:46:23.891417    2567 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/74336f8d-ef31-4383-970f-cf65586ed643-openshift-service-ca\") on node \"ip-10-0-128-241.ec2.internal\" DevicePath \"\""
Sep 10 15:46:23.891476 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:46:23.891443    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zlwp2\" (UniqueName: \"kubernetes.io/projected/74336f8d-ef31-4383-970f-cf65586ed643-kube-api-access-zlwp2\") on node \"ip-10-0-128-241.ec2.internal\" DevicePath \"\""
Sep 10 15:46:23.891476 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:46:23.891459    2567 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/74336f8d-ef31-4383-970f-cf65586ed643-maas-api-tls\") on node \"ip-10-0-128-241.ec2.internal\" DevicePath \"\""
Sep 10 15:46:23.891476 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:46:23.891471    2567 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/74336f8d-ef31-4383-970f-cf65586ed643-metrics-certs\") on node \"ip-10-0-128-241.ec2.internal\" DevicePath \"\""
Sep 10 15:46:23.891572 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:46:23.891519    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/53520aaf-fc71-4178-8b2e-1c244f904eb8-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "53520aaf-fc71-4178-8b2e-1c244f904eb8" (UID: "53520aaf-fc71-4178-8b2e-1c244f904eb8"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 15:46:23.893277 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:46:23.893259    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/53520aaf-fc71-4178-8b2e-1c244f904eb8-kube-api-access-ts8pl" (OuterVolumeSpecName: "kube-api-access-ts8pl") pod "53520aaf-fc71-4178-8b2e-1c244f904eb8" (UID: "53520aaf-fc71-4178-8b2e-1c244f904eb8"). InnerVolumeSpecName "kube-api-access-ts8pl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 15:46:23.979204 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:46:23.979175    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:46:23.992484 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:46:23.992451    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ts8pl\" (UniqueName: \"kubernetes.io/projected/53520aaf-fc71-4178-8b2e-1c244f904eb8-kube-api-access-ts8pl\") on node \"ip-10-0-128-241.ec2.internal\" DevicePath \"\""
Sep 10 15:46:23.992484 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:46:23.992476    2567 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/53520aaf-fc71-4178-8b2e-1c244f904eb8-plugins-config-volume\") on node \"ip-10-0-128-241.ec2.internal\" DevicePath \"\""
Sep 10 15:46:24.076260 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:46:24.076210    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:46:24.130914 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:46:24.130881    2567 generic.go:356] "Generic (PLEG): container finished" podID="53520aaf-fc71-4178-8b2e-1c244f904eb8" containerID="f97d49caa7126fff04e74b7fc2d6fdc9ad8eb3349ee8270d348f08a55b8644a4" exitCode=0
Sep 10 15:46:24.131111 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:46:24.130922    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-f4a9663c-797b686986-mfgsw"
Sep 10 15:46:24.131111 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:46:24.130964    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-f4a9663c-797b686986-mfgsw" event={"ID":"53520aaf-fc71-4178-8b2e-1c244f904eb8","Type":"ContainerDied","Data":"f97d49caa7126fff04e74b7fc2d6fdc9ad8eb3349ee8270d348f08a55b8644a4"}
Sep 10 15:46:24.131111 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:46:24.131012    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-f4a9663c-797b686986-mfgsw" event={"ID":"53520aaf-fc71-4178-8b2e-1c244f904eb8","Type":"ContainerDied","Data":"458687f665918c3212a630ed06e3468ffd9e662a7419b297ace9e9f87fe5997f"}
Sep 10 15:46:24.131111 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:46:24.131036    2567 scope.go:117] "RemoveContainer" containerID="f97d49caa7126fff04e74b7fc2d6fdc9ad8eb3349ee8270d348f08a55b8644a4"
Sep 10 15:46:24.132347 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:46:24.132326    2567 generic.go:356] "Generic (PLEG): container finished" podID="74336f8d-ef31-4383-970f-cf65586ed643" containerID="e6a264dd521bf12b87af9d62bf7c525d7d4988fe9d08e6789b9ccbee9effd7d0" exitCode=0
Sep 10 15:46:24.132431 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:46:24.132380    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-f4a9663c-674bc67747-8vltc"
Sep 10 15:46:24.132483 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:46:24.132376    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-f4a9663c-674bc67747-8vltc" event={"ID":"74336f8d-ef31-4383-970f-cf65586ed643","Type":"ContainerDied","Data":"e6a264dd521bf12b87af9d62bf7c525d7d4988fe9d08e6789b9ccbee9effd7d0"}
Sep 10 15:46:24.132518 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:46:24.132500    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-f4a9663c-674bc67747-8vltc" event={"ID":"74336f8d-ef31-4383-970f-cf65586ed643","Type":"ContainerDied","Data":"0f51179e3001824a4b933554f25c6ff9bd4d7f2e4d962efe842454ea6e98d422"}
Sep 10 15:46:24.141457 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:46:24.141219    2567 scope.go:117] "RemoveContainer" containerID="f97d49caa7126fff04e74b7fc2d6fdc9ad8eb3349ee8270d348f08a55b8644a4"
Sep 10 15:46:24.141725 ip-10-0-128-241 kubenswrapper[2567]: E0910 15:46:24.141702    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f97d49caa7126fff04e74b7fc2d6fdc9ad8eb3349ee8270d348f08a55b8644a4\": container with ID starting with f97d49caa7126fff04e74b7fc2d6fdc9ad8eb3349ee8270d348f08a55b8644a4 not found: ID does not exist" containerID="f97d49caa7126fff04e74b7fc2d6fdc9ad8eb3349ee8270d348f08a55b8644a4"
Sep 10 15:46:24.141792 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:46:24.141733    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f97d49caa7126fff04e74b7fc2d6fdc9ad8eb3349ee8270d348f08a55b8644a4"} err="failed to get container status \"f97d49caa7126fff04e74b7fc2d6fdc9ad8eb3349ee8270d348f08a55b8644a4\": rpc error: code = NotFound desc = could not find container \"f97d49caa7126fff04e74b7fc2d6fdc9ad8eb3349ee8270d348f08a55b8644a4\": container with ID starting with f97d49caa7126fff04e74b7fc2d6fdc9ad8eb3349ee8270d348f08a55b8644a4 not found: ID does not exist"
Sep 10 15:46:24.141792 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:46:24.141748    2567 scope.go:117] "RemoveContainer" containerID="e6a264dd521bf12b87af9d62bf7c525d7d4988fe9d08e6789b9ccbee9effd7d0"
Sep 10 15:46:24.149882 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:46:24.149865    2567 scope.go:117] "RemoveContainer" containerID="e6a264dd521bf12b87af9d62bf7c525d7d4988fe9d08e6789b9ccbee9effd7d0"
Sep 10 15:46:24.150125 ip-10-0-128-241 kubenswrapper[2567]: E0910 15:46:24.150111    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e6a264dd521bf12b87af9d62bf7c525d7d4988fe9d08e6789b9ccbee9effd7d0\": container with ID starting with e6a264dd521bf12b87af9d62bf7c525d7d4988fe9d08e6789b9ccbee9effd7d0 not found: ID does not exist" containerID="e6a264dd521bf12b87af9d62bf7c525d7d4988fe9d08e6789b9ccbee9effd7d0"
Sep 10 15:46:24.150167 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:46:24.150132    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e6a264dd521bf12b87af9d62bf7c525d7d4988fe9d08e6789b9ccbee9effd7d0"} err="failed to get container status \"e6a264dd521bf12b87af9d62bf7c525d7d4988fe9d08e6789b9ccbee9effd7d0\": rpc error: code = NotFound desc = could not find container \"e6a264dd521bf12b87af9d62bf7c525d7d4988fe9d08e6789b9ccbee9effd7d0\": container with ID starting with e6a264dd521bf12b87af9d62bf7c525d7d4988fe9d08e6789b9ccbee9effd7d0 not found: ID does not exist"
Sep 10 15:46:24.158352 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:46:24.158331    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-f4a9663c-674bc67747-8vltc"]
Sep 10 15:46:24.162819 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:46:24.162798    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-f4a9663c-674bc67747-8vltc"]
Sep 10 15:46:24.179142 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:46:24.179110    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-f4a9663c-797b686986-mfgsw"]
Sep 10 15:46:24.184953 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:46:24.184930    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-f4a9663c-797b686986-mfgsw"]
Sep 10 15:46:24.895695 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:46:24.895660    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:46:24.904382 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:46:24.904355    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="53520aaf-fc71-4178-8b2e-1c244f904eb8" path="/var/lib/kubelet/pods/53520aaf-fc71-4178-8b2e-1c244f904eb8/volumes"
Sep 10 15:46:24.904765 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:46:24.904751    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="74336f8d-ef31-4383-970f-cf65586ed643" path="/var/lib/kubelet/pods/74336f8d-ef31-4383-970f-cf65586ed643/volumes"
Sep 10 15:46:29.774593 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:46:29.774560    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:46:29.798809 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:46:29.798776    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-90ca97-846c64b6fd-frz4x"]
Sep 10 15:46:29.799064 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:46:29.799040    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-90ca97-846c64b6fd-frz4x" podUID="0931ea56-b8ce-43cc-861c-07b7c1a51893" containerName="payload-pre-processing" containerID="cri-o://d7c6e741dfcf675d9eca93b0cb47ddd4262c084945f1dde30980906e28a85f72" gracePeriod=30
Sep 10 15:46:29.844955 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:46:29.844879    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:46:29.946260 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:46:29.946210    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:46:30.014829 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:46:30.014624    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:46:30.064184 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:46:30.064155    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-90ca97-846c64b6fd-frz4x"
Sep 10 15:46:30.163602 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:46:30.163556    2567 generic.go:356] "Generic (PLEG): container finished" podID="0931ea56-b8ce-43cc-861c-07b7c1a51893" containerID="d7c6e741dfcf675d9eca93b0cb47ddd4262c084945f1dde30980906e28a85f72" exitCode=0
Sep 10 15:46:30.163602 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:46:30.163600    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-90ca97-846c64b6fd-frz4x"
Sep 10 15:46:30.163872 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:46:30.163636    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-90ca97-846c64b6fd-frz4x" event={"ID":"0931ea56-b8ce-43cc-861c-07b7c1a51893","Type":"ContainerDied","Data":"d7c6e741dfcf675d9eca93b0cb47ddd4262c084945f1dde30980906e28a85f72"}
Sep 10 15:46:30.163872 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:46:30.163711    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-90ca97-846c64b6fd-frz4x" event={"ID":"0931ea56-b8ce-43cc-861c-07b7c1a51893","Type":"ContainerDied","Data":"15b50f426aec34e7f7417cfb67eb4c5a4360684bc336a06565c49c2abcf1e1f4"}
Sep 10 15:46:30.163872 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:46:30.163733    2567 scope.go:117] "RemoveContainer" containerID="d7c6e741dfcf675d9eca93b0cb47ddd4262c084945f1dde30980906e28a85f72"
Sep 10 15:46:30.173561 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:46:30.173538    2567 scope.go:117] "RemoveContainer" containerID="d7c6e741dfcf675d9eca93b0cb47ddd4262c084945f1dde30980906e28a85f72"
Sep 10 15:46:30.173877 ip-10-0-128-241 kubenswrapper[2567]: E0910 15:46:30.173858    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d7c6e741dfcf675d9eca93b0cb47ddd4262c084945f1dde30980906e28a85f72\": container with ID starting with d7c6e741dfcf675d9eca93b0cb47ddd4262c084945f1dde30980906e28a85f72 not found: ID does not exist" containerID="d7c6e741dfcf675d9eca93b0cb47ddd4262c084945f1dde30980906e28a85f72"
Sep 10 15:46:30.173956 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:46:30.173886    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d7c6e741dfcf675d9eca93b0cb47ddd4262c084945f1dde30980906e28a85f72"} err="failed to get container status \"d7c6e741dfcf675d9eca93b0cb47ddd4262c084945f1dde30980906e28a85f72\": rpc error: code = NotFound desc = could not find container \"d7c6e741dfcf675d9eca93b0cb47ddd4262c084945f1dde30980906e28a85f72\": container with ID starting with d7c6e741dfcf675d9eca93b0cb47ddd4262c084945f1dde30980906e28a85f72 not found: ID does not exist"
Sep 10 15:46:30.252809 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:46:30.252762    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-phck5\" (UniqueName: \"kubernetes.io/projected/0931ea56-b8ce-43cc-861c-07b7c1a51893-kube-api-access-phck5\") pod \"0931ea56-b8ce-43cc-861c-07b7c1a51893\" (UID: \"0931ea56-b8ce-43cc-861c-07b7c1a51893\") "
Sep 10 15:46:30.253007 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:46:30.252821    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0931ea56-b8ce-43cc-861c-07b7c1a51893-plugins-config-volume\") pod \"0931ea56-b8ce-43cc-861c-07b7c1a51893\" (UID: \"0931ea56-b8ce-43cc-861c-07b7c1a51893\") "
Sep 10 15:46:30.253188 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:46:30.253163    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0931ea56-b8ce-43cc-861c-07b7c1a51893-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "0931ea56-b8ce-43cc-861c-07b7c1a51893" (UID: "0931ea56-b8ce-43cc-861c-07b7c1a51893"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 15:46:30.254817 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:46:30.254787    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0931ea56-b8ce-43cc-861c-07b7c1a51893-kube-api-access-phck5" (OuterVolumeSpecName: "kube-api-access-phck5") pod "0931ea56-b8ce-43cc-861c-07b7c1a51893" (UID: "0931ea56-b8ce-43cc-861c-07b7c1a51893"). InnerVolumeSpecName "kube-api-access-phck5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 15:46:30.339890 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:46:30.339803    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:46:30.353589 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:46:30.353557    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-phck5\" (UniqueName: \"kubernetes.io/projected/0931ea56-b8ce-43cc-861c-07b7c1a51893-kube-api-access-phck5\") on node \"ip-10-0-128-241.ec2.internal\" DevicePath \"\""
Sep 10 15:46:30.353589 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:46:30.353584    2567 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0931ea56-b8ce-43cc-861c-07b7c1a51893-plugins-config-volume\") on node \"ip-10-0-128-241.ec2.internal\" DevicePath \"\""
Sep 10 15:46:30.440486 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:46:30.440457    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:46:30.486106 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:46:30.486074    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-90ca97-846c64b6fd-frz4x"]
Sep 10 15:46:30.493034 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:46:30.493009    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-90ca97-846c64b6fd-frz4x"]
Sep 10 15:46:30.904014 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:46:30.903968    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0931ea56-b8ce-43cc-861c-07b7c1a51893" path="/var/lib/kubelet/pods/0931ea56-b8ce-43cc-861c-07b7c1a51893/volumes"
Sep 10 15:46:30.951158 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:46:30.951125    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:46:31.109814 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:46:31.109785    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:46:31.244542 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:46:31.244508    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:46:31.573917 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:46:31.573835    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:46:31.672867 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:46:31.672831    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:46:31.787584 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:46:31.787550    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:46:37.201626 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:46:37.201564    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:46:37.314081 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:46:37.314041    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:46:37.443207 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:46:37.443176    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:46:37.743733 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:46:37.743696    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:46:38.085827 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:46:38.085738    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:46:38.939120 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:46:38.939084    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:46:39.045470 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:46:39.045441    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:46:39.138712 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:46:39.138679    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:46:39.281933 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:46:39.281848    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:46:41.025780 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:46:41.025739    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:46:41.575124 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:46:41.575093    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:46:42.170943 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:46:42.170910    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-c7257b-7bcfbc88b8-lkjp7"]
Sep 10 15:46:42.172020 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:46:42.171502    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8ad3cae7-ba50-4435-8ee1-7494b6899ee1" containerName="payload-pre-processing"
Sep 10 15:46:42.172020 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:46:42.171529    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="8ad3cae7-ba50-4435-8ee1-7494b6899ee1" containerName="payload-pre-processing"
Sep 10 15:46:42.172020 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:46:42.171566    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="30ae33ce-692b-4493-bfc4-e7b45c56dfb2" containerName="main"
Sep 10 15:46:42.172020 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:46:42.171575    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="30ae33ce-692b-4493-bfc4-e7b45c56dfb2" containerName="main"
Sep 10 15:46:42.172020 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:46:42.171595    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="74336f8d-ef31-4383-970f-cf65586ed643" containerName="maas-api"
Sep 10 15:46:42.172020 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:46:42.171604    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="74336f8d-ef31-4383-970f-cf65586ed643" containerName="maas-api"
Sep 10 15:46:42.172020 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:46:42.171615    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="53520aaf-fc71-4178-8b2e-1c244f904eb8" containerName="payload-pre-processing"
Sep 10 15:46:42.172020 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:46:42.171622    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="53520aaf-fc71-4178-8b2e-1c244f904eb8" containerName="payload-pre-processing"
Sep 10 15:46:42.172020 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:46:42.171635    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="53bb17de-fec9-4485-9850-f2d9a1c33d9e" containerName="maas-api"
Sep 10 15:46:42.172020 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:46:42.171643    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="53bb17de-fec9-4485-9850-f2d9a1c33d9e" containerName="maas-api"
Sep 10 15:46:42.172020 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:46:42.171656    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0931ea56-b8ce-43cc-861c-07b7c1a51893" containerName="payload-pre-processing"
Sep 10 15:46:42.172020 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:46:42.171664    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="0931ea56-b8ce-43cc-861c-07b7c1a51893" containerName="payload-pre-processing"
Sep 10 15:46:42.172020 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:46:42.171761    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="30ae33ce-692b-4493-bfc4-e7b45c56dfb2" containerName="main"
Sep 10 15:46:42.172020 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:46:42.171775    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="8ad3cae7-ba50-4435-8ee1-7494b6899ee1" containerName="payload-pre-processing"
Sep 10 15:46:42.172020 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:46:42.171787    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="53bb17de-fec9-4485-9850-f2d9a1c33d9e" containerName="maas-api"
Sep 10 15:46:42.172020 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:46:42.171796    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="74336f8d-ef31-4383-970f-cf65586ed643" containerName="maas-api"
Sep 10 15:46:42.172020 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:46:42.171806    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="53520aaf-fc71-4178-8b2e-1c244f904eb8" containerName="payload-pre-processing"
Sep 10 15:46:42.172020 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:46:42.171815    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="0931ea56-b8ce-43cc-861c-07b7c1a51893" containerName="payload-pre-processing"
Sep 10 15:46:42.175000 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:46:42.174977    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-c7257b-7bcfbc88b8-lkjp7"
Sep 10 15:46:42.177775 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:46:42.177753    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-w3-c7257b-serving-cert\""
Sep 10 15:46:42.186205 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:46:42.186183    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-c7257b-7bcfbc88b8-lkjp7"]
Sep 10 15:46:42.207475 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:46:42.207444    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:46:42.253210 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:46:42.253170    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ee2c75eb-0049-4111-94f7-39cc5e7896d1-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w3-c7257b-7bcfbc88b8-lkjp7\" (UID: \"ee2c75eb-0049-4111-94f7-39cc5e7896d1\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-c7257b-7bcfbc88b8-lkjp7"
Sep 10 15:46:42.253388 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:46:42.253256    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vh8tf\" (UniqueName: \"kubernetes.io/projected/ee2c75eb-0049-4111-94f7-39cc5e7896d1-kube-api-access-vh8tf\") pod \"maas-api-e2e-shared-a-w3-c7257b-7bcfbc88b8-lkjp7\" (UID: \"ee2c75eb-0049-4111-94f7-39cc5e7896d1\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-c7257b-7bcfbc88b8-lkjp7"
Sep 10 15:46:42.253388 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:46:42.253369    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ee2c75eb-0049-4111-94f7-39cc5e7896d1-metrics-certs\") pod \"maas-api-e2e-shared-a-w3-c7257b-7bcfbc88b8-lkjp7\" (UID: \"ee2c75eb-0049-4111-94f7-39cc5e7896d1\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-c7257b-7bcfbc88b8-lkjp7"
Sep 10 15:46:42.253502 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:46:42.253414    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ee2c75eb-0049-4111-94f7-39cc5e7896d1-maas-api-tls\") pod \"maas-api-e2e-shared-a-w3-c7257b-7bcfbc88b8-lkjp7\" (UID: \"ee2c75eb-0049-4111-94f7-39cc5e7896d1\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-c7257b-7bcfbc88b8-lkjp7"
Sep 10 15:46:42.274746 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:46:42.274689    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:46:42.354409 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:46:42.354367    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ee2c75eb-0049-4111-94f7-39cc5e7896d1-metrics-certs\") pod \"maas-api-e2e-shared-a-w3-c7257b-7bcfbc88b8-lkjp7\" (UID: \"ee2c75eb-0049-4111-94f7-39cc5e7896d1\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-c7257b-7bcfbc88b8-lkjp7"
Sep 10 15:46:42.354581 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:46:42.354415    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ee2c75eb-0049-4111-94f7-39cc5e7896d1-maas-api-tls\") pod \"maas-api-e2e-shared-a-w3-c7257b-7bcfbc88b8-lkjp7\" (UID: \"ee2c75eb-0049-4111-94f7-39cc5e7896d1\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-c7257b-7bcfbc88b8-lkjp7"
Sep 10 15:46:42.354581 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:46:42.354469    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ee2c75eb-0049-4111-94f7-39cc5e7896d1-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w3-c7257b-7bcfbc88b8-lkjp7\" (UID: \"ee2c75eb-0049-4111-94f7-39cc5e7896d1\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-c7257b-7bcfbc88b8-lkjp7"
Sep 10 15:46:42.354581 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:46:42.354529    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vh8tf\" (UniqueName: \"kubernetes.io/projected/ee2c75eb-0049-4111-94f7-39cc5e7896d1-kube-api-access-vh8tf\") pod \"maas-api-e2e-shared-a-w3-c7257b-7bcfbc88b8-lkjp7\" (UID: \"ee2c75eb-0049-4111-94f7-39cc5e7896d1\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-c7257b-7bcfbc88b8-lkjp7"
Sep 10 15:46:42.354581 ip-10-0-128-241 kubenswrapper[2567]: E0910 15:46:42.354553    2567 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-c7257b-serving-cert: secret "maas-api-e2e-shared-a-w3-c7257b-serving-cert" not found
Sep 10 15:46:42.354815 ip-10-0-128-241 kubenswrapper[2567]: E0910 15:46:42.354633    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ee2c75eb-0049-4111-94f7-39cc5e7896d1-maas-api-tls podName:ee2c75eb-0049-4111-94f7-39cc5e7896d1 nodeName:}" failed. No retries permitted until 2026-09-10 15:46:42.854611507 +0000 UTC m=+1188.422843234 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/ee2c75eb-0049-4111-94f7-39cc5e7896d1-maas-api-tls") pod "maas-api-e2e-shared-a-w3-c7257b-7bcfbc88b8-lkjp7" (UID: "ee2c75eb-0049-4111-94f7-39cc5e7896d1") : secret "maas-api-e2e-shared-a-w3-c7257b-serving-cert" not found
Sep 10 15:46:42.355192 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:46:42.355171    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ee2c75eb-0049-4111-94f7-39cc5e7896d1-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w3-c7257b-7bcfbc88b8-lkjp7\" (UID: \"ee2c75eb-0049-4111-94f7-39cc5e7896d1\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-c7257b-7bcfbc88b8-lkjp7"
Sep 10 15:46:42.356928 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:46:42.356905    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ee2c75eb-0049-4111-94f7-39cc5e7896d1-metrics-certs\") pod \"maas-api-e2e-shared-a-w3-c7257b-7bcfbc88b8-lkjp7\" (UID: \"ee2c75eb-0049-4111-94f7-39cc5e7896d1\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-c7257b-7bcfbc88b8-lkjp7"
Sep 10 15:46:42.363360 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:46:42.363331    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vh8tf\" (UniqueName: \"kubernetes.io/projected/ee2c75eb-0049-4111-94f7-39cc5e7896d1-kube-api-access-vh8tf\") pod \"maas-api-e2e-shared-a-w3-c7257b-7bcfbc88b8-lkjp7\" (UID: \"ee2c75eb-0049-4111-94f7-39cc5e7896d1\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-c7257b-7bcfbc88b8-lkjp7"
Sep 10 15:46:42.858783 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:46:42.858738    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ee2c75eb-0049-4111-94f7-39cc5e7896d1-maas-api-tls\") pod \"maas-api-e2e-shared-a-w3-c7257b-7bcfbc88b8-lkjp7\" (UID: \"ee2c75eb-0049-4111-94f7-39cc5e7896d1\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-c7257b-7bcfbc88b8-lkjp7"
Sep 10 15:46:42.861143 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:46:42.861120    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ee2c75eb-0049-4111-94f7-39cc5e7896d1-maas-api-tls\") pod \"maas-api-e2e-shared-a-w3-c7257b-7bcfbc88b8-lkjp7\" (UID: \"ee2c75eb-0049-4111-94f7-39cc5e7896d1\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-c7257b-7bcfbc88b8-lkjp7"
Sep 10 15:46:42.929065 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:46:42.929017    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-c7257b-5569d4b646-7pn87"]
Sep 10 15:46:42.932700 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:46:42.932677    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-c7257b-5569d4b646-7pn87"
Sep 10 15:46:42.934974 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:46:42.934947    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-w3-c7257b\""
Sep 10 15:46:42.935090 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:46:42.935024    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-w3-c7257b-dockercfg-f8fv8\""
Sep 10 15:46:42.943516 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:46:42.943493    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-c7257b-5569d4b646-7pn87"]
Sep 10 15:46:42.959989 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:46:42.959961    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9deb994d-b35c-4651-bb7b-aba81f779366-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w3-c7257b-5569d4b646-7pn87\" (UID: \"9deb994d-b35c-4651-bb7b-aba81f779366\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-c7257b-5569d4b646-7pn87"
Sep 10 15:46:42.960153 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:46:42.960111    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-865hl\" (UniqueName: \"kubernetes.io/projected/9deb994d-b35c-4651-bb7b-aba81f779366-kube-api-access-865hl\") pod \"payload-pre-processing-e2e-shared-a-w3-c7257b-5569d4b646-7pn87\" (UID: \"9deb994d-b35c-4651-bb7b-aba81f779366\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-c7257b-5569d4b646-7pn87"
Sep 10 15:46:43.060924 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:46:43.060883    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-865hl\" (UniqueName: \"kubernetes.io/projected/9deb994d-b35c-4651-bb7b-aba81f779366-kube-api-access-865hl\") pod \"payload-pre-processing-e2e-shared-a-w3-c7257b-5569d4b646-7pn87\" (UID: \"9deb994d-b35c-4651-bb7b-aba81f779366\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-c7257b-5569d4b646-7pn87"
Sep 10 15:46:43.061125 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:46:43.060954    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9deb994d-b35c-4651-bb7b-aba81f779366-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w3-c7257b-5569d4b646-7pn87\" (UID: \"9deb994d-b35c-4651-bb7b-aba81f779366\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-c7257b-5569d4b646-7pn87"
Sep 10 15:46:43.061624 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:46:43.061600    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9deb994d-b35c-4651-bb7b-aba81f779366-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w3-c7257b-5569d4b646-7pn87\" (UID: \"9deb994d-b35c-4651-bb7b-aba81f779366\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-c7257b-5569d4b646-7pn87"
Sep 10 15:46:43.071173 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:46:43.071144    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-865hl\" (UniqueName: \"kubernetes.io/projected/9deb994d-b35c-4651-bb7b-aba81f779366-kube-api-access-865hl\") pod \"payload-pre-processing-e2e-shared-a-w3-c7257b-5569d4b646-7pn87\" (UID: \"9deb994d-b35c-4651-bb7b-aba81f779366\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-c7257b-5569d4b646-7pn87"
Sep 10 15:46:43.088106 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:46:43.088061    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-c7257b-7bcfbc88b8-lkjp7"
Sep 10 15:46:43.106652 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:46:43.106617    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:46:43.243227 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:46:43.243194    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-c7257b-5569d4b646-7pn87"
Sep 10 15:46:43.315084 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:46:43.315045    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:46:43.362822 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:46:43.362793    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-c7257b-7bcfbc88b8-lkjp7"]
Sep 10 15:46:43.365282 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:46:43.365255    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podee2c75eb_0049_4111_94f7_39cc5e7896d1.slice/crio-80486dbb2b6aff80f3a1eb9e9f775bcf1c3891ec4581425eefeec1458fb21df1 WatchSource:0}: Error finding container 80486dbb2b6aff80f3a1eb9e9f775bcf1c3891ec4581425eefeec1458fb21df1: Status 404 returned error can't find the container with id 80486dbb2b6aff80f3a1eb9e9f775bcf1c3891ec4581425eefeec1458fb21df1
Sep 10 15:46:43.516617 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:46:43.516593    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-c7257b-5569d4b646-7pn87"]
Sep 10 15:46:43.520651 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:46:43.520616    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9deb994d_b35c_4651_bb7b_aba81f779366.slice/crio-c444c98fd37cd0c9da1e437fd96bcde12dff23234c78be97342875cc24f2a7bf WatchSource:0}: Error finding container c444c98fd37cd0c9da1e437fd96bcde12dff23234c78be97342875cc24f2a7bf: Status 404 returned error can't find the container with id c444c98fd37cd0c9da1e437fd96bcde12dff23234c78be97342875cc24f2a7bf
Sep 10 15:46:44.230660 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:46:44.230625    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-c7257b-5569d4b646-7pn87" event={"ID":"9deb994d-b35c-4651-bb7b-aba81f779366","Type":"ContainerStarted","Data":"c400a209fcada5676853c2ec9d2a389f9a20fe63679baa4a3ae1d16eb5508b4d"}
Sep 10 15:46:44.230870 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:46:44.230689    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-c7257b-5569d4b646-7pn87" event={"ID":"9deb994d-b35c-4651-bb7b-aba81f779366","Type":"ContainerStarted","Data":"c444c98fd37cd0c9da1e437fd96bcde12dff23234c78be97342875cc24f2a7bf"}
Sep 10 15:46:44.230870 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:46:44.230714    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-c7257b-5569d4b646-7pn87"
Sep 10 15:46:44.232155 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:46:44.232136    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-c7257b-7bcfbc88b8-lkjp7" event={"ID":"ee2c75eb-0049-4111-94f7-39cc5e7896d1","Type":"ContainerStarted","Data":"41726d1bc8d00639351e629004e5c78f8f0b705edda01d693deb4f56ffa2b3e5"}
Sep 10 15:46:44.232223 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:46:44.232172    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-c7257b-7bcfbc88b8-lkjp7" event={"ID":"ee2c75eb-0049-4111-94f7-39cc5e7896d1","Type":"ContainerStarted","Data":"80486dbb2b6aff80f3a1eb9e9f775bcf1c3891ec4581425eefeec1458fb21df1"}
Sep 10 15:46:44.232302 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:46:44.232270    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-c7257b-7bcfbc88b8-lkjp7"
Sep 10 15:46:44.250101 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:46:44.250053    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-c7257b-5569d4b646-7pn87" podStartSLOduration=2.2500372 podStartE2EDuration="2.2500372s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 15:46:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 15:46:44.249089285 +0000 UTC m=+1189.817321017" watchObservedRunningTime="2026-09-10 15:46:44.2500372 +0000 UTC m=+1189.818268998"
Sep 10 15:46:44.268927 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:46:44.268878    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-c7257b-7bcfbc88b8-lkjp7" podStartSLOduration=1.8540616540000001 podStartE2EDuration="2.268862226s" totalImagesPullingTime="414.800572ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 15:46:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 15:46:44.266183676 +0000 UTC m=+1189.834415409" watchObservedRunningTime="2026-09-10 15:46:44.268862226 +0000 UTC m=+1189.837093957"
Sep 10 15:46:45.539446 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:46:45.539415    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:46:45.639827 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:46:45.639792    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:46:46.407134 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:46:46.407100    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:46:46.784281 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:46:46.784221    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:46:47.278272 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:46:47.278224    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:46:47.858592 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:46:47.858557    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:46:48.909627 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:46:48.909581    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:46:49.705996 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:46:49.705954    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:46:49.806484 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:46:49.806441    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:46:50.071573 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:46:50.071487    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:46:50.242965 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:46:50.242934    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-c7257b-7bcfbc88b8-lkjp7"
Sep 10 15:46:50.849537 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:46:50.849500    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:46:51.705634 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:46:51.705599    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-30d70c-5bbb87697b-sc5pg"]
Sep 10 15:46:51.709490 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:46:51.709464    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-30d70c-5bbb87697b-sc5pg"
Sep 10 15:46:51.712118 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:46:51.712094    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-w2-30d70c-serving-cert\""
Sep 10 15:46:51.721352 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:46:51.721327    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-30d70c-5bbb87697b-sc5pg"]
Sep 10 15:46:51.747690 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:46:51.747658    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:46:51.845543 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:46:51.845504    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7d5hn\" (UniqueName: \"kubernetes.io/projected/52c8b4d6-193b-4ee5-aeed-88a414860ef2-kube-api-access-7d5hn\") pod \"maas-api-e2e-shared-a-w2-30d70c-5bbb87697b-sc5pg\" (UID: \"52c8b4d6-193b-4ee5-aeed-88a414860ef2\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-30d70c-5bbb87697b-sc5pg"
Sep 10 15:46:51.845715 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:46:51.845564    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/52c8b4d6-193b-4ee5-aeed-88a414860ef2-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w2-30d70c-5bbb87697b-sc5pg\" (UID: \"52c8b4d6-193b-4ee5-aeed-88a414860ef2\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-30d70c-5bbb87697b-sc5pg"
Sep 10 15:46:51.845715 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:46:51.845703    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/52c8b4d6-193b-4ee5-aeed-88a414860ef2-maas-api-tls\") pod \"maas-api-e2e-shared-a-w2-30d70c-5bbb87697b-sc5pg\" (UID: \"52c8b4d6-193b-4ee5-aeed-88a414860ef2\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-30d70c-5bbb87697b-sc5pg"
Sep 10 15:46:51.845809 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:46:51.845741    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/52c8b4d6-193b-4ee5-aeed-88a414860ef2-metrics-certs\") pod \"maas-api-e2e-shared-a-w2-30d70c-5bbb87697b-sc5pg\" (UID: \"52c8b4d6-193b-4ee5-aeed-88a414860ef2\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-30d70c-5bbb87697b-sc5pg"
Sep 10 15:46:51.848064 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:46:51.848042    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:46:51.947109 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:46:51.947069    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/52c8b4d6-193b-4ee5-aeed-88a414860ef2-maas-api-tls\") pod \"maas-api-e2e-shared-a-w2-30d70c-5bbb87697b-sc5pg\" (UID: \"52c8b4d6-193b-4ee5-aeed-88a414860ef2\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-30d70c-5bbb87697b-sc5pg"
Sep 10 15:46:51.947350 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:46:51.947128    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/52c8b4d6-193b-4ee5-aeed-88a414860ef2-metrics-certs\") pod \"maas-api-e2e-shared-a-w2-30d70c-5bbb87697b-sc5pg\" (UID: \"52c8b4d6-193b-4ee5-aeed-88a414860ef2\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-30d70c-5bbb87697b-sc5pg"
Sep 10 15:46:51.947350 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:46:51.947183    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7d5hn\" (UniqueName: \"kubernetes.io/projected/52c8b4d6-193b-4ee5-aeed-88a414860ef2-kube-api-access-7d5hn\") pod \"maas-api-e2e-shared-a-w2-30d70c-5bbb87697b-sc5pg\" (UID: \"52c8b4d6-193b-4ee5-aeed-88a414860ef2\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-30d70c-5bbb87697b-sc5pg"
Sep 10 15:46:51.947350 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:46:51.947211    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/52c8b4d6-193b-4ee5-aeed-88a414860ef2-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w2-30d70c-5bbb87697b-sc5pg\" (UID: \"52c8b4d6-193b-4ee5-aeed-88a414860ef2\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-30d70c-5bbb87697b-sc5pg"
Sep 10 15:46:51.947955 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:46:51.947926    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:46:51.948066 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:46:51.947970    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/52c8b4d6-193b-4ee5-aeed-88a414860ef2-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w2-30d70c-5bbb87697b-sc5pg\" (UID: \"52c8b4d6-193b-4ee5-aeed-88a414860ef2\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-30d70c-5bbb87697b-sc5pg"
Sep 10 15:46:51.949793 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:46:51.949768    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/52c8b4d6-193b-4ee5-aeed-88a414860ef2-maas-api-tls\") pod \"maas-api-e2e-shared-a-w2-30d70c-5bbb87697b-sc5pg\" (UID: \"52c8b4d6-193b-4ee5-aeed-88a414860ef2\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-30d70c-5bbb87697b-sc5pg"
Sep 10 15:46:51.949890 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:46:51.949768    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/52c8b4d6-193b-4ee5-aeed-88a414860ef2-metrics-certs\") pod \"maas-api-e2e-shared-a-w2-30d70c-5bbb87697b-sc5pg\" (UID: \"52c8b4d6-193b-4ee5-aeed-88a414860ef2\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-30d70c-5bbb87697b-sc5pg"
Sep 10 15:46:51.956392 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:46:51.956337    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7d5hn\" (UniqueName: \"kubernetes.io/projected/52c8b4d6-193b-4ee5-aeed-88a414860ef2-kube-api-access-7d5hn\") pod \"maas-api-e2e-shared-a-w2-30d70c-5bbb87697b-sc5pg\" (UID: \"52c8b4d6-193b-4ee5-aeed-88a414860ef2\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-30d70c-5bbb87697b-sc5pg"
Sep 10 15:46:52.023728 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:46:52.023681    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-30d70c-5bbb87697b-sc5pg"
Sep 10 15:46:52.282368 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:46:52.282289    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:46:52.307652 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:46:52.307604    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-30d70c-5bbb87697b-sc5pg"]
Sep 10 15:46:52.309988 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:46:52.309962    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod52c8b4d6_193b_4ee5_aeed_88a414860ef2.slice/crio-c6ce4f0e43aca9407d20dc2171cce85d2bd81cf9aad3310c624ad9a5fee9d971 WatchSource:0}: Error finding container c6ce4f0e43aca9407d20dc2171cce85d2bd81cf9aad3310c624ad9a5fee9d971: Status 404 returned error can't find the container with id c6ce4f0e43aca9407d20dc2171cce85d2bd81cf9aad3310c624ad9a5fee9d971
Sep 10 15:46:52.379919 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:46:52.379890    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:46:52.737379 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:46:52.737341    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:46:53.273879 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:46:53.273843    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-30d70c-5bbb87697b-sc5pg" event={"ID":"52c8b4d6-193b-4ee5-aeed-88a414860ef2","Type":"ContainerStarted","Data":"c6ce4f0e43aca9407d20dc2171cce85d2bd81cf9aad3310c624ad9a5fee9d971"}
Sep 10 15:46:53.547501 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:46:53.547473    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:46:54.280100 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:46:54.280061    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-30d70c-5bbb87697b-sc5pg" event={"ID":"52c8b4d6-193b-4ee5-aeed-88a414860ef2","Type":"ContainerStarted","Data":"9f428e4d1779e8ca7b6ee9136ffe699132bda326088f24aaa231ef4f3128409e"}
Sep 10 15:46:54.280491 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:46:54.280178    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-30d70c-5bbb87697b-sc5pg"
Sep 10 15:46:54.300392 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:46:54.300339    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-30d70c-5bbb87697b-sc5pg" podStartSLOduration=2.075450597 podStartE2EDuration="3.300323017s" totalImagesPullingTime="1.22487242s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 15:46:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 15:46:54.297744099 +0000 UTC m=+1199.865975828" watchObservedRunningTime="2026-09-10 15:46:54.300323017 +0000 UTC m=+1199.868554743"
Sep 10 15:46:54.399929 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:46:54.399896    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:46:54.909126 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:46:54.909090    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-t96pm_0b9d2cda-73e3-4e7c-9e17-1a3d903d95cf/console-operator/2.log"
Sep 10 15:46:54.912133 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:46:54.912105    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-t96pm_0b9d2cda-73e3-4e7c-9e17-1a3d903d95cf/console-operator/2.log"
Sep 10 15:46:55.239174 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:46:55.239143    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-c7257b-5569d4b646-7pn87"
Sep 10 15:46:55.652560 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:46:55.652472    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:46:56.137794 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:46:56.137734    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:46:56.308516 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:46:56.308476    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:46:56.426470 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:46:56.426428    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:46:59.129659 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:46:59.129625    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:46:59.785254 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:46:59.785188    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:46:59.878004 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:46:59.877965    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:46:59.978447 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:46:59.978417    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:47:00.292169 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:47:00.292136    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-30d70c-5bbb87697b-sc5pg"
Sep 10 15:47:01.015117 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:47:01.015073    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:47:01.812660 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:47:01.812623    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-3650b0-5bb8755959-pqcth"]
Sep 10 15:47:01.818310 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:47:01.818284    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-3650b0-5bb8755959-pqcth"
Sep 10 15:47:01.821397 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:47:01.821371    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-w3-3650b0-serving-cert\""
Sep 10 15:47:01.828034 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:47:01.828005    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-3650b0-5bb8755959-pqcth"]
Sep 10 15:47:01.860563 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:47:01.860530    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:47:01.937012 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:47:01.936965    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/524402b2-7893-418c-8e4b-2364a39cad2c-metrics-certs\") pod \"maas-api-e2e-shared-b-w3-3650b0-5bb8755959-pqcth\" (UID: \"524402b2-7893-418c-8e4b-2364a39cad2c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-3650b0-5bb8755959-pqcth"
Sep 10 15:47:01.937203 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:47:01.937077    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/524402b2-7893-418c-8e4b-2364a39cad2c-maas-api-tls\") pod \"maas-api-e2e-shared-b-w3-3650b0-5bb8755959-pqcth\" (UID: \"524402b2-7893-418c-8e4b-2364a39cad2c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-3650b0-5bb8755959-pqcth"
Sep 10 15:47:01.937203 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:47:01.937103    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/524402b2-7893-418c-8e4b-2364a39cad2c-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w3-3650b0-5bb8755959-pqcth\" (UID: \"524402b2-7893-418c-8e4b-2364a39cad2c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-3650b0-5bb8755959-pqcth"
Sep 10 15:47:01.937203 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:47:01.937176    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-782j9\" (UniqueName: \"kubernetes.io/projected/524402b2-7893-418c-8e4b-2364a39cad2c-kube-api-access-782j9\") pod \"maas-api-e2e-shared-b-w3-3650b0-5bb8755959-pqcth\" (UID: \"524402b2-7893-418c-8e4b-2364a39cad2c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-3650b0-5bb8755959-pqcth"
Sep 10 15:47:01.959592 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:47:01.959562    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:47:02.038502 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:47:02.038463    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/524402b2-7893-418c-8e4b-2364a39cad2c-maas-api-tls\") pod \"maas-api-e2e-shared-b-w3-3650b0-5bb8755959-pqcth\" (UID: \"524402b2-7893-418c-8e4b-2364a39cad2c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-3650b0-5bb8755959-pqcth"
Sep 10 15:47:02.038699 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:47:02.038506    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/524402b2-7893-418c-8e4b-2364a39cad2c-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w3-3650b0-5bb8755959-pqcth\" (UID: \"524402b2-7893-418c-8e4b-2364a39cad2c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-3650b0-5bb8755959-pqcth"
Sep 10 15:47:02.038699 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:47:02.038571    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-782j9\" (UniqueName: \"kubernetes.io/projected/524402b2-7893-418c-8e4b-2364a39cad2c-kube-api-access-782j9\") pod \"maas-api-e2e-shared-b-w3-3650b0-5bb8755959-pqcth\" (UID: \"524402b2-7893-418c-8e4b-2364a39cad2c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-3650b0-5bb8755959-pqcth"
Sep 10 15:47:02.038699 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:47:02.038600    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/524402b2-7893-418c-8e4b-2364a39cad2c-metrics-certs\") pod \"maas-api-e2e-shared-b-w3-3650b0-5bb8755959-pqcth\" (UID: \"524402b2-7893-418c-8e4b-2364a39cad2c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-3650b0-5bb8755959-pqcth"
Sep 10 15:47:02.038699 ip-10-0-128-241 kubenswrapper[2567]: E0910 15:47:02.038626    2567 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-3650b0-serving-cert: secret "maas-api-e2e-shared-b-w3-3650b0-serving-cert" not found
Sep 10 15:47:02.038699 ip-10-0-128-241 kubenswrapper[2567]: E0910 15:47:02.038694    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/524402b2-7893-418c-8e4b-2364a39cad2c-maas-api-tls podName:524402b2-7893-418c-8e4b-2364a39cad2c nodeName:}" failed. No retries permitted until 2026-09-10 15:47:02.538676463 +0000 UTC m=+1208.106908194 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/524402b2-7893-418c-8e4b-2364a39cad2c-maas-api-tls") pod "maas-api-e2e-shared-b-w3-3650b0-5bb8755959-pqcth" (UID: "524402b2-7893-418c-8e4b-2364a39cad2c") : secret "maas-api-e2e-shared-b-w3-3650b0-serving-cert" not found
Sep 10 15:47:02.039367 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:47:02.039343    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/524402b2-7893-418c-8e4b-2364a39cad2c-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w3-3650b0-5bb8755959-pqcth\" (UID: \"524402b2-7893-418c-8e4b-2364a39cad2c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-3650b0-5bb8755959-pqcth"
Sep 10 15:47:02.041108 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:47:02.041091    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/524402b2-7893-418c-8e4b-2364a39cad2c-metrics-certs\") pod \"maas-api-e2e-shared-b-w3-3650b0-5bb8755959-pqcth\" (UID: \"524402b2-7893-418c-8e4b-2364a39cad2c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-3650b0-5bb8755959-pqcth"
Sep 10 15:47:02.047313 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:47:02.047291    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-782j9\" (UniqueName: \"kubernetes.io/projected/524402b2-7893-418c-8e4b-2364a39cad2c-kube-api-access-782j9\") pod \"maas-api-e2e-shared-b-w3-3650b0-5bb8755959-pqcth\" (UID: \"524402b2-7893-418c-8e4b-2364a39cad2c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-3650b0-5bb8755959-pqcth"
Sep 10 15:47:02.245692 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:47:02.245656    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:47:02.431544 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:47:02.431493    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:47:02.544302 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:47:02.544201    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/524402b2-7893-418c-8e4b-2364a39cad2c-maas-api-tls\") pod \"maas-api-e2e-shared-b-w3-3650b0-5bb8755959-pqcth\" (UID: \"524402b2-7893-418c-8e4b-2364a39cad2c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-3650b0-5bb8755959-pqcth"
Sep 10 15:47:02.546769 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:47:02.546746    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/524402b2-7893-418c-8e4b-2364a39cad2c-maas-api-tls\") pod \"maas-api-e2e-shared-b-w3-3650b0-5bb8755959-pqcth\" (UID: \"524402b2-7893-418c-8e4b-2364a39cad2c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-3650b0-5bb8755959-pqcth"
Sep 10 15:47:02.609321 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:47:02.609283    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-3650b0-599b79859d-w2qxr"]
Sep 10 15:47:02.612951 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:47:02.612929    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-3650b0-599b79859d-w2qxr"
Sep 10 15:47:02.615553 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:47:02.615532    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-b-w3-3650b0-dockercfg-vjwcp\""
Sep 10 15:47:02.615666 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:47:02.615567    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-w3-3650b0\""
Sep 10 15:47:02.624712 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:47:02.624687    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-3650b0-599b79859d-w2qxr"]
Sep 10 15:47:02.648452 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:47:02.648420    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:47:02.732168 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:47:02.732131    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-3650b0-5bb8755959-pqcth"
Sep 10 15:47:02.745468 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:47:02.745427    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/738e3d0d-8d31-4cec-bfe3-efa3c955d119-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w3-3650b0-599b79859d-w2qxr\" (UID: \"738e3d0d-8d31-4cec-bfe3-efa3c955d119\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-3650b0-599b79859d-w2qxr"
Sep 10 15:47:02.745604 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:47:02.745527    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9lnkt\" (UniqueName: \"kubernetes.io/projected/738e3d0d-8d31-4cec-bfe3-efa3c955d119-kube-api-access-9lnkt\") pod \"payload-pre-processing-e2e-shared-b-w3-3650b0-599b79859d-w2qxr\" (UID: \"738e3d0d-8d31-4cec-bfe3-efa3c955d119\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-3650b0-599b79859d-w2qxr"
Sep 10 15:47:02.846702 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:47:02.846612    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9lnkt\" (UniqueName: \"kubernetes.io/projected/738e3d0d-8d31-4cec-bfe3-efa3c955d119-kube-api-access-9lnkt\") pod \"payload-pre-processing-e2e-shared-b-w3-3650b0-599b79859d-w2qxr\" (UID: \"738e3d0d-8d31-4cec-bfe3-efa3c955d119\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-3650b0-599b79859d-w2qxr"
Sep 10 15:47:02.846702 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:47:02.846689    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/738e3d0d-8d31-4cec-bfe3-efa3c955d119-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w3-3650b0-599b79859d-w2qxr\" (UID: \"738e3d0d-8d31-4cec-bfe3-efa3c955d119\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-3650b0-599b79859d-w2qxr"
Sep 10 15:47:02.847465 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:47:02.847445    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/738e3d0d-8d31-4cec-bfe3-efa3c955d119-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w3-3650b0-599b79859d-w2qxr\" (UID: \"738e3d0d-8d31-4cec-bfe3-efa3c955d119\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-3650b0-599b79859d-w2qxr"
Sep 10 15:47:02.856006 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:47:02.855979    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9lnkt\" (UniqueName: \"kubernetes.io/projected/738e3d0d-8d31-4cec-bfe3-efa3c955d119-kube-api-access-9lnkt\") pod \"payload-pre-processing-e2e-shared-b-w3-3650b0-599b79859d-w2qxr\" (UID: \"738e3d0d-8d31-4cec-bfe3-efa3c955d119\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-3650b0-599b79859d-w2qxr"
Sep 10 15:47:02.862394 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:47:02.862366    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:47:02.924428 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:47:02.924397    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-3650b0-599b79859d-w2qxr"
Sep 10 15:47:03.011833 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:47:03.011800    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-3650b0-5bb8755959-pqcth"]
Sep 10 15:47:03.013130 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:47:03.013105    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod524402b2_7893_418c_8e4b_2364a39cad2c.slice/crio-fa77b53ff966b53912124c1993a520217080666f7f2d906c1392fa443b098e4a WatchSource:0}: Error finding container fa77b53ff966b53912124c1993a520217080666f7f2d906c1392fa443b098e4a: Status 404 returned error can't find the container with id fa77b53ff966b53912124c1993a520217080666f7f2d906c1392fa443b098e4a
Sep 10 15:47:03.200871 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:47:03.200846    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-3650b0-599b79859d-w2qxr"]
Sep 10 15:47:03.203221 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:47:03.203184    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod738e3d0d_8d31_4cec_bfe3_efa3c955d119.slice/crio-7c7511aed79574d8fda1a6b43518aec12f40e58d1c42c197aa8c61e25afccaf5 WatchSource:0}: Error finding container 7c7511aed79574d8fda1a6b43518aec12f40e58d1c42c197aa8c61e25afccaf5: Status 404 returned error can't find the container with id 7c7511aed79574d8fda1a6b43518aec12f40e58d1c42c197aa8c61e25afccaf5
Sep 10 15:47:03.324186 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:47:03.324155    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-3650b0-5bb8755959-pqcth" event={"ID":"524402b2-7893-418c-8e4b-2364a39cad2c","Type":"ContainerStarted","Data":"fa77b53ff966b53912124c1993a520217080666f7f2d906c1392fa443b098e4a"}
Sep 10 15:47:03.325948 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:47:03.325921    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-3650b0-599b79859d-w2qxr" event={"ID":"738e3d0d-8d31-4cec-bfe3-efa3c955d119","Type":"ContainerStarted","Data":"1f52ae7592e282c35b9586da976b93583c9ed17c66c1eb5883c6fe45afa02c8d"}
Sep 10 15:47:03.326084 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:47:03.325965    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-3650b0-599b79859d-w2qxr" event={"ID":"738e3d0d-8d31-4cec-bfe3-efa3c955d119","Type":"ContainerStarted","Data":"7c7511aed79574d8fda1a6b43518aec12f40e58d1c42c197aa8c61e25afccaf5"}
Sep 10 15:47:03.326084 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:47:03.326050    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-3650b0-599b79859d-w2qxr"
Sep 10 15:47:03.344300 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:47:03.344252    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-3650b0-599b79859d-w2qxr" podStartSLOduration=1.344222906 podStartE2EDuration="1.344222906s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 15:47:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 15:47:03.342773509 +0000 UTC m=+1208.911005240" watchObservedRunningTime="2026-09-10 15:47:03.344222906 +0000 UTC m=+1208.912454635"
Sep 10 15:47:03.726653 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:47:03.726525    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:47:04.332202 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:47:04.332158    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-3650b0-5bb8755959-pqcth" event={"ID":"524402b2-7893-418c-8e4b-2364a39cad2c","Type":"ContainerStarted","Data":"0a554680fc168b8aaf434aae533a085c84d8d9db61d543e2ed7151982f3a5da1"}
Sep 10 15:47:04.332650 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:47:04.332363    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-3650b0-5bb8755959-pqcth"
Sep 10 15:47:04.364076 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:47:04.364026    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-3650b0-5bb8755959-pqcth" podStartSLOduration=3.000418297 podStartE2EDuration="3.3640101s" totalImagesPullingTime="363.591803ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 15:47:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 15:47:04.362459823 +0000 UTC m=+1209.930691554" watchObservedRunningTime="2026-09-10 15:47:04.3640101 +0000 UTC m=+1209.932241830"
Sep 10 15:47:04.434412 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:47:04.434374    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:47:05.451576 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:47:05.451541    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:47:05.551551 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:47:05.551496    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:47:05.666223 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:47:05.666192    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:47:05.761298 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:47:05.761202    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:47:07.173351 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:47:07.173315    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:47:07.869685 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:47:07.869648    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:47:07.961874 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:47:07.961840    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:47:08.381431 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:47:08.381394    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:47:09.456941 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:47:09.456905    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:47:10.343005 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:47:10.342971    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-3650b0-5bb8755959-pqcth"
Sep 10 15:47:10.481458 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:47:10.481427    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:47:11.271717 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:47:11.271686    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:47:11.341033 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:47:11.341002    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:47:11.476249 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:47:11.476205    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:47:11.916922 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:47:11.916889    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:47:12.016833 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:47:12.016797    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:47:12.025266 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:47:12.025241    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-ec08ac-7d9964cf54-fk5zt"]
Sep 10 15:47:12.028705 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:47:12.028687    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-ec08ac-7d9964cf54-fk5zt"
Sep 10 15:47:12.031144 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:47:12.031124    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-b-w2-ec08ac-dockercfg-sprr5\""
Sep 10 15:47:12.031264 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:47:12.031124    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-w2-ec08ac\""
Sep 10 15:47:12.038918 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:47:12.038887    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-ec08ac-7d9964cf54-fk5zt"]
Sep 10 15:47:12.042719 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:47:12.042697    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-ec08ac-68959b96f9-dnnld"]
Sep 10 15:47:12.046400 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:47:12.046381    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-ec08ac-68959b96f9-dnnld"
Sep 10 15:47:12.057617 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:47:12.057591    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-ec08ac-68959b96f9-dnnld"]
Sep 10 15:47:12.144517 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:47:12.144471    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tvzhj\" (UniqueName: \"kubernetes.io/projected/1ea2c51d-8376-4903-88e6-7db46a6f6264-kube-api-access-tvzhj\") pod \"payload-pre-processing-e2e-shared-b-w2-ec08ac-7d9964cf54-fk5zt\" (UID: \"1ea2c51d-8376-4903-88e6-7db46a6f6264\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-ec08ac-7d9964cf54-fk5zt"
Sep 10 15:47:12.144707 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:47:12.144541    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1ea2c51d-8376-4903-88e6-7db46a6f6264-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w2-ec08ac-7d9964cf54-fk5zt\" (UID: \"1ea2c51d-8376-4903-88e6-7db46a6f6264\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-ec08ac-7d9964cf54-fk5zt"
Sep 10 15:47:12.245734 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:47:12.245695    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7e4bd2b6-bf50-4289-88a6-f9e123c50409-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w2-ec08ac-68959b96f9-dnnld\" (UID: \"7e4bd2b6-bf50-4289-88a6-f9e123c50409\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-ec08ac-68959b96f9-dnnld"
Sep 10 15:47:12.245918 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:47:12.245762    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pwcvx\" (UniqueName: \"kubernetes.io/projected/7e4bd2b6-bf50-4289-88a6-f9e123c50409-kube-api-access-pwcvx\") pod \"payload-processing-e2e-shared-b-w2-ec08ac-68959b96f9-dnnld\" (UID: \"7e4bd2b6-bf50-4289-88a6-f9e123c50409\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-ec08ac-68959b96f9-dnnld"
Sep 10 15:47:12.245918 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:47:12.245797    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tvzhj\" (UniqueName: \"kubernetes.io/projected/1ea2c51d-8376-4903-88e6-7db46a6f6264-kube-api-access-tvzhj\") pod \"payload-pre-processing-e2e-shared-b-w2-ec08ac-7d9964cf54-fk5zt\" (UID: \"1ea2c51d-8376-4903-88e6-7db46a6f6264\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-ec08ac-7d9964cf54-fk5zt"
Sep 10 15:47:12.245918 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:47:12.245827    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1ea2c51d-8376-4903-88e6-7db46a6f6264-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w2-ec08ac-7d9964cf54-fk5zt\" (UID: \"1ea2c51d-8376-4903-88e6-7db46a6f6264\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-ec08ac-7d9964cf54-fk5zt"
Sep 10 15:47:12.246596 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:47:12.246561    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1ea2c51d-8376-4903-88e6-7db46a6f6264-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w2-ec08ac-7d9964cf54-fk5zt\" (UID: \"1ea2c51d-8376-4903-88e6-7db46a6f6264\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-ec08ac-7d9964cf54-fk5zt"
Sep 10 15:47:12.254906 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:47:12.254876    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tvzhj\" (UniqueName: \"kubernetes.io/projected/1ea2c51d-8376-4903-88e6-7db46a6f6264-kube-api-access-tvzhj\") pod \"payload-pre-processing-e2e-shared-b-w2-ec08ac-7d9964cf54-fk5zt\" (UID: \"1ea2c51d-8376-4903-88e6-7db46a6f6264\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-ec08ac-7d9964cf54-fk5zt"
Sep 10 15:47:12.339600 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:47:12.339564    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-ec08ac-7d9964cf54-fk5zt"
Sep 10 15:47:12.346904 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:47:12.346880    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7e4bd2b6-bf50-4289-88a6-f9e123c50409-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w2-ec08ac-68959b96f9-dnnld\" (UID: \"7e4bd2b6-bf50-4289-88a6-f9e123c50409\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-ec08ac-68959b96f9-dnnld"
Sep 10 15:47:12.346999 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:47:12.346922    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pwcvx\" (UniqueName: \"kubernetes.io/projected/7e4bd2b6-bf50-4289-88a6-f9e123c50409-kube-api-access-pwcvx\") pod \"payload-processing-e2e-shared-b-w2-ec08ac-68959b96f9-dnnld\" (UID: \"7e4bd2b6-bf50-4289-88a6-f9e123c50409\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-ec08ac-68959b96f9-dnnld"
Sep 10 15:47:12.347730 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:47:12.347708    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7e4bd2b6-bf50-4289-88a6-f9e123c50409-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w2-ec08ac-68959b96f9-dnnld\" (UID: \"7e4bd2b6-bf50-4289-88a6-f9e123c50409\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-ec08ac-68959b96f9-dnnld"
Sep 10 15:47:12.356640 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:47:12.356609    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pwcvx\" (UniqueName: \"kubernetes.io/projected/7e4bd2b6-bf50-4289-88a6-f9e123c50409-kube-api-access-pwcvx\") pod \"payload-processing-e2e-shared-b-w2-ec08ac-68959b96f9-dnnld\" (UID: \"7e4bd2b6-bf50-4289-88a6-f9e123c50409\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-ec08ac-68959b96f9-dnnld"
Sep 10 15:47:12.358249 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:47:12.358205    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-ec08ac-68959b96f9-dnnld"
Sep 10 15:47:12.623055 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:47:12.623023    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-ec08ac-7d9964cf54-fk5zt"]
Sep 10 15:47:12.624445 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:47:12.624417    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1ea2c51d_8376_4903_88e6_7db46a6f6264.slice/crio-fa4545909dc07d80095fc4f1ea2724f1ac7354c5f68f7ee23b1c052ec8d92ab4 WatchSource:0}: Error finding container fa4545909dc07d80095fc4f1ea2724f1ac7354c5f68f7ee23b1c052ec8d92ab4: Status 404 returned error can't find the container with id fa4545909dc07d80095fc4f1ea2724f1ac7354c5f68f7ee23b1c052ec8d92ab4
Sep 10 15:47:12.652491 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:47:12.652468    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-ec08ac-68959b96f9-dnnld"]
Sep 10 15:47:12.658329 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:47:12.658305    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7e4bd2b6_bf50_4289_88a6_f9e123c50409.slice/crio-cc7e579f369cf97a2efaf4831ede1af0a2711828040c41b9af2ed5e714744d54 WatchSource:0}: Error finding container cc7e579f369cf97a2efaf4831ede1af0a2711828040c41b9af2ed5e714744d54: Status 404 returned error can't find the container with id cc7e579f369cf97a2efaf4831ede1af0a2711828040c41b9af2ed5e714744d54
Sep 10 15:47:13.386850 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:47:13.386809    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-ec08ac-68959b96f9-dnnld" event={"ID":"7e4bd2b6-bf50-4289-88a6-f9e123c50409","Type":"ContainerStarted","Data":"9c1f24fd9a31ebe52b9c0b957206d3dd6b753f782c50efc3d00c7737283dd203"}
Sep 10 15:47:13.387336 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:47:13.386881    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-ec08ac-68959b96f9-dnnld" event={"ID":"7e4bd2b6-bf50-4289-88a6-f9e123c50409","Type":"ContainerStarted","Data":"cc7e579f369cf97a2efaf4831ede1af0a2711828040c41b9af2ed5e714744d54"}
Sep 10 15:47:13.387336 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:47:13.386914    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-ec08ac-68959b96f9-dnnld"
Sep 10 15:47:13.388484 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:47:13.388464    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-ec08ac-7d9964cf54-fk5zt" event={"ID":"1ea2c51d-8376-4903-88e6-7db46a6f6264","Type":"ContainerStarted","Data":"34d9b5beda566f37f6cf20bcfb631424cbb222aaa8a44ad79fe84cef06235191"}
Sep 10 15:47:13.388584 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:47:13.388505    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-ec08ac-7d9964cf54-fk5zt" event={"ID":"1ea2c51d-8376-4903-88e6-7db46a6f6264","Type":"ContainerStarted","Data":"fa4545909dc07d80095fc4f1ea2724f1ac7354c5f68f7ee23b1c052ec8d92ab4"}
Sep 10 15:47:13.388584 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:47:13.388579    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-ec08ac-7d9964cf54-fk5zt"
Sep 10 15:47:13.408620 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:47:13.408570    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-ec08ac-68959b96f9-dnnld" podStartSLOduration=1.408555691 podStartE2EDuration="1.408555691s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 15:47:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 15:47:13.407718469 +0000 UTC m=+1218.975950197" watchObservedRunningTime="2026-09-10 15:47:13.408555691 +0000 UTC m=+1218.976787420"
Sep 10 15:47:13.442128 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:47:13.442079    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-ec08ac-7d9964cf54-fk5zt" podStartSLOduration=1.44206376 podStartE2EDuration="1.44206376s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 15:47:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 15:47:13.441055669 +0000 UTC m=+1219.009287384" watchObservedRunningTime="2026-09-10 15:47:13.44206376 +0000 UTC m=+1219.010295524"
Sep 10 15:47:13.476629 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:47:13.476599    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:47:13.961038 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:47:13.961008    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:47:14.335182 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:47:14.335090    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-3650b0-599b79859d-w2qxr"
Sep 10 15:47:22.858856 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:47:22.858822    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:47:24.350597 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:47:24.350558    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:47:24.406364 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:47:24.406330    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-ec08ac-68959b96f9-dnnld"
Sep 10 15:47:24.406570 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:47:24.406412    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-ec08ac-7d9964cf54-fk5zt"
Sep 10 15:47:24.736227 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:47:24.736194    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:47:25.264396 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:47:25.264359    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:47:26.067690 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:47:26.067659    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:47:27.016009 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:47:27.015977    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:47:27.691084 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:47:27.691045    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:47:27.814362 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:47:27.814320    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:47:28.774056 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:47:28.774021    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:47:29.633938 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:47:29.633905    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:47:30.541988 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:47:30.541947    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:47:31.312829 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:47:31.312792    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:47:31.755645 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:47:31.755614    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:47:32.347744 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:47:32.347709    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:47:32.905320 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:47:32.905291    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7749-926c65"]
Sep 10 15:47:32.908888 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:47:32.908868    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7749-926c65"
Sep 10 15:47:32.911562 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:47:32.911541    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-wtngs\""
Sep 10 15:47:32.914581 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:47:32.914562    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7749-926c65"]
Sep 10 15:47:33.035973 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:47:33.035942    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xk6pg\" (UniqueName: \"kubernetes.io/projected/556e7723-ea7b-4b32-b8c6-690eda7c0283-kube-api-access-xk6pg\") pod \"test-curl-7749-926c65\" (UID: \"556e7723-ea7b-4b32-b8c6-690eda7c0283\") " pod="opendatahub/test-curl-7749-926c65"
Sep 10 15:47:33.137062 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:47:33.137025    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xk6pg\" (UniqueName: \"kubernetes.io/projected/556e7723-ea7b-4b32-b8c6-690eda7c0283-kube-api-access-xk6pg\") pod \"test-curl-7749-926c65\" (UID: \"556e7723-ea7b-4b32-b8c6-690eda7c0283\") " pod="opendatahub/test-curl-7749-926c65"
Sep 10 15:47:33.145412 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:47:33.145389    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xk6pg\" (UniqueName: \"kubernetes.io/projected/556e7723-ea7b-4b32-b8c6-690eda7c0283-kube-api-access-xk6pg\") pod \"test-curl-7749-926c65\" (UID: \"556e7723-ea7b-4b32-b8c6-690eda7c0283\") " pod="opendatahub/test-curl-7749-926c65"
Sep 10 15:47:33.220467 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:47:33.220438    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7749-926c65"
Sep 10 15:47:33.491496 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:47:33.491466    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7749-926c65"]
Sep 10 15:47:33.494374 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:47:33.494337    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod556e7723_ea7b_4b32_b8c6_690eda7c0283.slice/crio-69c21e3c88bbb2a0253d67c9e4838f6fd7e2b8ee2a0c33a446cf4e6a8be223d6 WatchSource:0}: Error finding container 69c21e3c88bbb2a0253d67c9e4838f6fd7e2b8ee2a0c33a446cf4e6a8be223d6: Status 404 returned error can't find the container with id 69c21e3c88bbb2a0253d67c9e4838f6fd7e2b8ee2a0c33a446cf4e6a8be223d6
Sep 10 15:47:34.504521 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:47:34.504451    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7749-926c65" event={"ID":"556e7723-ea7b-4b32-b8c6-690eda7c0283","Type":"ContainerStarted","Data":"69c21e3c88bbb2a0253d67c9e4838f6fd7e2b8ee2a0c33a446cf4e6a8be223d6"}
Sep 10 15:47:35.510800 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:47:35.510769    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7749-926c65" event={"ID":"556e7723-ea7b-4b32-b8c6-690eda7c0283","Type":"ContainerStarted","Data":"0773cfc2c9f64a2f9bf1380583a54e59552d4f15eaa3ca581842d28fda5df44f"}
Sep 10 15:47:35.525651 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:47:35.525603    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7749-926c65" podStartSLOduration=1.588691162 podStartE2EDuration="3.525587684s" totalImagesPullingTime="1.936896522s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 15:47:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 15:47:35.524700061 +0000 UTC m=+1241.092931792" watchObservedRunningTime="2026-09-10 15:47:35.525587684 +0000 UTC m=+1241.093819448"
Sep 10 15:47:36.099030 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:47:36.098993    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7749-926c65"]
Sep 10 15:47:36.104636 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:47:36.104606    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7749-926c65"]
Sep 10 15:47:36.515645 ip-10-0-128-241 kubenswrapper[2567]: E0910 15:47:36.515613    2567 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7749-926c65" container="test-curl-7749-926c65"
Sep 10 15:47:37.519619 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:47:37.519571    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7749-926c65" podUID="556e7723-ea7b-4b32-b8c6-690eda7c0283" containerName="test-curl-7749-926c65" containerID="cri-o://0773cfc2c9f64a2f9bf1380583a54e59552d4f15eaa3ca581842d28fda5df44f" gracePeriod=2
Sep 10 15:47:39.655457 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:47:39.655422    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:47:39.699874 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:47:39.699851    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7749-926c65"
Sep 10 15:47:39.804988 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:47:39.804898    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xk6pg\" (UniqueName: \"kubernetes.io/projected/556e7723-ea7b-4b32-b8c6-690eda7c0283-kube-api-access-xk6pg\") pod \"556e7723-ea7b-4b32-b8c6-690eda7c0283\" (UID: \"556e7723-ea7b-4b32-b8c6-690eda7c0283\") "
Sep 10 15:47:39.806944 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:47:39.806907    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/556e7723-ea7b-4b32-b8c6-690eda7c0283-kube-api-access-xk6pg" (OuterVolumeSpecName: "kube-api-access-xk6pg") pod "556e7723-ea7b-4b32-b8c6-690eda7c0283" (UID: "556e7723-ea7b-4b32-b8c6-690eda7c0283"). InnerVolumeSpecName "kube-api-access-xk6pg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 15:47:39.906544 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:47:39.906505    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xk6pg\" (UniqueName: \"kubernetes.io/projected/556e7723-ea7b-4b32-b8c6-690eda7c0283-kube-api-access-xk6pg\") on node \"ip-10-0-128-241.ec2.internal\" DevicePath \"\""
Sep 10 15:47:40.537957 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:47:40.537917    2567 generic.go:356] "Generic (PLEG): container finished" podID="556e7723-ea7b-4b32-b8c6-690eda7c0283" containerID="0773cfc2c9f64a2f9bf1380583a54e59552d4f15eaa3ca581842d28fda5df44f" exitCode=137
Sep 10 15:47:40.538170 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:47:40.537968    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7749-926c65"
Sep 10 15:47:40.538170 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:47:40.538019    2567 scope.go:117] "RemoveContainer" containerID="0773cfc2c9f64a2f9bf1380583a54e59552d4f15eaa3ca581842d28fda5df44f"
Sep 10 15:47:40.548501 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:47:40.548186    2567 scope.go:117] "RemoveContainer" containerID="0773cfc2c9f64a2f9bf1380583a54e59552d4f15eaa3ca581842d28fda5df44f"
Sep 10 15:47:40.548586 ip-10-0-128-241 kubenswrapper[2567]: E0910 15:47:40.548562    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0773cfc2c9f64a2f9bf1380583a54e59552d4f15eaa3ca581842d28fda5df44f\": container with ID starting with 0773cfc2c9f64a2f9bf1380583a54e59552d4f15eaa3ca581842d28fda5df44f not found: ID does not exist" containerID="0773cfc2c9f64a2f9bf1380583a54e59552d4f15eaa3ca581842d28fda5df44f"
Sep 10 15:47:40.548650 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:47:40.548599    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0773cfc2c9f64a2f9bf1380583a54e59552d4f15eaa3ca581842d28fda5df44f"} err="failed to get container status \"0773cfc2c9f64a2f9bf1380583a54e59552d4f15eaa3ca581842d28fda5df44f\": rpc error: code = NotFound desc = could not find container \"0773cfc2c9f64a2f9bf1380583a54e59552d4f15eaa3ca581842d28fda5df44f\": container with ID starting with 0773cfc2c9f64a2f9bf1380583a54e59552d4f15eaa3ca581842d28fda5df44f not found: ID does not exist"
Sep 10 15:47:40.550954 ip-10-0-128-241 kubenswrapper[2567]: E0910 15:47:40.550931    2567 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7749-926c65\" is forbidden: User \"system:node:ip-10-0-128-241.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-128-241.ec2.internal' and this object" podUID="556e7723-ea7b-4b32-b8c6-690eda7c0283" pod="opendatahub/test-curl-7749-926c65"
Sep 10 15:47:40.903978 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:47:40.903882    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="556e7723-ea7b-4b32-b8c6-690eda7c0283" path="/var/lib/kubelet/pods/556e7723-ea7b-4b32-b8c6-690eda7c0283/volumes"
Sep 10 15:47:41.308208 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:47:41.308176    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:47:42.711466 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:47:42.711433    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:47:43.361209 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:47:43.361175    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:47:43.457948 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:47:43.457914    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:47:43.561362 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:47:43.561326    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:47:44.560811 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:47:44.560779    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:47:45.370327 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:47:45.370292    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:47:45.617396 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:47:45.617362    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:47:45.716476 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:47:45.716442    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:47:46.141905 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:47:46.141424    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:47:47.153578 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:47:47.153479    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:47:47.329533 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:47:47.329505    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:47:47.430629 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:47:47.430595    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:47:47.836590 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:47:47.836504    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:47:47.937628 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:47:47.937598    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:47:47.974519 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:47:47.974482    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-81f538c6-7546b749d5-s45bb"]
Sep 10 15:47:47.974876 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:47:47.974861    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="556e7723-ea7b-4b32-b8c6-690eda7c0283" containerName="test-curl-7749-926c65"
Sep 10 15:47:47.974925 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:47:47.974879    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="556e7723-ea7b-4b32-b8c6-690eda7c0283" containerName="test-curl-7749-926c65"
Sep 10 15:47:47.975002 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:47:47.974991    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="556e7723-ea7b-4b32-b8c6-690eda7c0283" containerName="test-curl-7749-926c65"
Sep 10 15:47:47.979375 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:47:47.979359    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-81f538c6-7546b749d5-s45bb"
Sep 10 15:47:47.981813 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:47:47.981794    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-81f538c6-dockercfg-fqncz\""
Sep 10 15:47:47.981880 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:47:47.981863    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-81f538c6\""
Sep 10 15:47:47.989695 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:47:47.989667    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-81f538c6-7546b749d5-s45bb"]
Sep 10 15:47:48.086715 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:47:48.086633    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mq7t7\" (UniqueName: \"kubernetes.io/projected/d0b7dd62-c5e6-4689-a0d5-dd41a15f6e96-kube-api-access-mq7t7\") pod \"payload-processing-e2e-mt-81f538c6-7546b749d5-s45bb\" (UID: \"d0b7dd62-c5e6-4689-a0d5-dd41a15f6e96\") " pod="openshift-ingress/payload-processing-e2e-mt-81f538c6-7546b749d5-s45bb"
Sep 10 15:47:48.086715 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:47:48.086693    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d0b7dd62-c5e6-4689-a0d5-dd41a15f6e96-plugins-config-volume\") pod \"payload-processing-e2e-mt-81f538c6-7546b749d5-s45bb\" (UID: \"d0b7dd62-c5e6-4689-a0d5-dd41a15f6e96\") " pod="openshift-ingress/payload-processing-e2e-mt-81f538c6-7546b749d5-s45bb"
Sep 10 15:47:48.187998 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:47:48.187963    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mq7t7\" (UniqueName: \"kubernetes.io/projected/d0b7dd62-c5e6-4689-a0d5-dd41a15f6e96-kube-api-access-mq7t7\") pod \"payload-processing-e2e-mt-81f538c6-7546b749d5-s45bb\" (UID: \"d0b7dd62-c5e6-4689-a0d5-dd41a15f6e96\") " pod="openshift-ingress/payload-processing-e2e-mt-81f538c6-7546b749d5-s45bb"
Sep 10 15:47:48.188442 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:47:48.188023    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d0b7dd62-c5e6-4689-a0d5-dd41a15f6e96-plugins-config-volume\") pod \"payload-processing-e2e-mt-81f538c6-7546b749d5-s45bb\" (UID: \"d0b7dd62-c5e6-4689-a0d5-dd41a15f6e96\") " pod="openshift-ingress/payload-processing-e2e-mt-81f538c6-7546b749d5-s45bb"
Sep 10 15:47:48.189042 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:47:48.189018    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d0b7dd62-c5e6-4689-a0d5-dd41a15f6e96-plugins-config-volume\") pod \"payload-processing-e2e-mt-81f538c6-7546b749d5-s45bb\" (UID: \"d0b7dd62-c5e6-4689-a0d5-dd41a15f6e96\") " pod="openshift-ingress/payload-processing-e2e-mt-81f538c6-7546b749d5-s45bb"
Sep 10 15:47:48.198263 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:47:48.198220    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mq7t7\" (UniqueName: \"kubernetes.io/projected/d0b7dd62-c5e6-4689-a0d5-dd41a15f6e96-kube-api-access-mq7t7\") pod \"payload-processing-e2e-mt-81f538c6-7546b749d5-s45bb\" (UID: \"d0b7dd62-c5e6-4689-a0d5-dd41a15f6e96\") " pod="openshift-ingress/payload-processing-e2e-mt-81f538c6-7546b749d5-s45bb"
Sep 10 15:47:48.278430 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:47:48.278398    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:47:48.290427 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:47:48.290401    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-81f538c6-7546b749d5-s45bb"
Sep 10 15:47:48.569740 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:47:48.569710    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-81f538c6-7546b749d5-s45bb"]
Sep 10 15:47:48.570633 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:47:48.570609    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd0b7dd62_c5e6_4689_a0d5_dd41a15f6e96.slice/crio-61141fed7fac09dc45a37a4871692f9a7237f826be539a81319c1db4a539b7c3 WatchSource:0}: Error finding container 61141fed7fac09dc45a37a4871692f9a7237f826be539a81319c1db4a539b7c3: Status 404 returned error can't find the container with id 61141fed7fac09dc45a37a4871692f9a7237f826be539a81319c1db4a539b7c3
Sep 10 15:47:48.579970 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:47:48.579945    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-81f538c6-7546b749d5-s45bb" event={"ID":"d0b7dd62-c5e6-4689-a0d5-dd41a15f6e96","Type":"ContainerStarted","Data":"61141fed7fac09dc45a37a4871692f9a7237f826be539a81319c1db4a539b7c3"}
Sep 10 15:47:49.110947 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:47:49.110899    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:47:49.586509 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:47:49.586475    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-81f538c6-7546b749d5-s45bb" event={"ID":"d0b7dd62-c5e6-4689-a0d5-dd41a15f6e96","Type":"ContainerStarted","Data":"cdf8ff889ab5ac91d2f7e435e99ddf7a3b1be601081e34f17a54503beb579974"}
Sep 10 15:47:49.586895 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:47:49.586553    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-mt-81f538c6-7546b749d5-s45bb"
Sep 10 15:47:49.609828 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:47:49.609778    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-mt-81f538c6-7546b749d5-s45bb" podStartSLOduration=2.6097618320000002 podStartE2EDuration="2.609761832s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 15:47:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 15:47:49.607796819 +0000 UTC m=+1255.176028562" watchObservedRunningTime="2026-09-10 15:47:49.609761832 +0000 UTC m=+1255.177993561"
Sep 10 15:47:59.156789 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:47:59.156752    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:48:00.593629 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:48:00.593600    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-mt-81f538c6-7546b749d5-s45bb"
Sep 10 15:48:09.455519 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:48:09.455485    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:48:10.361505 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:48:10.361462    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:48:11.095180 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:48:11.095145    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:48:11.906022 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:48:11.905982    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:48:13.006695 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:48:13.006660    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:48:13.466093 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:48:13.466060    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:48:13.852348 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:48:13.852253    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:48:17.205830 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:48:17.205792    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-mt-81f538c6/e2e-lifecycle-model-81f538c6-kserve-96c8b68b7-4ggk5"]
Sep 10 15:48:17.213772 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:48:17.213743    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-81f538c6/e2e-lifecycle-model-81f538c6-kserve-96c8b68b7-4ggk5"
Sep 10 15:48:17.216339 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:48:17.216312    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-81f538c6\"/\"kube-root-ca.crt\""
Sep 10 15:48:17.217266 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:48:17.217224    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-81f538c6\"/\"default-dockercfg-2847g\""
Sep 10 15:48:17.217419 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:48:17.217397    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-81f538c6\"/\"e2e-lifecycle-model-81f538c6-kserve-self-signed-certs\""
Sep 10 15:48:17.217501 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:48:17.217251    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-81f538c6\"/\"openshift-service-ca.crt\""
Sep 10 15:48:17.225098 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:48:17.225071    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-81f538c6/e2e-lifecycle-model-81f538c6-kserve-96c8b68b7-4ggk5"]
Sep 10 15:48:17.247427 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:48:17.247391    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:48:17.373087 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:48:17.373042    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b8642e38-0a2a-4fdf-b0b2-7eaef169d1f5-home\") pod \"e2e-lifecycle-model-81f538c6-kserve-96c8b68b7-4ggk5\" (UID: \"b8642e38-0a2a-4fdf-b0b2-7eaef169d1f5\") " pod="ai-tenant-e2e-mt-81f538c6/e2e-lifecycle-model-81f538c6-kserve-96c8b68b7-4ggk5"
Sep 10 15:48:17.373087 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:48:17.373099    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b8642e38-0a2a-4fdf-b0b2-7eaef169d1f5-tls-certs\") pod \"e2e-lifecycle-model-81f538c6-kserve-96c8b68b7-4ggk5\" (UID: \"b8642e38-0a2a-4fdf-b0b2-7eaef169d1f5\") " pod="ai-tenant-e2e-mt-81f538c6/e2e-lifecycle-model-81f538c6-kserve-96c8b68b7-4ggk5"
Sep 10 15:48:17.373403 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:48:17.373183    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b8642e38-0a2a-4fdf-b0b2-7eaef169d1f5-dshm\") pod \"e2e-lifecycle-model-81f538c6-kserve-96c8b68b7-4ggk5\" (UID: \"b8642e38-0a2a-4fdf-b0b2-7eaef169d1f5\") " pod="ai-tenant-e2e-mt-81f538c6/e2e-lifecycle-model-81f538c6-kserve-96c8b68b7-4ggk5"
Sep 10 15:48:17.373403 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:48:17.373318    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p78dr\" (UniqueName: \"kubernetes.io/projected/b8642e38-0a2a-4fdf-b0b2-7eaef169d1f5-kube-api-access-p78dr\") pod \"e2e-lifecycle-model-81f538c6-kserve-96c8b68b7-4ggk5\" (UID: \"b8642e38-0a2a-4fdf-b0b2-7eaef169d1f5\") " pod="ai-tenant-e2e-mt-81f538c6/e2e-lifecycle-model-81f538c6-kserve-96c8b68b7-4ggk5"
Sep 10 15:48:17.373403 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:48:17.373387    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b8642e38-0a2a-4fdf-b0b2-7eaef169d1f5-model-cache\") pod \"e2e-lifecycle-model-81f538c6-kserve-96c8b68b7-4ggk5\" (UID: \"b8642e38-0a2a-4fdf-b0b2-7eaef169d1f5\") " pod="ai-tenant-e2e-mt-81f538c6/e2e-lifecycle-model-81f538c6-kserve-96c8b68b7-4ggk5"
Sep 10 15:48:17.373533 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:48:17.373410    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b8642e38-0a2a-4fdf-b0b2-7eaef169d1f5-tmp-dir\") pod \"e2e-lifecycle-model-81f538c6-kserve-96c8b68b7-4ggk5\" (UID: \"b8642e38-0a2a-4fdf-b0b2-7eaef169d1f5\") " pod="ai-tenant-e2e-mt-81f538c6/e2e-lifecycle-model-81f538c6-kserve-96c8b68b7-4ggk5"
Sep 10 15:48:17.474279 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:48:17.474147    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b8642e38-0a2a-4fdf-b0b2-7eaef169d1f5-home\") pod \"e2e-lifecycle-model-81f538c6-kserve-96c8b68b7-4ggk5\" (UID: \"b8642e38-0a2a-4fdf-b0b2-7eaef169d1f5\") " pod="ai-tenant-e2e-mt-81f538c6/e2e-lifecycle-model-81f538c6-kserve-96c8b68b7-4ggk5"
Sep 10 15:48:17.474279 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:48:17.474197    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b8642e38-0a2a-4fdf-b0b2-7eaef169d1f5-tls-certs\") pod \"e2e-lifecycle-model-81f538c6-kserve-96c8b68b7-4ggk5\" (UID: \"b8642e38-0a2a-4fdf-b0b2-7eaef169d1f5\") " pod="ai-tenant-e2e-mt-81f538c6/e2e-lifecycle-model-81f538c6-kserve-96c8b68b7-4ggk5"
Sep 10 15:48:17.474279 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:48:17.474270    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b8642e38-0a2a-4fdf-b0b2-7eaef169d1f5-dshm\") pod \"e2e-lifecycle-model-81f538c6-kserve-96c8b68b7-4ggk5\" (UID: \"b8642e38-0a2a-4fdf-b0b2-7eaef169d1f5\") " pod="ai-tenant-e2e-mt-81f538c6/e2e-lifecycle-model-81f538c6-kserve-96c8b68b7-4ggk5"
Sep 10 15:48:17.474611 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:48:17.474350    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p78dr\" (UniqueName: \"kubernetes.io/projected/b8642e38-0a2a-4fdf-b0b2-7eaef169d1f5-kube-api-access-p78dr\") pod \"e2e-lifecycle-model-81f538c6-kserve-96c8b68b7-4ggk5\" (UID: \"b8642e38-0a2a-4fdf-b0b2-7eaef169d1f5\") " pod="ai-tenant-e2e-mt-81f538c6/e2e-lifecycle-model-81f538c6-kserve-96c8b68b7-4ggk5"
Sep 10 15:48:17.474611 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:48:17.474409    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b8642e38-0a2a-4fdf-b0b2-7eaef169d1f5-model-cache\") pod \"e2e-lifecycle-model-81f538c6-kserve-96c8b68b7-4ggk5\" (UID: \"b8642e38-0a2a-4fdf-b0b2-7eaef169d1f5\") " pod="ai-tenant-e2e-mt-81f538c6/e2e-lifecycle-model-81f538c6-kserve-96c8b68b7-4ggk5"
Sep 10 15:48:17.474611 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:48:17.474434    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b8642e38-0a2a-4fdf-b0b2-7eaef169d1f5-tmp-dir\") pod \"e2e-lifecycle-model-81f538c6-kserve-96c8b68b7-4ggk5\" (UID: \"b8642e38-0a2a-4fdf-b0b2-7eaef169d1f5\") " pod="ai-tenant-e2e-mt-81f538c6/e2e-lifecycle-model-81f538c6-kserve-96c8b68b7-4ggk5"
Sep 10 15:48:17.474813 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:48:17.474734    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b8642e38-0a2a-4fdf-b0b2-7eaef169d1f5-home\") pod \"e2e-lifecycle-model-81f538c6-kserve-96c8b68b7-4ggk5\" (UID: \"b8642e38-0a2a-4fdf-b0b2-7eaef169d1f5\") " pod="ai-tenant-e2e-mt-81f538c6/e2e-lifecycle-model-81f538c6-kserve-96c8b68b7-4ggk5"
Sep 10 15:48:17.474945 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:48:17.474921    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b8642e38-0a2a-4fdf-b0b2-7eaef169d1f5-model-cache\") pod \"e2e-lifecycle-model-81f538c6-kserve-96c8b68b7-4ggk5\" (UID: \"b8642e38-0a2a-4fdf-b0b2-7eaef169d1f5\") " pod="ai-tenant-e2e-mt-81f538c6/e2e-lifecycle-model-81f538c6-kserve-96c8b68b7-4ggk5"
Sep 10 15:48:17.475043 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:48:17.474996    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b8642e38-0a2a-4fdf-b0b2-7eaef169d1f5-tmp-dir\") pod \"e2e-lifecycle-model-81f538c6-kserve-96c8b68b7-4ggk5\" (UID: \"b8642e38-0a2a-4fdf-b0b2-7eaef169d1f5\") " pod="ai-tenant-e2e-mt-81f538c6/e2e-lifecycle-model-81f538c6-kserve-96c8b68b7-4ggk5"
Sep 10 15:48:17.476997 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:48:17.476970    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b8642e38-0a2a-4fdf-b0b2-7eaef169d1f5-dshm\") pod \"e2e-lifecycle-model-81f538c6-kserve-96c8b68b7-4ggk5\" (UID: \"b8642e38-0a2a-4fdf-b0b2-7eaef169d1f5\") " pod="ai-tenant-e2e-mt-81f538c6/e2e-lifecycle-model-81f538c6-kserve-96c8b68b7-4ggk5"
Sep 10 15:48:17.477272 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:48:17.477248    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b8642e38-0a2a-4fdf-b0b2-7eaef169d1f5-tls-certs\") pod \"e2e-lifecycle-model-81f538c6-kserve-96c8b68b7-4ggk5\" (UID: \"b8642e38-0a2a-4fdf-b0b2-7eaef169d1f5\") " pod="ai-tenant-e2e-mt-81f538c6/e2e-lifecycle-model-81f538c6-kserve-96c8b68b7-4ggk5"
Sep 10 15:48:17.483613 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:48:17.483586    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p78dr\" (UniqueName: \"kubernetes.io/projected/b8642e38-0a2a-4fdf-b0b2-7eaef169d1f5-kube-api-access-p78dr\") pod \"e2e-lifecycle-model-81f538c6-kserve-96c8b68b7-4ggk5\" (UID: \"b8642e38-0a2a-4fdf-b0b2-7eaef169d1f5\") " pod="ai-tenant-e2e-mt-81f538c6/e2e-lifecycle-model-81f538c6-kserve-96c8b68b7-4ggk5"
Sep 10 15:48:17.525529 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:48:17.525492    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-81f538c6/e2e-lifecycle-model-81f538c6-kserve-96c8b68b7-4ggk5"
Sep 10 15:48:17.799663 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:48:17.799637    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-81f538c6/e2e-lifecycle-model-81f538c6-kserve-96c8b68b7-4ggk5"]
Sep 10 15:48:17.801799 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:48:17.801765    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb8642e38_0a2a_4fdf_b0b2_7eaef169d1f5.slice/crio-4642db5b8a458962963ab84d0ab168813868aad0b63ecc4f7c39732f77577f1b WatchSource:0}: Error finding container 4642db5b8a458962963ab84d0ab168813868aad0b63ecc4f7c39732f77577f1b: Status 404 returned error can't find the container with id 4642db5b8a458962963ab84d0ab168813868aad0b63ecc4f7c39732f77577f1b
Sep 10 15:48:18.733071 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:48:18.733020    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-81f538c6/e2e-lifecycle-model-81f538c6-kserve-96c8b68b7-4ggk5" event={"ID":"b8642e38-0a2a-4fdf-b0b2-7eaef169d1f5","Type":"ContainerStarted","Data":"4005bc3308ca91d73e5f103465ea3bdb78bdf5583604e3bbedabfc908968d57c"}
Sep 10 15:48:18.733480 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:48:18.733081    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-81f538c6/e2e-lifecycle-model-81f538c6-kserve-96c8b68b7-4ggk5" event={"ID":"b8642e38-0a2a-4fdf-b0b2-7eaef169d1f5","Type":"ContainerStarted","Data":"4642db5b8a458962963ab84d0ab168813868aad0b63ecc4f7c39732f77577f1b"}
Sep 10 15:48:18.751562 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:48:18.751511    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-mt-81f538c6/e2e-lifecycle-model-81f538c6-kserve-96c8b68b7-4ggk5" podStartSLOduration=1.751493494 podStartE2EDuration="1.751493494s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 15:48:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 15:48:18.749555376 +0000 UTC m=+1284.317787108" watchObservedRunningTime="2026-09-10 15:48:18.751493494 +0000 UTC m=+1284.319725224"
Sep 10 15:48:27.526606 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:48:27.526560    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-81f538c6/e2e-lifecycle-model-81f538c6-kserve-96c8b68b7-4ggk5"
Sep 10 15:48:27.526606 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:48:27.526618    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-mt-81f538c6/e2e-lifecycle-model-81f538c6-kserve-96c8b68b7-4ggk5"
Sep 10 15:48:27.539726 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:48:27.539699    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-mt-81f538c6/e2e-lifecycle-model-81f538c6-kserve-96c8b68b7-4ggk5"
Sep 10 15:48:27.787414 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:48:27.787320    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-81f538c6/e2e-lifecycle-model-81f538c6-kserve-96c8b68b7-4ggk5"
Sep 10 15:48:58.883436 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:48:58.883400    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:48:59.830136 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:48:59.830100    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:49:00.700981 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:49:00.700949    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:49:01.468996 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:49:01.468947    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:49:02.303219 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:49:02.303160    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:49:03.050694 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:49:03.050652    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:49:13.776921 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:49:13.776882    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:49:13.809111 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:49:13.809062    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-81f538c6-7546b749d5-s45bb"]
Sep 10 15:49:13.809400 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:49:13.809339    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-mt-81f538c6-7546b749d5-s45bb" podUID="d0b7dd62-c5e6-4689-a0d5-dd41a15f6e96" containerName="payload-processing" containerID="cri-o://cdf8ff889ab5ac91d2f7e435e99ddf7a3b1be601081e34f17a54503beb579974" gracePeriod=30
Sep 10 15:49:13.875366 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:49:13.875332    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:49:13.998384 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:49:13.998346    2567 generic.go:356] "Generic (PLEG): container finished" podID="d0b7dd62-c5e6-4689-a0d5-dd41a15f6e96" containerID="cdf8ff889ab5ac91d2f7e435e99ddf7a3b1be601081e34f17a54503beb579974" exitCode=0
Sep 10 15:49:13.998528 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:49:13.998409    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-81f538c6-7546b749d5-s45bb" event={"ID":"d0b7dd62-c5e6-4689-a0d5-dd41a15f6e96","Type":"ContainerDied","Data":"cdf8ff889ab5ac91d2f7e435e99ddf7a3b1be601081e34f17a54503beb579974"}
Sep 10 15:49:14.069323 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:49:14.069290    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-81f538c6-7546b749d5-s45bb"
Sep 10 15:49:14.114449 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:49:14.114419    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mq7t7\" (UniqueName: \"kubernetes.io/projected/d0b7dd62-c5e6-4689-a0d5-dd41a15f6e96-kube-api-access-mq7t7\") pod \"d0b7dd62-c5e6-4689-a0d5-dd41a15f6e96\" (UID: \"d0b7dd62-c5e6-4689-a0d5-dd41a15f6e96\") "
Sep 10 15:49:14.114635 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:49:14.114457    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d0b7dd62-c5e6-4689-a0d5-dd41a15f6e96-plugins-config-volume\") pod \"d0b7dd62-c5e6-4689-a0d5-dd41a15f6e96\" (UID: \"d0b7dd62-c5e6-4689-a0d5-dd41a15f6e96\") "
Sep 10 15:49:14.114868 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:49:14.114836    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d0b7dd62-c5e6-4689-a0d5-dd41a15f6e96-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "d0b7dd62-c5e6-4689-a0d5-dd41a15f6e96" (UID: "d0b7dd62-c5e6-4689-a0d5-dd41a15f6e96"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 15:49:14.116568 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:49:14.116542    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d0b7dd62-c5e6-4689-a0d5-dd41a15f6e96-kube-api-access-mq7t7" (OuterVolumeSpecName: "kube-api-access-mq7t7") pod "d0b7dd62-c5e6-4689-a0d5-dd41a15f6e96" (UID: "d0b7dd62-c5e6-4689-a0d5-dd41a15f6e96"). InnerVolumeSpecName "kube-api-access-mq7t7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 15:49:14.148979 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:49:14.148920    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:49:14.216146 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:49:14.216094    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mq7t7\" (UniqueName: \"kubernetes.io/projected/d0b7dd62-c5e6-4689-a0d5-dd41a15f6e96-kube-api-access-mq7t7\") on node \"ip-10-0-128-241.ec2.internal\" DevicePath \"\""
Sep 10 15:49:14.216146 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:49:14.216120    2567 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d0b7dd62-c5e6-4689-a0d5-dd41a15f6e96-plugins-config-volume\") on node \"ip-10-0-128-241.ec2.internal\" DevicePath \"\""
Sep 10 15:49:14.240932 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:49:14.240884    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:49:14.417748 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:49:14.417642    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:49:15.005077 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:49:15.005047    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-81f538c6-7546b749d5-s45bb"
Sep 10 15:49:15.005077 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:49:15.005055    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-81f538c6-7546b749d5-s45bb" event={"ID":"d0b7dd62-c5e6-4689-a0d5-dd41a15f6e96","Type":"ContainerDied","Data":"61141fed7fac09dc45a37a4871692f9a7237f826be539a81319c1db4a539b7c3"}
Sep 10 15:49:15.005613 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:49:15.005111    2567 scope.go:117] "RemoveContainer" containerID="cdf8ff889ab5ac91d2f7e435e99ddf7a3b1be601081e34f17a54503beb579974"
Sep 10 15:49:15.025603 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:49:15.025572    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-81f538c6-7546b749d5-s45bb"]
Sep 10 15:49:15.030004 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:49:15.029975    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-81f538c6-7546b749d5-s45bb"]
Sep 10 15:49:15.388556 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:49:15.388520    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:49:16.250175 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:49:16.250139    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:49:16.904434 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:49:16.904397    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d0b7dd62-c5e6-4689-a0d5-dd41a15f6e96" path="/var/lib/kubelet/pods/d0b7dd62-c5e6-4689-a0d5-dd41a15f6e96/volumes"
Sep 10 15:49:17.119111 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:49:17.119032    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:49:17.224910 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:49:17.224875    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:49:18.053782 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:49:18.053751    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:49:22.195677 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:49:22.195645    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-81f538c6/e2e-lifecycle-model-81f538c6-kserve-96c8b68b7-4ggk5"]
Sep 10 15:49:22.290722 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:49:22.290688    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:49:37.215350 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:49:37.215303    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-mt-81f538c6/e2e-lifecycle-model-81f538c6-kserve-96c8b68b7-4ggk5" podUID="b8642e38-0a2a-4fdf-b0b2-7eaef169d1f5" containerName="main" containerID="cri-o://4005bc3308ca91d73e5f103465ea3bdb78bdf5583604e3bbedabfc908968d57c" gracePeriod=45
Sep 10 15:49:37.467905 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:49:37.467841    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-81f538c6/e2e-lifecycle-model-81f538c6-kserve-96c8b68b7-4ggk5"
Sep 10 15:49:37.633097 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:49:37.633063    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b8642e38-0a2a-4fdf-b0b2-7eaef169d1f5-model-cache\") pod \"b8642e38-0a2a-4fdf-b0b2-7eaef169d1f5\" (UID: \"b8642e38-0a2a-4fdf-b0b2-7eaef169d1f5\") "
Sep 10 15:49:37.633298 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:49:37.633132    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b8642e38-0a2a-4fdf-b0b2-7eaef169d1f5-home\") pod \"b8642e38-0a2a-4fdf-b0b2-7eaef169d1f5\" (UID: \"b8642e38-0a2a-4fdf-b0b2-7eaef169d1f5\") "
Sep 10 15:49:37.633298 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:49:37.633156    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b8642e38-0a2a-4fdf-b0b2-7eaef169d1f5-dshm\") pod \"b8642e38-0a2a-4fdf-b0b2-7eaef169d1f5\" (UID: \"b8642e38-0a2a-4fdf-b0b2-7eaef169d1f5\") "
Sep 10 15:49:37.633298 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:49:37.633183    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b8642e38-0a2a-4fdf-b0b2-7eaef169d1f5-tls-certs\") pod \"b8642e38-0a2a-4fdf-b0b2-7eaef169d1f5\" (UID: \"b8642e38-0a2a-4fdf-b0b2-7eaef169d1f5\") "
Sep 10 15:49:37.633298 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:49:37.633205    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p78dr\" (UniqueName: \"kubernetes.io/projected/b8642e38-0a2a-4fdf-b0b2-7eaef169d1f5-kube-api-access-p78dr\") pod \"b8642e38-0a2a-4fdf-b0b2-7eaef169d1f5\" (UID: \"b8642e38-0a2a-4fdf-b0b2-7eaef169d1f5\") "
Sep 10 15:49:37.633298 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:49:37.633268    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b8642e38-0a2a-4fdf-b0b2-7eaef169d1f5-tmp-dir\") pod \"b8642e38-0a2a-4fdf-b0b2-7eaef169d1f5\" (UID: \"b8642e38-0a2a-4fdf-b0b2-7eaef169d1f5\") "
Sep 10 15:49:37.633584 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:49:37.633408    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b8642e38-0a2a-4fdf-b0b2-7eaef169d1f5-model-cache" (OuterVolumeSpecName: "model-cache") pod "b8642e38-0a2a-4fdf-b0b2-7eaef169d1f5" (UID: "b8642e38-0a2a-4fdf-b0b2-7eaef169d1f5"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 15:49:37.633584 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:49:37.633466    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b8642e38-0a2a-4fdf-b0b2-7eaef169d1f5-home" (OuterVolumeSpecName: "home") pod "b8642e38-0a2a-4fdf-b0b2-7eaef169d1f5" (UID: "b8642e38-0a2a-4fdf-b0b2-7eaef169d1f5"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 15:49:37.633584 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:49:37.633571    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b8642e38-0a2a-4fdf-b0b2-7eaef169d1f5-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "b8642e38-0a2a-4fdf-b0b2-7eaef169d1f5" (UID: "b8642e38-0a2a-4fdf-b0b2-7eaef169d1f5"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 15:49:37.633716 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:49:37.633661    2567 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b8642e38-0a2a-4fdf-b0b2-7eaef169d1f5-model-cache\") on node \"ip-10-0-128-241.ec2.internal\" DevicePath \"\""
Sep 10 15:49:37.633716 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:49:37.633675    2567 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b8642e38-0a2a-4fdf-b0b2-7eaef169d1f5-home\") on node \"ip-10-0-128-241.ec2.internal\" DevicePath \"\""
Sep 10 15:49:37.633716 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:49:37.633683    2567 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b8642e38-0a2a-4fdf-b0b2-7eaef169d1f5-tmp-dir\") on node \"ip-10-0-128-241.ec2.internal\" DevicePath \"\""
Sep 10 15:49:37.635489 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:49:37.635466    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b8642e38-0a2a-4fdf-b0b2-7eaef169d1f5-kube-api-access-p78dr" (OuterVolumeSpecName: "kube-api-access-p78dr") pod "b8642e38-0a2a-4fdf-b0b2-7eaef169d1f5" (UID: "b8642e38-0a2a-4fdf-b0b2-7eaef169d1f5"). InnerVolumeSpecName "kube-api-access-p78dr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 15:49:37.635737 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:49:37.635717    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b8642e38-0a2a-4fdf-b0b2-7eaef169d1f5-dshm" (OuterVolumeSpecName: "dshm") pod "b8642e38-0a2a-4fdf-b0b2-7eaef169d1f5" (UID: "b8642e38-0a2a-4fdf-b0b2-7eaef169d1f5"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 15:49:37.635796 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:49:37.635733    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b8642e38-0a2a-4fdf-b0b2-7eaef169d1f5-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "b8642e38-0a2a-4fdf-b0b2-7eaef169d1f5" (UID: "b8642e38-0a2a-4fdf-b0b2-7eaef169d1f5"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 15:49:37.734657 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:49:37.734556    2567 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b8642e38-0a2a-4fdf-b0b2-7eaef169d1f5-dshm\") on node \"ip-10-0-128-241.ec2.internal\" DevicePath \"\""
Sep 10 15:49:37.734657 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:49:37.734594    2567 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b8642e38-0a2a-4fdf-b0b2-7eaef169d1f5-tls-certs\") on node \"ip-10-0-128-241.ec2.internal\" DevicePath \"\""
Sep 10 15:49:37.734657 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:49:37.734607    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p78dr\" (UniqueName: \"kubernetes.io/projected/b8642e38-0a2a-4fdf-b0b2-7eaef169d1f5-kube-api-access-p78dr\") on node \"ip-10-0-128-241.ec2.internal\" DevicePath \"\""
Sep 10 15:49:38.121536 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:49:38.121445    2567 generic.go:356] "Generic (PLEG): container finished" podID="b8642e38-0a2a-4fdf-b0b2-7eaef169d1f5" containerID="4005bc3308ca91d73e5f103465ea3bdb78bdf5583604e3bbedabfc908968d57c" exitCode=0
Sep 10 15:49:38.121536 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:49:38.121490    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-81f538c6/e2e-lifecycle-model-81f538c6-kserve-96c8b68b7-4ggk5"
Sep 10 15:49:38.121726 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:49:38.121492    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-81f538c6/e2e-lifecycle-model-81f538c6-kserve-96c8b68b7-4ggk5" event={"ID":"b8642e38-0a2a-4fdf-b0b2-7eaef169d1f5","Type":"ContainerDied","Data":"4005bc3308ca91d73e5f103465ea3bdb78bdf5583604e3bbedabfc908968d57c"}
Sep 10 15:49:38.121726 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:49:38.121600    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-81f538c6/e2e-lifecycle-model-81f538c6-kserve-96c8b68b7-4ggk5" event={"ID":"b8642e38-0a2a-4fdf-b0b2-7eaef169d1f5","Type":"ContainerDied","Data":"4642db5b8a458962963ab84d0ab168813868aad0b63ecc4f7c39732f77577f1b"}
Sep 10 15:49:38.121726 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:49:38.121617    2567 scope.go:117] "RemoveContainer" containerID="4005bc3308ca91d73e5f103465ea3bdb78bdf5583604e3bbedabfc908968d57c"
Sep 10 15:49:38.130716 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:49:38.130698    2567 scope.go:117] "RemoveContainer" containerID="4005bc3308ca91d73e5f103465ea3bdb78bdf5583604e3bbedabfc908968d57c"
Sep 10 15:49:38.130979 ip-10-0-128-241 kubenswrapper[2567]: E0910 15:49:38.130961    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4005bc3308ca91d73e5f103465ea3bdb78bdf5583604e3bbedabfc908968d57c\": container with ID starting with 4005bc3308ca91d73e5f103465ea3bdb78bdf5583604e3bbedabfc908968d57c not found: ID does not exist" containerID="4005bc3308ca91d73e5f103465ea3bdb78bdf5583604e3bbedabfc908968d57c"
Sep 10 15:49:38.131018 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:49:38.130989    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4005bc3308ca91d73e5f103465ea3bdb78bdf5583604e3bbedabfc908968d57c"} err="failed to get container status \"4005bc3308ca91d73e5f103465ea3bdb78bdf5583604e3bbedabfc908968d57c\": rpc error: code = NotFound desc = could not find container \"4005bc3308ca91d73e5f103465ea3bdb78bdf5583604e3bbedabfc908968d57c\": container with ID starting with 4005bc3308ca91d73e5f103465ea3bdb78bdf5583604e3bbedabfc908968d57c not found: ID does not exist"
Sep 10 15:49:38.160250 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:49:38.160201    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-81f538c6/e2e-lifecycle-model-81f538c6-kserve-96c8b68b7-4ggk5"]
Sep 10 15:49:38.166634 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:49:38.166606    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-mt-81f538c6/e2e-lifecycle-model-81f538c6-kserve-96c8b68b7-4ggk5"]
Sep 10 15:49:38.904267 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:49:38.904220    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b8642e38-0a2a-4fdf-b0b2-7eaef169d1f5" path="/var/lib/kubelet/pods/b8642e38-0a2a-4fdf-b0b2-7eaef169d1f5/volumes"
Sep 10 15:49:46.679465 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:49:46.679432    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:49:57.421706 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:49:57.421671    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:49:57.481335 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:49:57.481300    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:49:57.542799 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:49:57.542763    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:50:00.172176 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:50:00.172142    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:50:00.586189 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:50:00.586054    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:50:06.982189 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:50:06.982139    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:50:12.750428 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:50:12.750390    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:50:12.849840 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:50:12.849808    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:50:13.024629 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:50:13.024537    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:50:14.029747 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:50:14.029713    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7752-0b45de"]
Sep 10 15:50:14.030183 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:50:14.030148    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d0b7dd62-c5e6-4689-a0d5-dd41a15f6e96" containerName="payload-processing"
Sep 10 15:50:14.030183 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:50:14.030163    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="d0b7dd62-c5e6-4689-a0d5-dd41a15f6e96" containerName="payload-processing"
Sep 10 15:50:14.030310 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:50:14.030195    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b8642e38-0a2a-4fdf-b0b2-7eaef169d1f5" containerName="main"
Sep 10 15:50:14.030310 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:50:14.030201    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="b8642e38-0a2a-4fdf-b0b2-7eaef169d1f5" containerName="main"
Sep 10 15:50:14.030310 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:50:14.030289    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="b8642e38-0a2a-4fdf-b0b2-7eaef169d1f5" containerName="main"
Sep 10 15:50:14.030310 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:50:14.030300    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="d0b7dd62-c5e6-4689-a0d5-dd41a15f6e96" containerName="payload-processing"
Sep 10 15:50:14.035059 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:50:14.035042    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7752-0b45de"
Sep 10 15:50:14.038174 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:50:14.038153    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-wtngs\""
Sep 10 15:50:14.039286 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:50:14.039261    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7752-0b45de"]
Sep 10 15:50:14.192125 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:50:14.192081    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zvd2q\" (UniqueName: \"kubernetes.io/projected/86b5ac55-1d5a-4892-b8ad-d76703e442d0-kube-api-access-zvd2q\") pod \"test-curl-7752-0b45de\" (UID: \"86b5ac55-1d5a-4892-b8ad-d76703e442d0\") " pod="opendatahub/test-curl-7752-0b45de"
Sep 10 15:50:14.283862 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:50:14.283758    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:50:14.293001 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:50:14.292967    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zvd2q\" (UniqueName: \"kubernetes.io/projected/86b5ac55-1d5a-4892-b8ad-d76703e442d0-kube-api-access-zvd2q\") pod \"test-curl-7752-0b45de\" (UID: \"86b5ac55-1d5a-4892-b8ad-d76703e442d0\") " pod="opendatahub/test-curl-7752-0b45de"
Sep 10 15:50:14.301868 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:50:14.301837    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zvd2q\" (UniqueName: \"kubernetes.io/projected/86b5ac55-1d5a-4892-b8ad-d76703e442d0-kube-api-access-zvd2q\") pod \"test-curl-7752-0b45de\" (UID: \"86b5ac55-1d5a-4892-b8ad-d76703e442d0\") " pod="opendatahub/test-curl-7752-0b45de"
Sep 10 15:50:14.346318 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:50:14.346280    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7752-0b45de"
Sep 10 15:50:14.379618 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:50:14.379590    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:50:14.480492 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:50:14.480457    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:50:14.617198 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:50:14.617058    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7752-0b45de"]
Sep 10 15:50:14.619457 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:50:14.619428    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod86b5ac55_1d5a_4892_b8ad_d76703e442d0.slice/crio-4573927068cf8f015bf63a080c3b6676459ef0d7ebfdaa1c369d0abf120615d3 WatchSource:0}: Error finding container 4573927068cf8f015bf63a080c3b6676459ef0d7ebfdaa1c369d0abf120615d3: Status 404 returned error can't find the container with id 4573927068cf8f015bf63a080c3b6676459ef0d7ebfdaa1c369d0abf120615d3
Sep 10 15:50:15.081962 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:50:15.081929    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:50:15.187535 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:50:15.187501    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:50:15.283304 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:50:15.283268    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:50:15.296419 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:50:15.296387    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7752-0b45de" event={"ID":"86b5ac55-1d5a-4892-b8ad-d76703e442d0","Type":"ContainerStarted","Data":"d62a88d352d6472d9420fe1eb92d281d63e2b057e9efca554c43db7fc9355500"}
Sep 10 15:50:15.296589 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:50:15.296434    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7752-0b45de" event={"ID":"86b5ac55-1d5a-4892-b8ad-d76703e442d0","Type":"ContainerStarted","Data":"4573927068cf8f015bf63a080c3b6676459ef0d7ebfdaa1c369d0abf120615d3"}
Sep 10 15:50:15.315057 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:50:15.315003    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7752-0b45de" podStartSLOduration=1.3149856149999999 podStartE2EDuration="1.314985615s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 15:50:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 15:50:15.313320005 +0000 UTC m=+1400.881551748" watchObservedRunningTime="2026-09-10 15:50:15.314985615 +0000 UTC m=+1400.883217375"
Sep 10 15:50:15.925766 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:50:15.925732    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7752-0b45de"]
Sep 10 15:50:15.931546 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:50:15.931517    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7752-0b45de"]
Sep 10 15:50:16.301527 ip-10-0-128-241 kubenswrapper[2567]: E0910 15:50:16.301449    2567 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7752-0b45de" container="test-curl-7752-0b45de"
Sep 10 15:50:16.566423 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:50:16.566343    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:50:17.308171 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:50:17.308130    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7752-0b45de" podUID="86b5ac55-1d5a-4892-b8ad-d76703e442d0" containerName="test-curl-7752-0b45de" containerID="cri-o://d62a88d352d6472d9420fe1eb92d281d63e2b057e9efca554c43db7fc9355500" gracePeriod=2
Sep 10 15:50:17.470726 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:50:17.470691    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:50:18.307734 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:50:18.307698    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:50:19.491587 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:50:19.491564    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7752-0b45de"
Sep 10 15:50:19.643329 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:50:19.643221    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zvd2q\" (UniqueName: \"kubernetes.io/projected/86b5ac55-1d5a-4892-b8ad-d76703e442d0-kube-api-access-zvd2q\") pod \"86b5ac55-1d5a-4892-b8ad-d76703e442d0\" (UID: \"86b5ac55-1d5a-4892-b8ad-d76703e442d0\") "
Sep 10 15:50:19.645358 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:50:19.645321    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/86b5ac55-1d5a-4892-b8ad-d76703e442d0-kube-api-access-zvd2q" (OuterVolumeSpecName: "kube-api-access-zvd2q") pod "86b5ac55-1d5a-4892-b8ad-d76703e442d0" (UID: "86b5ac55-1d5a-4892-b8ad-d76703e442d0"). InnerVolumeSpecName "kube-api-access-zvd2q". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 15:50:19.744487 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:50:19.744450    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zvd2q\" (UniqueName: \"kubernetes.io/projected/86b5ac55-1d5a-4892-b8ad-d76703e442d0-kube-api-access-zvd2q\") on node \"ip-10-0-128-241.ec2.internal\" DevicePath \"\""
Sep 10 15:50:20.323910 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:50:20.323873    2567 generic.go:356] "Generic (PLEG): container finished" podID="86b5ac55-1d5a-4892-b8ad-d76703e442d0" containerID="d62a88d352d6472d9420fe1eb92d281d63e2b057e9efca554c43db7fc9355500" exitCode=137
Sep 10 15:50:20.324096 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:50:20.323917    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7752-0b45de"
Sep 10 15:50:20.324096 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:50:20.323976    2567 scope.go:117] "RemoveContainer" containerID="d62a88d352d6472d9420fe1eb92d281d63e2b057e9efca554c43db7fc9355500"
Sep 10 15:50:20.333032 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:50:20.333015    2567 scope.go:117] "RemoveContainer" containerID="d62a88d352d6472d9420fe1eb92d281d63e2b057e9efca554c43db7fc9355500"
Sep 10 15:50:20.333349 ip-10-0-128-241 kubenswrapper[2567]: E0910 15:50:20.333322    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d62a88d352d6472d9420fe1eb92d281d63e2b057e9efca554c43db7fc9355500\": container with ID starting with d62a88d352d6472d9420fe1eb92d281d63e2b057e9efca554c43db7fc9355500 not found: ID does not exist" containerID="d62a88d352d6472d9420fe1eb92d281d63e2b057e9efca554c43db7fc9355500"
Sep 10 15:50:20.333425 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:50:20.333359    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d62a88d352d6472d9420fe1eb92d281d63e2b057e9efca554c43db7fc9355500"} err="failed to get container status \"d62a88d352d6472d9420fe1eb92d281d63e2b057e9efca554c43db7fc9355500\": rpc error: code = NotFound desc = could not find container \"d62a88d352d6472d9420fe1eb92d281d63e2b057e9efca554c43db7fc9355500\": container with ID starting with d62a88d352d6472d9420fe1eb92d281d63e2b057e9efca554c43db7fc9355500 not found: ID does not exist"
Sep 10 15:50:20.334470 ip-10-0-128-241 kubenswrapper[2567]: E0910 15:50:20.334445    2567 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7752-0b45de\" is forbidden: User \"system:node:ip-10-0-128-241.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-128-241.ec2.internal' and this object" podUID="86b5ac55-1d5a-4892-b8ad-d76703e442d0" pod="opendatahub/test-curl-7752-0b45de"
Sep 10 15:50:20.903869 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:50:20.903831    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="86b5ac55-1d5a-4892-b8ad-d76703e442d0" path="/var/lib/kubelet/pods/86b5ac55-1d5a-4892-b8ad-d76703e442d0/volumes"
Sep 10 15:50:21.287758 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:50:21.287720    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:50:22.679501 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:50:22.679459    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:50:22.784346 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:50:22.784313    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:50:23.333350 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:50:23.333314    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:50:29.918000 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:50:29.917956    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:50:30.020742 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:50:30.020705    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:50:30.123358 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:50:30.123326    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:50:33.855218 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:50:33.855187    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:50:37.115581 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:50:37.115541    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:50:40.649663 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:50:40.649627    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:50:40.883013 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:50:40.882980    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:51:01.827138 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:51:01.827051    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:51:07.733101 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:51:07.733070    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:51:09.212147 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:51:09.212109    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:51:09.294475 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:51:09.294442    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:51:09.567427 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:51:09.567319    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:51:09.909553 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:51:09.909466    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:51:10.417473 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:51:10.417442    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:51:12.040006 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:51:12.039968    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:51:12.647569 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:51:12.647525    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:51:12.742307 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:51:12.742272    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:51:12.977348 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:51:12.977314    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:51:13.907009 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:51:13.906954    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:51:14.754844 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:51:14.754813    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:51:15.648617 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:51:15.648582    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:51:15.870625 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:51:15.870588    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:51:16.438794 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:51:16.438758    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-1c1464-7b9c69898f-h4fw7"]
Sep 10 15:51:16.439204 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:51:16.439186    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="86b5ac55-1d5a-4892-b8ad-d76703e442d0" containerName="test-curl-7752-0b45de"
Sep 10 15:51:16.439204 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:51:16.439205    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="86b5ac55-1d5a-4892-b8ad-d76703e442d0" containerName="test-curl-7752-0b45de"
Sep 10 15:51:16.439405 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:51:16.439300    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="86b5ac55-1d5a-4892-b8ad-d76703e442d0" containerName="test-curl-7752-0b45de"
Sep 10 15:51:16.443429 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:51:16.443396    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-1c1464-7b9c69898f-h4fw7"
Sep 10 15:51:16.445761 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:51:16.445734    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-a-1c1464-serving-cert\""
Sep 10 15:51:16.452606 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:51:16.452578    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-1c1464-7b9c69898f-h4fw7"]
Sep 10 15:51:16.467574 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:51:16.467545    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:51:16.567875 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:51:16.567829    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3837fc27-5440-4736-995c-6e0f12ba4033-openshift-service-ca\") pod \"maas-api-e2e-api-a-1c1464-7b9c69898f-h4fw7\" (UID: \"3837fc27-5440-4736-995c-6e0f12ba4033\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-1c1464-7b9c69898f-h4fw7"
Sep 10 15:51:16.568077 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:51:16.567928    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t58vp\" (UniqueName: \"kubernetes.io/projected/3837fc27-5440-4736-995c-6e0f12ba4033-kube-api-access-t58vp\") pod \"maas-api-e2e-api-a-1c1464-7b9c69898f-h4fw7\" (UID: \"3837fc27-5440-4736-995c-6e0f12ba4033\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-1c1464-7b9c69898f-h4fw7"
Sep 10 15:51:16.568077 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:51:16.567985    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3837fc27-5440-4736-995c-6e0f12ba4033-maas-api-tls\") pod \"maas-api-e2e-api-a-1c1464-7b9c69898f-h4fw7\" (UID: \"3837fc27-5440-4736-995c-6e0f12ba4033\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-1c1464-7b9c69898f-h4fw7"
Sep 10 15:51:16.568203 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:51:16.568163    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3837fc27-5440-4736-995c-6e0f12ba4033-metrics-certs\") pod \"maas-api-e2e-api-a-1c1464-7b9c69898f-h4fw7\" (UID: \"3837fc27-5440-4736-995c-6e0f12ba4033\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-1c1464-7b9c69898f-h4fw7"
Sep 10 15:51:16.568594 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:51:16.568563    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:51:16.669411 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:51:16.669368    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3837fc27-5440-4736-995c-6e0f12ba4033-metrics-certs\") pod \"maas-api-e2e-api-a-1c1464-7b9c69898f-h4fw7\" (UID: \"3837fc27-5440-4736-995c-6e0f12ba4033\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-1c1464-7b9c69898f-h4fw7"
Sep 10 15:51:16.669411 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:51:16.669415    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3837fc27-5440-4736-995c-6e0f12ba4033-openshift-service-ca\") pod \"maas-api-e2e-api-a-1c1464-7b9c69898f-h4fw7\" (UID: \"3837fc27-5440-4736-995c-6e0f12ba4033\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-1c1464-7b9c69898f-h4fw7"
Sep 10 15:51:16.669857 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:51:16.669454    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t58vp\" (UniqueName: \"kubernetes.io/projected/3837fc27-5440-4736-995c-6e0f12ba4033-kube-api-access-t58vp\") pod \"maas-api-e2e-api-a-1c1464-7b9c69898f-h4fw7\" (UID: \"3837fc27-5440-4736-995c-6e0f12ba4033\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-1c1464-7b9c69898f-h4fw7"
Sep 10 15:51:16.669857 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:51:16.669496    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3837fc27-5440-4736-995c-6e0f12ba4033-maas-api-tls\") pod \"maas-api-e2e-api-a-1c1464-7b9c69898f-h4fw7\" (UID: \"3837fc27-5440-4736-995c-6e0f12ba4033\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-1c1464-7b9c69898f-h4fw7"
Sep 10 15:51:16.669857 ip-10-0-128-241 kubenswrapper[2567]: E0910 15:51:16.669639    2567 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-a-1c1464-serving-cert: secret "maas-api-e2e-api-a-1c1464-serving-cert" not found
Sep 10 15:51:16.669857 ip-10-0-128-241 kubenswrapper[2567]: E0910 15:51:16.669737    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3837fc27-5440-4736-995c-6e0f12ba4033-maas-api-tls podName:3837fc27-5440-4736-995c-6e0f12ba4033 nodeName:}" failed. No retries permitted until 2026-09-10 15:51:17.169715703 +0000 UTC m=+1462.737947430 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/3837fc27-5440-4736-995c-6e0f12ba4033-maas-api-tls") pod "maas-api-e2e-api-a-1c1464-7b9c69898f-h4fw7" (UID: "3837fc27-5440-4736-995c-6e0f12ba4033") : secret "maas-api-e2e-api-a-1c1464-serving-cert" not found
Sep 10 15:51:16.670306 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:51:16.670278    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3837fc27-5440-4736-995c-6e0f12ba4033-openshift-service-ca\") pod \"maas-api-e2e-api-a-1c1464-7b9c69898f-h4fw7\" (UID: \"3837fc27-5440-4736-995c-6e0f12ba4033\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-1c1464-7b9c69898f-h4fw7"
Sep 10 15:51:16.671812 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:51:16.671794    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3837fc27-5440-4736-995c-6e0f12ba4033-metrics-certs\") pod \"maas-api-e2e-api-a-1c1464-7b9c69898f-h4fw7\" (UID: \"3837fc27-5440-4736-995c-6e0f12ba4033\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-1c1464-7b9c69898f-h4fw7"
Sep 10 15:51:16.678582 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:51:16.678555    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t58vp\" (UniqueName: \"kubernetes.io/projected/3837fc27-5440-4736-995c-6e0f12ba4033-kube-api-access-t58vp\") pod \"maas-api-e2e-api-a-1c1464-7b9c69898f-h4fw7\" (UID: \"3837fc27-5440-4736-995c-6e0f12ba4033\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-1c1464-7b9c69898f-h4fw7"
Sep 10 15:51:16.821961 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:51:16.821858    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:51:16.984763 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:51:16.984717    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:51:17.077621 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:51:17.077526    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:51:17.160898 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:51:17.160867    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-1c1464-7866cc94fd-rp5nd"]
Sep 10 15:51:17.164691 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:51:17.164672    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-1c1464-7866cc94fd-rp5nd"
Sep 10 15:51:17.167211 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:51:17.167186    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-a-1c1464\""
Sep 10 15:51:17.167368 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:51:17.167261    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-a-1c1464-dockercfg-z5ht2\""
Sep 10 15:51:17.174674 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:51:17.174643    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-1c1464-7866cc94fd-rp5nd"]
Sep 10 15:51:17.175030 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:51:17.175008    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3837fc27-5440-4736-995c-6e0f12ba4033-maas-api-tls\") pod \"maas-api-e2e-api-a-1c1464-7b9c69898f-h4fw7\" (UID: \"3837fc27-5440-4736-995c-6e0f12ba4033\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-1c1464-7b9c69898f-h4fw7"
Sep 10 15:51:17.177679 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:51:17.177656    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3837fc27-5440-4736-995c-6e0f12ba4033-maas-api-tls\") pod \"maas-api-e2e-api-a-1c1464-7b9c69898f-h4fw7\" (UID: \"3837fc27-5440-4736-995c-6e0f12ba4033\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-1c1464-7b9c69898f-h4fw7"
Sep 10 15:51:17.276008 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:51:17.275965    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zzqlr\" (UniqueName: \"kubernetes.io/projected/f21e844c-809e-4b7b-a675-b12ad2accd4f-kube-api-access-zzqlr\") pod \"payload-pre-processing-e2e-api-a-1c1464-7866cc94fd-rp5nd\" (UID: \"f21e844c-809e-4b7b-a675-b12ad2accd4f\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-1c1464-7866cc94fd-rp5nd"
Sep 10 15:51:17.276197 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:51:17.276020    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f21e844c-809e-4b7b-a675-b12ad2accd4f-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-1c1464-7866cc94fd-rp5nd\" (UID: \"f21e844c-809e-4b7b-a675-b12ad2accd4f\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-1c1464-7866cc94fd-rp5nd"
Sep 10 15:51:17.356153 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:51:17.356072    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-1c1464-7b9c69898f-h4fw7"
Sep 10 15:51:17.377431 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:51:17.377396    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zzqlr\" (UniqueName: \"kubernetes.io/projected/f21e844c-809e-4b7b-a675-b12ad2accd4f-kube-api-access-zzqlr\") pod \"payload-pre-processing-e2e-api-a-1c1464-7866cc94fd-rp5nd\" (UID: \"f21e844c-809e-4b7b-a675-b12ad2accd4f\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-1c1464-7866cc94fd-rp5nd"
Sep 10 15:51:17.377596 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:51:17.377453    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f21e844c-809e-4b7b-a675-b12ad2accd4f-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-1c1464-7866cc94fd-rp5nd\" (UID: \"f21e844c-809e-4b7b-a675-b12ad2accd4f\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-1c1464-7866cc94fd-rp5nd"
Sep 10 15:51:17.378302 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:51:17.378276    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f21e844c-809e-4b7b-a675-b12ad2accd4f-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-1c1464-7866cc94fd-rp5nd\" (UID: \"f21e844c-809e-4b7b-a675-b12ad2accd4f\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-1c1464-7866cc94fd-rp5nd"
Sep 10 15:51:17.386207 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:51:17.386182    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zzqlr\" (UniqueName: \"kubernetes.io/projected/f21e844c-809e-4b7b-a675-b12ad2accd4f-kube-api-access-zzqlr\") pod \"payload-pre-processing-e2e-api-a-1c1464-7866cc94fd-rp5nd\" (UID: \"f21e844c-809e-4b7b-a675-b12ad2accd4f\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-1c1464-7866cc94fd-rp5nd"
Sep 10 15:51:17.476489 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:51:17.476445    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-1c1464-7866cc94fd-rp5nd"
Sep 10 15:51:17.629982 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:51:17.629957    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-1c1464-7b9c69898f-h4fw7"]
Sep 10 15:51:17.634813 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:51:17.634584    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3837fc27_5440_4736_995c_6e0f12ba4033.slice/crio-8d19c374523ceebbcb52256af069da3e870013c3a1dc1b63d491a1ddd21b6232 WatchSource:0}: Error finding container 8d19c374523ceebbcb52256af069da3e870013c3a1dc1b63d491a1ddd21b6232: Status 404 returned error can't find the container with id 8d19c374523ceebbcb52256af069da3e870013c3a1dc1b63d491a1ddd21b6232
Sep 10 15:51:17.635745 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:51:17.635712    2567 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 10 15:51:17.752046 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:51:17.752017    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-1c1464-7866cc94fd-rp5nd"]
Sep 10 15:51:17.754874 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:51:17.754850    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf21e844c_809e_4b7b_a675_b12ad2accd4f.slice/crio-2525864f254b272bf615406980c72ce98bb7a7f0fa368dea3faeef40561c688e WatchSource:0}: Error finding container 2525864f254b272bf615406980c72ce98bb7a7f0fa368dea3faeef40561c688e: Status 404 returned error can't find the container with id 2525864f254b272bf615406980c72ce98bb7a7f0fa368dea3faeef40561c688e
Sep 10 15:51:18.607150 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:51:18.607111    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-1c1464-7866cc94fd-rp5nd" event={"ID":"f21e844c-809e-4b7b-a675-b12ad2accd4f","Type":"ContainerStarted","Data":"6f44efd43344fbaf61d150a550a6dbd3162eba211242d8bcbc84cb22956dff41"}
Sep 10 15:51:18.607377 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:51:18.607193    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-1c1464-7866cc94fd-rp5nd" event={"ID":"f21e844c-809e-4b7b-a675-b12ad2accd4f","Type":"ContainerStarted","Data":"2525864f254b272bf615406980c72ce98bb7a7f0fa368dea3faeef40561c688e"}
Sep 10 15:51:18.607377 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:51:18.607256    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-1c1464-7866cc94fd-rp5nd"
Sep 10 15:51:18.608912 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:51:18.608886    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-1c1464-7b9c69898f-h4fw7" event={"ID":"3837fc27-5440-4736-995c-6e0f12ba4033","Type":"ContainerStarted","Data":"d8d236286f1de37f59267ddf42a0c38a1f026f544d90b57cb7fff13d28666798"}
Sep 10 15:51:18.609072 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:51:18.608936    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-1c1464-7b9c69898f-h4fw7" event={"ID":"3837fc27-5440-4736-995c-6e0f12ba4033","Type":"ContainerStarted","Data":"8d19c374523ceebbcb52256af069da3e870013c3a1dc1b63d491a1ddd21b6232"}
Sep 10 15:51:18.609072 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:51:18.608996    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-1c1464-7b9c69898f-h4fw7"
Sep 10 15:51:18.628692 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:51:18.628637    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-api-a-1c1464-7866cc94fd-rp5nd" podStartSLOduration=1.628622613 podStartE2EDuration="1.628622613s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 15:51:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 15:51:18.626640242 +0000 UTC m=+1464.194871973" watchObservedRunningTime="2026-09-10 15:51:18.628622613 +0000 UTC m=+1464.196854347"
Sep 10 15:51:18.644429 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:51:18.644372    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-1c1464-7b9c69898f-h4fw7" podStartSLOduration=2.165844543 podStartE2EDuration="2.644353616s" totalImagesPullingTime="478.509073ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 15:51:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 15:51:18.643765926 +0000 UTC m=+1464.211997658" watchObservedRunningTime="2026-09-10 15:51:18.644353616 +0000 UTC m=+1464.212585365"
Sep 10 15:51:24.619843 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:51:24.619812    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-1c1464-7b9c69898f-h4fw7"
Sep 10 15:51:29.616599 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:51:29.616566    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-1c1464-7866cc94fd-rp5nd"
Sep 10 15:51:38.658530 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:51:38.658494    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:51:39.627378 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:51:39.627346    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:51:40.673274 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:51:40.673201    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:51:41.238537 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:51:41.238502    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:51:41.952106 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:51:41.952069    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:51:42.174811 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:51:42.174750    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:51:42.723162 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:51:42.723115    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:51:43.053965 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:51:43.053853    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:51:43.182456 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:51:43.182401    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:51:43.496763 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:51:43.496729    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:51:43.821466 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:51:43.821369    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:51:45.841709 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:51:45.841675    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:51:46.525492 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:51:46.525457    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:51:46.650135 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:51:46.650102    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:51:46.811541 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:51:46.811456    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:51:46.905068 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:51:46.905029    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:51:47.807521 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:51:47.807485    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:51:48.614486 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:51:48.614428    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:51:49.404148 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:51:49.404106    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:51:50.297572 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:51:50.297526    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:51:51.267376 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:51:51.267335    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:51:51.367487 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:51:51.367452    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:51:51.734673 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:51:51.734636    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:51:51.834638 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:51:51.834608    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:51:51.913220 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:51:51.913183    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-59c481-78ff4666df-jn22f"]
Sep 10 15:51:51.917211 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:51:51.917188    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-59c481-78ff4666df-jn22f"
Sep 10 15:51:51.920168 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:51:51.920133    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-b-59c481\""
Sep 10 15:51:51.920357 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:51:51.920331    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-b-59c481-dockercfg-bbwld\""
Sep 10 15:51:51.928313 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:51:51.928286    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:51:51.929373 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:51:51.929353    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-59c481-78ff4666df-jn22f"]
Sep 10 15:51:51.995330 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:51:51.995210    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bgf59\" (UniqueName: \"kubernetes.io/projected/2d52e90c-ab6d-47ee-a2ba-dc945af6d473-kube-api-access-bgf59\") pod \"payload-processing-e2e-api-b-59c481-78ff4666df-jn22f\" (UID: \"2d52e90c-ab6d-47ee-a2ba-dc945af6d473\") " pod="openshift-ingress/payload-processing-e2e-api-b-59c481-78ff4666df-jn22f"
Sep 10 15:51:51.995330 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:51:51.995321    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2d52e90c-ab6d-47ee-a2ba-dc945af6d473-plugins-config-volume\") pod \"payload-processing-e2e-api-b-59c481-78ff4666df-jn22f\" (UID: \"2d52e90c-ab6d-47ee-a2ba-dc945af6d473\") " pod="openshift-ingress/payload-processing-e2e-api-b-59c481-78ff4666df-jn22f"
Sep 10 15:51:52.096197 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:51:52.096163    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bgf59\" (UniqueName: \"kubernetes.io/projected/2d52e90c-ab6d-47ee-a2ba-dc945af6d473-kube-api-access-bgf59\") pod \"payload-processing-e2e-api-b-59c481-78ff4666df-jn22f\" (UID: \"2d52e90c-ab6d-47ee-a2ba-dc945af6d473\") " pod="openshift-ingress/payload-processing-e2e-api-b-59c481-78ff4666df-jn22f"
Sep 10 15:51:52.096394 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:51:52.096225    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2d52e90c-ab6d-47ee-a2ba-dc945af6d473-plugins-config-volume\") pod \"payload-processing-e2e-api-b-59c481-78ff4666df-jn22f\" (UID: \"2d52e90c-ab6d-47ee-a2ba-dc945af6d473\") " pod="openshift-ingress/payload-processing-e2e-api-b-59c481-78ff4666df-jn22f"
Sep 10 15:51:52.096959 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:51:52.096933    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2d52e90c-ab6d-47ee-a2ba-dc945af6d473-plugins-config-volume\") pod \"payload-processing-e2e-api-b-59c481-78ff4666df-jn22f\" (UID: \"2d52e90c-ab6d-47ee-a2ba-dc945af6d473\") " pod="openshift-ingress/payload-processing-e2e-api-b-59c481-78ff4666df-jn22f"
Sep 10 15:51:52.106302 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:51:52.106276    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bgf59\" (UniqueName: \"kubernetes.io/projected/2d52e90c-ab6d-47ee-a2ba-dc945af6d473-kube-api-access-bgf59\") pod \"payload-processing-e2e-api-b-59c481-78ff4666df-jn22f\" (UID: \"2d52e90c-ab6d-47ee-a2ba-dc945af6d473\") " pod="openshift-ingress/payload-processing-e2e-api-b-59c481-78ff4666df-jn22f"
Sep 10 15:51:52.229224 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:51:52.229183    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-59c481-78ff4666df-jn22f"
Sep 10 15:51:52.502443 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:51:52.502289    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-59c481-78ff4666df-jn22f"]
Sep 10 15:51:52.505690 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:51:52.505662    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2d52e90c_ab6d_47ee_a2ba_dc945af6d473.slice/crio-6d2e9e60f59992568551c0f342acab1f26e06d512b89c71d7dec2ae2ce6fa974 WatchSource:0}: Error finding container 6d2e9e60f59992568551c0f342acab1f26e06d512b89c71d7dec2ae2ce6fa974: Status 404 returned error can't find the container with id 6d2e9e60f59992568551c0f342acab1f26e06d512b89c71d7dec2ae2ce6fa974
Sep 10 15:51:52.575034 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:51:52.575003    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:51:52.782852 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:51:52.782811    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-59c481-78ff4666df-jn22f" event={"ID":"2d52e90c-ab6d-47ee-a2ba-dc945af6d473","Type":"ContainerStarted","Data":"777b6e2a8d2e8bfc35d6e21811bafe4db998ca01b6735023afcce6eafd561f48"}
Sep 10 15:51:52.783058 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:51:52.782879    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-59c481-78ff4666df-jn22f" event={"ID":"2d52e90c-ab6d-47ee-a2ba-dc945af6d473","Type":"ContainerStarted","Data":"6d2e9e60f59992568551c0f342acab1f26e06d512b89c71d7dec2ae2ce6fa974"}
Sep 10 15:51:52.783058 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:51:52.782907    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-b-59c481-78ff4666df-jn22f"
Sep 10 15:51:52.812359 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:51:52.812222    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-b-59c481-78ff4666df-jn22f" podStartSLOduration=1.8122013 podStartE2EDuration="1.8122013s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 15:51:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 15:51:52.809329431 +0000 UTC m=+1498.377561167" watchObservedRunningTime="2026-09-10 15:51:52.8122013 +0000 UTC m=+1498.380433033"
Sep 10 15:51:53.180549 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:51:53.180508    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7752-758a1e"]
Sep 10 15:51:53.184921 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:51:53.184897    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7752-758a1e"
Sep 10 15:51:53.189264 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:51:53.189014    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-wtngs\""
Sep 10 15:51:53.191638 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:51:53.191606    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7752-758a1e"]
Sep 10 15:51:53.309728 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:51:53.309686    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7p5dt\" (UniqueName: \"kubernetes.io/projected/3a123a37-77d5-45fb-8ba3-f20e331b9575-kube-api-access-7p5dt\") pod \"test-curl-7752-758a1e\" (UID: \"3a123a37-77d5-45fb-8ba3-f20e331b9575\") " pod="opendatahub/test-curl-7752-758a1e"
Sep 10 15:51:53.410598 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:51:53.410559    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7p5dt\" (UniqueName: \"kubernetes.io/projected/3a123a37-77d5-45fb-8ba3-f20e331b9575-kube-api-access-7p5dt\") pod \"test-curl-7752-758a1e\" (UID: \"3a123a37-77d5-45fb-8ba3-f20e331b9575\") " pod="opendatahub/test-curl-7752-758a1e"
Sep 10 15:51:53.421529 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:51:53.421497    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7p5dt\" (UniqueName: \"kubernetes.io/projected/3a123a37-77d5-45fb-8ba3-f20e331b9575-kube-api-access-7p5dt\") pod \"test-curl-7752-758a1e\" (UID: \"3a123a37-77d5-45fb-8ba3-f20e331b9575\") " pod="opendatahub/test-curl-7752-758a1e"
Sep 10 15:51:53.497602 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:51:53.497503    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7752-758a1e"
Sep 10 15:51:53.772576 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:51:53.772534    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7752-758a1e"]
Sep 10 15:51:53.775375 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:51:53.775347    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3a123a37_77d5_45fb_8ba3_f20e331b9575.slice/crio-4993790d30353caa08dde50d86d40729a7c6384f8a8e97d8cb63412c9cc17cd1 WatchSource:0}: Error finding container 4993790d30353caa08dde50d86d40729a7c6384f8a8e97d8cb63412c9cc17cd1: Status 404 returned error can't find the container with id 4993790d30353caa08dde50d86d40729a7c6384f8a8e97d8cb63412c9cc17cd1
Sep 10 15:51:53.788129 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:51:53.788100    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7752-758a1e" event={"ID":"3a123a37-77d5-45fb-8ba3-f20e331b9575","Type":"ContainerStarted","Data":"4993790d30353caa08dde50d86d40729a7c6384f8a8e97d8cb63412c9cc17cd1"}
Sep 10 15:51:54.795057 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:51:54.795025    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7752-758a1e" event={"ID":"3a123a37-77d5-45fb-8ba3-f20e331b9575","Type":"ContainerStarted","Data":"4d42d9160c65d5481929967c7c96f15a3f7791ce13bc59dce67f131f16ad890f"}
Sep 10 15:51:54.811119 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:51:54.811069    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7752-758a1e" podStartSLOduration=1.8110542459999999 podStartE2EDuration="1.811054246s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 15:51:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 15:51:54.809078555 +0000 UTC m=+1500.377310288" watchObservedRunningTime="2026-09-10 15:51:54.811054246 +0000 UTC m=+1500.379285978"
Sep 10 15:51:54.945076 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:51:54.945049    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-t96pm_0b9d2cda-73e3-4e7c-9e17-1a3d903d95cf/console-operator/2.log"
Sep 10 15:51:54.949844 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:51:54.949811    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-t96pm_0b9d2cda-73e3-4e7c-9e17-1a3d903d95cf/console-operator/2.log"
Sep 10 15:51:55.396481 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:51:55.396442    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7752-758a1e"]
Sep 10 15:51:55.398847 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:51:55.398823    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7752-758a1e"]
Sep 10 15:51:55.508657 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:51:55.508568    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7752-59c482"]
Sep 10 15:51:55.509006 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:51:55.508992    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3a123a37-77d5-45fb-8ba3-f20e331b9575" containerName="test-curl-7752-758a1e"
Sep 10 15:51:55.509006 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:51:55.509006    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="3a123a37-77d5-45fb-8ba3-f20e331b9575" containerName="test-curl-7752-758a1e"
Sep 10 15:51:55.509110 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:51:55.509071    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="3a123a37-77d5-45fb-8ba3-f20e331b9575" containerName="test-curl-7752-758a1e"
Sep 10 15:51:55.512104 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:51:55.512087    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7752-59c482"
Sep 10 15:51:55.516173 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:51:55.516142    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7752-59c482"]
Sep 10 15:51:55.634065 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:51:55.634023    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n6cbs\" (UniqueName: \"kubernetes.io/projected/c079d979-0e43-461c-b6e5-f2a99850e50c-kube-api-access-n6cbs\") pod \"test-curl-7752-59c482\" (UID: \"c079d979-0e43-461c-b6e5-f2a99850e50c\") " pod="opendatahub/test-curl-7752-59c482"
Sep 10 15:51:55.735171 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:51:55.735126    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n6cbs\" (UniqueName: \"kubernetes.io/projected/c079d979-0e43-461c-b6e5-f2a99850e50c-kube-api-access-n6cbs\") pod \"test-curl-7752-59c482\" (UID: \"c079d979-0e43-461c-b6e5-f2a99850e50c\") " pod="opendatahub/test-curl-7752-59c482"
Sep 10 15:51:55.744623 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:51:55.744591    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n6cbs\" (UniqueName: \"kubernetes.io/projected/c079d979-0e43-461c-b6e5-f2a99850e50c-kube-api-access-n6cbs\") pod \"test-curl-7752-59c482\" (UID: \"c079d979-0e43-461c-b6e5-f2a99850e50c\") " pod="opendatahub/test-curl-7752-59c482"
Sep 10 15:51:55.800412 ip-10-0-128-241 kubenswrapper[2567]: E0910 15:51:55.800326    2567 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7752-758a1e" container="test-curl-7752-758a1e"
Sep 10 15:51:55.824023 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:51:55.823991    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7752-59c482"
Sep 10 15:51:56.096382 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:51:56.096290    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7752-59c482"]
Sep 10 15:51:56.805954 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:51:56.805917    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7752-59c482" event={"ID":"c079d979-0e43-461c-b6e5-f2a99850e50c","Type":"ContainerStarted","Data":"44782420821e5b0181579bc26cb9de67b604a5f3870401773fd95c9a3c3bb79a"}
Sep 10 15:51:56.806399 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:51:56.805968    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7752-59c482" event={"ID":"c079d979-0e43-461c-b6e5-f2a99850e50c","Type":"ContainerStarted","Data":"3ee7a505d418d7c0ee032e88777c34433b2d3631e9842f391362876a9e8e0510"}
Sep 10 15:51:56.806399 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:51:56.806021    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7752-758a1e" podUID="3a123a37-77d5-45fb-8ba3-f20e331b9575" containerName="test-curl-7752-758a1e" containerID="cri-o://4d42d9160c65d5481929967c7c96f15a3f7791ce13bc59dce67f131f16ad890f" gracePeriod=2
Sep 10 15:51:56.808203 ip-10-0-128-241 kubenswrapper[2567]: E0910 15:51:56.808181    2567 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7752-758a1e\" is forbidden: User \"system:node:ip-10-0-128-241.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-128-241.ec2.internal' and this object" podUID="3a123a37-77d5-45fb-8ba3-f20e331b9575" pod="opendatahub/test-curl-7752-758a1e"
Sep 10 15:51:56.823858 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:51:56.823794    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7752-59c482" podStartSLOduration=1.8237780369999999 podStartE2EDuration="1.823778037s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 15:51:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 15:51:56.822668058 +0000 UTC m=+1502.390899785" watchObservedRunningTime="2026-09-10 15:51:56.823778037 +0000 UTC m=+1502.392009767"
Sep 10 15:51:56.824899 ip-10-0-128-241 kubenswrapper[2567]: E0910 15:51:56.824876    2567 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7752-758a1e\" is forbidden: User \"system:node:ip-10-0-128-241.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-128-241.ec2.internal' and this object" podUID="3a123a37-77d5-45fb-8ba3-f20e331b9575" pod="opendatahub/test-curl-7752-758a1e"
Sep 10 15:51:57.413829 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:51:57.413790    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7752-59c482"]
Sep 10 15:51:57.418815 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:51:57.418776    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7752-59c482"]
Sep 10 15:51:57.811272 ip-10-0-128-241 kubenswrapper[2567]: E0910 15:51:57.811243    2567 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7752-59c482" container="test-curl-7752-59c482"
Sep 10 15:51:58.816430 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:51:58.816395    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7752-59c482" podUID="c079d979-0e43-461c-b6e5-f2a99850e50c" containerName="test-curl-7752-59c482" containerID="cri-o://44782420821e5b0181579bc26cb9de67b604a5f3870401773fd95c9a3c3bb79a" gracePeriod=2
Sep 10 15:51:58.818838 ip-10-0-128-241 kubenswrapper[2567]: E0910 15:51:58.818807    2567 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7752-59c482\" is forbidden: User \"system:node:ip-10-0-128-241.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-128-241.ec2.internal' and this object" podUID="c079d979-0e43-461c-b6e5-f2a99850e50c" pod="opendatahub/test-curl-7752-59c482"
Sep 10 15:51:58.979245 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:51:58.979206    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7752-758a1e"
Sep 10 15:51:59.067481 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:51:59.067394    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7p5dt\" (UniqueName: \"kubernetes.io/projected/3a123a37-77d5-45fb-8ba3-f20e331b9575-kube-api-access-7p5dt\") pod \"3a123a37-77d5-45fb-8ba3-f20e331b9575\" (UID: \"3a123a37-77d5-45fb-8ba3-f20e331b9575\") "
Sep 10 15:51:59.069475 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:51:59.069449    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3a123a37-77d5-45fb-8ba3-f20e331b9575-kube-api-access-7p5dt" (OuterVolumeSpecName: "kube-api-access-7p5dt") pod "3a123a37-77d5-45fb-8ba3-f20e331b9575" (UID: "3a123a37-77d5-45fb-8ba3-f20e331b9575"). InnerVolumeSpecName "kube-api-access-7p5dt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 15:51:59.168612 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:51:59.168578    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7p5dt\" (UniqueName: \"kubernetes.io/projected/3a123a37-77d5-45fb-8ba3-f20e331b9575-kube-api-access-7p5dt\") on node \"ip-10-0-128-241.ec2.internal\" DevicePath \"\""
Sep 10 15:51:59.508668 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:51:59.508634    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:51:59.822823 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:51:59.822735    2567 generic.go:356] "Generic (PLEG): container finished" podID="3a123a37-77d5-45fb-8ba3-f20e331b9575" containerID="4d42d9160c65d5481929967c7c96f15a3f7791ce13bc59dce67f131f16ad890f" exitCode=137
Sep 10 15:51:59.822823 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:51:59.822778    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7752-758a1e"
Sep 10 15:51:59.822823 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:51:59.822818    2567 scope.go:117] "RemoveContainer" containerID="4d42d9160c65d5481929967c7c96f15a3f7791ce13bc59dce67f131f16ad890f"
Sep 10 15:51:59.833420 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:51:59.833396    2567 scope.go:117] "RemoveContainer" containerID="4d42d9160c65d5481929967c7c96f15a3f7791ce13bc59dce67f131f16ad890f"
Sep 10 15:51:59.833785 ip-10-0-128-241 kubenswrapper[2567]: E0910 15:51:59.833758    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4d42d9160c65d5481929967c7c96f15a3f7791ce13bc59dce67f131f16ad890f\": container with ID starting with 4d42d9160c65d5481929967c7c96f15a3f7791ce13bc59dce67f131f16ad890f not found: ID does not exist" containerID="4d42d9160c65d5481929967c7c96f15a3f7791ce13bc59dce67f131f16ad890f"
Sep 10 15:51:59.833876 ip-10-0-128-241 kubenswrapper[2567]: E0910 15:51:59.833777    2567 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7752-758a1e\" is forbidden: User \"system:node:ip-10-0-128-241.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-128-241.ec2.internal' and this object" podUID="3a123a37-77d5-45fb-8ba3-f20e331b9575" pod="opendatahub/test-curl-7752-758a1e"
Sep 10 15:51:59.833876 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:51:59.833818    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4d42d9160c65d5481929967c7c96f15a3f7791ce13bc59dce67f131f16ad890f"} err="failed to get container status \"4d42d9160c65d5481929967c7c96f15a3f7791ce13bc59dce67f131f16ad890f\": rpc error: code = NotFound desc = could not find container \"4d42d9160c65d5481929967c7c96f15a3f7791ce13bc59dce67f131f16ad890f\": container with ID starting with 4d42d9160c65d5481929967c7c96f15a3f7791ce13bc59dce67f131f16ad890f not found: ID does not exist"
Sep 10 15:52:00.411077 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:52:00.411047    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:52:00.905454 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:52:00.905417    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3a123a37-77d5-45fb-8ba3-f20e331b9575" path="/var/lib/kubelet/pods/3a123a37-77d5-45fb-8ba3-f20e331b9575/volumes"
Sep 10 15:52:01.001442 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:52:01.001415    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7752-59c482"
Sep 10 15:52:01.188953 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:52:01.188922    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-n6cbs\" (UniqueName: \"kubernetes.io/projected/c079d979-0e43-461c-b6e5-f2a99850e50c-kube-api-access-n6cbs\") pod \"c079d979-0e43-461c-b6e5-f2a99850e50c\" (UID: \"c079d979-0e43-461c-b6e5-f2a99850e50c\") "
Sep 10 15:52:01.190970 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:52:01.190943    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c079d979-0e43-461c-b6e5-f2a99850e50c-kube-api-access-n6cbs" (OuterVolumeSpecName: "kube-api-access-n6cbs") pod "c079d979-0e43-461c-b6e5-f2a99850e50c" (UID: "c079d979-0e43-461c-b6e5-f2a99850e50c"). InnerVolumeSpecName "kube-api-access-n6cbs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 15:52:01.289933 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:52:01.289886    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n6cbs\" (UniqueName: \"kubernetes.io/projected/c079d979-0e43-461c-b6e5-f2a99850e50c-kube-api-access-n6cbs\") on node \"ip-10-0-128-241.ec2.internal\" DevicePath \"\""
Sep 10 15:52:01.834428 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:52:01.834386    2567 generic.go:356] "Generic (PLEG): container finished" podID="c079d979-0e43-461c-b6e5-f2a99850e50c" containerID="44782420821e5b0181579bc26cb9de67b604a5f3870401773fd95c9a3c3bb79a" exitCode=137
Sep 10 15:52:01.834428 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:52:01.834432    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7752-59c482"
Sep 10 15:52:01.834665 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:52:01.834447    2567 scope.go:117] "RemoveContainer" containerID="44782420821e5b0181579bc26cb9de67b604a5f3870401773fd95c9a3c3bb79a"
Sep 10 15:52:01.844009 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:52:01.843991    2567 scope.go:117] "RemoveContainer" containerID="44782420821e5b0181579bc26cb9de67b604a5f3870401773fd95c9a3c3bb79a"
Sep 10 15:52:01.844364 ip-10-0-128-241 kubenswrapper[2567]: E0910 15:52:01.844344    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"44782420821e5b0181579bc26cb9de67b604a5f3870401773fd95c9a3c3bb79a\": container with ID starting with 44782420821e5b0181579bc26cb9de67b604a5f3870401773fd95c9a3c3bb79a not found: ID does not exist" containerID="44782420821e5b0181579bc26cb9de67b604a5f3870401773fd95c9a3c3bb79a"
Sep 10 15:52:01.844429 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:52:01.844378    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"44782420821e5b0181579bc26cb9de67b604a5f3870401773fd95c9a3c3bb79a"} err="failed to get container status \"44782420821e5b0181579bc26cb9de67b604a5f3870401773fd95c9a3c3bb79a\": rpc error: code = NotFound desc = could not find container \"44782420821e5b0181579bc26cb9de67b604a5f3870401773fd95c9a3c3bb79a\": container with ID starting with 44782420821e5b0181579bc26cb9de67b604a5f3870401773fd95c9a3c3bb79a not found: ID does not exist"
Sep 10 15:52:01.846135 ip-10-0-128-241 kubenswrapper[2567]: E0910 15:52:01.846108    2567 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7752-59c482\" is forbidden: User \"system:node:ip-10-0-128-241.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-128-241.ec2.internal' and this object" podUID="c079d979-0e43-461c-b6e5-f2a99850e50c" pod="opendatahub/test-curl-7752-59c482"
Sep 10 15:52:02.904167 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:52:02.904133    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c079d979-0e43-461c-b6e5-f2a99850e50c" path="/var/lib/kubelet/pods/c079d979-0e43-461c-b6e5-f2a99850e50c/volumes"
Sep 10 15:52:03.791144 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:52:03.791106    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-b-59c481-78ff4666df-jn22f"
Sep 10 15:52:05.092855 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:52:05.092804    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:52:05.695699 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:52:05.695663    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:52:12.343048 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:52:12.342994    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:52:13.706522 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:52:13.706483    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:52:14.577863 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:52:14.577811    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:52:15.558161 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:52:15.558128    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:52:16.444610 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:52:16.444569    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:52:16.636532 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:52:16.636501    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:52:17.282953 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:52:17.282872    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:52:18.164136 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:52:18.164103    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:52:18.956858 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:52:18.956816    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:52:19.925168 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:52:19.925133    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:52:20.838767 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:52:20.838735    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:52:21.644660 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:52:21.644624    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:52:22.436183 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:52:22.436132    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:52:23.267461 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:52:23.267423    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:52:24.133699 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:52:24.133662    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:52:24.912841 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:52:24.912796    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:52:25.035249 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:52:25.035201    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:52:25.821572 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:52:25.821535    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:52:33.326977 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:52:33.326942    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:52:35.145636 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:52:35.145597    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-1c1464-7b9c69898f-h4fw7"]
Sep 10 15:52:35.146064 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:52:35.145868    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-1c1464-7b9c69898f-h4fw7" podUID="3837fc27-5440-4736-995c-6e0f12ba4033" containerName="maas-api" containerID="cri-o://d8d236286f1de37f59267ddf42a0c38a1f026f544d90b57cb7fff13d28666798" gracePeriod=30
Sep 10 15:52:35.182389 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:52:35.182357    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:52:35.238279 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:52:35.231912    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-1c1464-7866cc94fd-rp5nd"]
Sep 10 15:52:35.238279 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:52:35.232257    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-api-a-1c1464-7866cc94fd-rp5nd" podUID="f21e844c-809e-4b7b-a675-b12ad2accd4f" containerName="payload-pre-processing" containerID="cri-o://6f44efd43344fbaf61d150a550a6dbd3162eba211242d8bcbc84cb22956dff41" gracePeriod=30
Sep 10 15:52:35.274567 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:52:35.274529    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:52:35.340134 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:52:35.340099    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:52:35.422357 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:52:35.422323    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-1c1464-7b9c69898f-h4fw7"
Sep 10 15:52:35.443057 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:52:35.442966    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:52:35.501958 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:52:35.501932    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-1c1464-7866cc94fd-rp5nd"
Sep 10 15:52:35.515589 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:52:35.515558    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:52:35.518731 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:52:35.518709    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3837fc27-5440-4736-995c-6e0f12ba4033-maas-api-tls\") pod \"3837fc27-5440-4736-995c-6e0f12ba4033\" (UID: \"3837fc27-5440-4736-995c-6e0f12ba4033\") "
Sep 10 15:52:35.518878 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:52:35.518743    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-t58vp\" (UniqueName: \"kubernetes.io/projected/3837fc27-5440-4736-995c-6e0f12ba4033-kube-api-access-t58vp\") pod \"3837fc27-5440-4736-995c-6e0f12ba4033\" (UID: \"3837fc27-5440-4736-995c-6e0f12ba4033\") "
Sep 10 15:52:35.519077 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:52:35.519012    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3837fc27-5440-4736-995c-6e0f12ba4033-openshift-service-ca\") pod \"3837fc27-5440-4736-995c-6e0f12ba4033\" (UID: \"3837fc27-5440-4736-995c-6e0f12ba4033\") "
Sep 10 15:52:35.519740 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:52:35.519162    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3837fc27-5440-4736-995c-6e0f12ba4033-metrics-certs\") pod \"3837fc27-5440-4736-995c-6e0f12ba4033\" (UID: \"3837fc27-5440-4736-995c-6e0f12ba4033\") "
Sep 10 15:52:35.519740 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:52:35.519669    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3837fc27-5440-4736-995c-6e0f12ba4033-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "3837fc27-5440-4736-995c-6e0f12ba4033" (UID: "3837fc27-5440-4736-995c-6e0f12ba4033"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 15:52:35.521687 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:52:35.521645    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3837fc27-5440-4736-995c-6e0f12ba4033-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "3837fc27-5440-4736-995c-6e0f12ba4033" (UID: "3837fc27-5440-4736-995c-6e0f12ba4033"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 15:52:35.522498 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:52:35.522428    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3837fc27-5440-4736-995c-6e0f12ba4033-kube-api-access-t58vp" (OuterVolumeSpecName: "kube-api-access-t58vp") pod "3837fc27-5440-4736-995c-6e0f12ba4033" (UID: "3837fc27-5440-4736-995c-6e0f12ba4033"). InnerVolumeSpecName "kube-api-access-t58vp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 15:52:35.523419 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:52:35.523390    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3837fc27-5440-4736-995c-6e0f12ba4033-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "3837fc27-5440-4736-995c-6e0f12ba4033" (UID: "3837fc27-5440-4736-995c-6e0f12ba4033"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 15:52:35.620435 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:52:35.620391    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f21e844c-809e-4b7b-a675-b12ad2accd4f-plugins-config-volume\") pod \"f21e844c-809e-4b7b-a675-b12ad2accd4f\" (UID: \"f21e844c-809e-4b7b-a675-b12ad2accd4f\") "
Sep 10 15:52:35.620435 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:52:35.620439    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zzqlr\" (UniqueName: \"kubernetes.io/projected/f21e844c-809e-4b7b-a675-b12ad2accd4f-kube-api-access-zzqlr\") pod \"f21e844c-809e-4b7b-a675-b12ad2accd4f\" (UID: \"f21e844c-809e-4b7b-a675-b12ad2accd4f\") "
Sep 10 15:52:35.620765 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:52:35.620748    2567 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3837fc27-5440-4736-995c-6e0f12ba4033-maas-api-tls\") on node \"ip-10-0-128-241.ec2.internal\" DevicePath \"\""
Sep 10 15:52:35.620834 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:52:35.620772    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t58vp\" (UniqueName: \"kubernetes.io/projected/3837fc27-5440-4736-995c-6e0f12ba4033-kube-api-access-t58vp\") on node \"ip-10-0-128-241.ec2.internal\" DevicePath \"\""
Sep 10 15:52:35.620834 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:52:35.620787    2567 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3837fc27-5440-4736-995c-6e0f12ba4033-openshift-service-ca\") on node \"ip-10-0-128-241.ec2.internal\" DevicePath \"\""
Sep 10 15:52:35.620834 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:52:35.620786    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f21e844c-809e-4b7b-a675-b12ad2accd4f-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "f21e844c-809e-4b7b-a675-b12ad2accd4f" (UID: "f21e844c-809e-4b7b-a675-b12ad2accd4f"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 15:52:35.620834 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:52:35.620800    2567 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3837fc27-5440-4736-995c-6e0f12ba4033-metrics-certs\") on node \"ip-10-0-128-241.ec2.internal\" DevicePath \"\""
Sep 10 15:52:35.622633 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:52:35.622608    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f21e844c-809e-4b7b-a675-b12ad2accd4f-kube-api-access-zzqlr" (OuterVolumeSpecName: "kube-api-access-zzqlr") pod "f21e844c-809e-4b7b-a675-b12ad2accd4f" (UID: "f21e844c-809e-4b7b-a675-b12ad2accd4f"). InnerVolumeSpecName "kube-api-access-zzqlr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 15:52:35.721897 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:52:35.721857    2567 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f21e844c-809e-4b7b-a675-b12ad2accd4f-plugins-config-volume\") on node \"ip-10-0-128-241.ec2.internal\" DevicePath \"\""
Sep 10 15:52:35.721897 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:52:35.721890    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zzqlr\" (UniqueName: \"kubernetes.io/projected/f21e844c-809e-4b7b-a675-b12ad2accd4f-kube-api-access-zzqlr\") on node \"ip-10-0-128-241.ec2.internal\" DevicePath \"\""
Sep 10 15:52:35.757601 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:52:35.757567    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:52:35.869590 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:52:35.869554    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:52:36.020602 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:52:36.020509    2567 generic.go:356] "Generic (PLEG): container finished" podID="f21e844c-809e-4b7b-a675-b12ad2accd4f" containerID="6f44efd43344fbaf61d150a550a6dbd3162eba211242d8bcbc84cb22956dff41" exitCode=0
Sep 10 15:52:36.020602 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:52:36.020572    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-1c1464-7866cc94fd-rp5nd"
Sep 10 15:52:36.020602 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:52:36.020588    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-1c1464-7866cc94fd-rp5nd" event={"ID":"f21e844c-809e-4b7b-a675-b12ad2accd4f","Type":"ContainerDied","Data":"6f44efd43344fbaf61d150a550a6dbd3162eba211242d8bcbc84cb22956dff41"}
Sep 10 15:52:36.020879 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:52:36.020653    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-1c1464-7866cc94fd-rp5nd" event={"ID":"f21e844c-809e-4b7b-a675-b12ad2accd4f","Type":"ContainerDied","Data":"2525864f254b272bf615406980c72ce98bb7a7f0fa368dea3faeef40561c688e"}
Sep 10 15:52:36.020879 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:52:36.020693    2567 scope.go:117] "RemoveContainer" containerID="6f44efd43344fbaf61d150a550a6dbd3162eba211242d8bcbc84cb22956dff41"
Sep 10 15:52:36.022061 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:52:36.022044    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-1c1464-7b9c69898f-h4fw7"
Sep 10 15:52:36.022149 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:52:36.022061    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-1c1464-7b9c69898f-h4fw7" event={"ID":"3837fc27-5440-4736-995c-6e0f12ba4033","Type":"ContainerDied","Data":"d8d236286f1de37f59267ddf42a0c38a1f026f544d90b57cb7fff13d28666798"}
Sep 10 15:52:36.022149 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:52:36.022042    2567 generic.go:356] "Generic (PLEG): container finished" podID="3837fc27-5440-4736-995c-6e0f12ba4033" containerID="d8d236286f1de37f59267ddf42a0c38a1f026f544d90b57cb7fff13d28666798" exitCode=0
Sep 10 15:52:36.022276 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:52:36.022257    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-1c1464-7b9c69898f-h4fw7" event={"ID":"3837fc27-5440-4736-995c-6e0f12ba4033","Type":"ContainerDied","Data":"8d19c374523ceebbcb52256af069da3e870013c3a1dc1b63d491a1ddd21b6232"}
Sep 10 15:52:36.031088 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:52:36.031065    2567 scope.go:117] "RemoveContainer" containerID="6f44efd43344fbaf61d150a550a6dbd3162eba211242d8bcbc84cb22956dff41"
Sep 10 15:52:36.031423 ip-10-0-128-241 kubenswrapper[2567]: E0910 15:52:36.031403    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6f44efd43344fbaf61d150a550a6dbd3162eba211242d8bcbc84cb22956dff41\": container with ID starting with 6f44efd43344fbaf61d150a550a6dbd3162eba211242d8bcbc84cb22956dff41 not found: ID does not exist" containerID="6f44efd43344fbaf61d150a550a6dbd3162eba211242d8bcbc84cb22956dff41"
Sep 10 15:52:36.031488 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:52:36.031430    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6f44efd43344fbaf61d150a550a6dbd3162eba211242d8bcbc84cb22956dff41"} err="failed to get container status \"6f44efd43344fbaf61d150a550a6dbd3162eba211242d8bcbc84cb22956dff41\": rpc error: code = NotFound desc = could not find container \"6f44efd43344fbaf61d150a550a6dbd3162eba211242d8bcbc84cb22956dff41\": container with ID starting with 6f44efd43344fbaf61d150a550a6dbd3162eba211242d8bcbc84cb22956dff41 not found: ID does not exist"
Sep 10 15:52:36.031488 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:52:36.031448    2567 scope.go:117] "RemoveContainer" containerID="d8d236286f1de37f59267ddf42a0c38a1f026f544d90b57cb7fff13d28666798"
Sep 10 15:52:36.039884 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:52:36.039867    2567 scope.go:117] "RemoveContainer" containerID="d8d236286f1de37f59267ddf42a0c38a1f026f544d90b57cb7fff13d28666798"
Sep 10 15:52:36.040154 ip-10-0-128-241 kubenswrapper[2567]: E0910 15:52:36.040138    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d8d236286f1de37f59267ddf42a0c38a1f026f544d90b57cb7fff13d28666798\": container with ID starting with d8d236286f1de37f59267ddf42a0c38a1f026f544d90b57cb7fff13d28666798 not found: ID does not exist" containerID="d8d236286f1de37f59267ddf42a0c38a1f026f544d90b57cb7fff13d28666798"
Sep 10 15:52:36.040203 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:52:36.040161    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d8d236286f1de37f59267ddf42a0c38a1f026f544d90b57cb7fff13d28666798"} err="failed to get container status \"d8d236286f1de37f59267ddf42a0c38a1f026f544d90b57cb7fff13d28666798\": rpc error: code = NotFound desc = could not find container \"d8d236286f1de37f59267ddf42a0c38a1f026f544d90b57cb7fff13d28666798\": container with ID starting with d8d236286f1de37f59267ddf42a0c38a1f026f544d90b57cb7fff13d28666798 not found: ID does not exist"
Sep 10 15:52:36.044562 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:52:36.044541    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-1c1464-7866cc94fd-rp5nd"]
Sep 10 15:52:36.049338 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:52:36.049309    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-1c1464-7866cc94fd-rp5nd"]
Sep 10 15:52:36.060940 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:52:36.060913    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-1c1464-7b9c69898f-h4fw7"]
Sep 10 15:52:36.065786 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:52:36.065763    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-1c1464-7b9c69898f-h4fw7"]
Sep 10 15:52:36.667764 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:52:36.667728    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:52:36.905209 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:52:36.905176    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3837fc27-5440-4736-995c-6e0f12ba4033" path="/var/lib/kubelet/pods/3837fc27-5440-4736-995c-6e0f12ba4033/volumes"
Sep 10 15:52:36.905620 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:52:36.905604    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f21e844c-809e-4b7b-a675-b12ad2accd4f" path="/var/lib/kubelet/pods/f21e844c-809e-4b7b-a675-b12ad2accd4f/volumes"
Sep 10 15:52:37.517599 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:52:37.517567    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:52:38.461047 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:52:38.461010    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:52:39.511604 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:52:39.511534    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:52:40.267043 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:52:40.267015    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:52:42.239254 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:52:42.239202    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:52:43.749842 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:52:43.749813    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:52:44.055749 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:52:44.055660    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:52:44.796353 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:52:44.796317    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:52:45.556894 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:52:45.556840    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:52:46.460820 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:52:46.460776    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:52:47.249860 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:52:47.249829    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:52:48.139759 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:52:48.139721    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:52:58.397051 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:52:58.397016    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:52:58.425027 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:52:58.424996    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-59c481-78ff4666df-jn22f"]
Sep 10 15:52:58.425287 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:52:58.425263    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-b-59c481-78ff4666df-jn22f" podUID="2d52e90c-ab6d-47ee-a2ba-dc945af6d473" containerName="payload-processing" containerID="cri-o://777b6e2a8d2e8bfc35d6e21811bafe4db998ca01b6735023afcce6eafd561f48" gracePeriod=30
Sep 10 15:52:58.498597 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:52:58.498563    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:52:58.598423 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:52:58.598364    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:52:58.683817 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:52:58.683787    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-59c481-78ff4666df-jn22f"
Sep 10 15:52:58.700332 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:52:58.700298    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:52:58.725880 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:52:58.725851    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2d52e90c-ab6d-47ee-a2ba-dc945af6d473-plugins-config-volume\") pod \"2d52e90c-ab6d-47ee-a2ba-dc945af6d473\" (UID: \"2d52e90c-ab6d-47ee-a2ba-dc945af6d473\") "
Sep 10 15:52:58.726047 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:52:58.725932    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bgf59\" (UniqueName: \"kubernetes.io/projected/2d52e90c-ab6d-47ee-a2ba-dc945af6d473-kube-api-access-bgf59\") pod \"2d52e90c-ab6d-47ee-a2ba-dc945af6d473\" (UID: \"2d52e90c-ab6d-47ee-a2ba-dc945af6d473\") "
Sep 10 15:52:58.726377 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:52:58.726344    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2d52e90c-ab6d-47ee-a2ba-dc945af6d473-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "2d52e90c-ab6d-47ee-a2ba-dc945af6d473" (UID: "2d52e90c-ab6d-47ee-a2ba-dc945af6d473"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 15:52:58.728032 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:52:58.728009    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2d52e90c-ab6d-47ee-a2ba-dc945af6d473-kube-api-access-bgf59" (OuterVolumeSpecName: "kube-api-access-bgf59") pod "2d52e90c-ab6d-47ee-a2ba-dc945af6d473" (UID: "2d52e90c-ab6d-47ee-a2ba-dc945af6d473"). InnerVolumeSpecName "kube-api-access-bgf59". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 15:52:58.827160 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:52:58.827118    2567 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2d52e90c-ab6d-47ee-a2ba-dc945af6d473-plugins-config-volume\") on node \"ip-10-0-128-241.ec2.internal\" DevicePath \"\""
Sep 10 15:52:58.827160 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:52:58.827151    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bgf59\" (UniqueName: \"kubernetes.io/projected/2d52e90c-ab6d-47ee-a2ba-dc945af6d473-kube-api-access-bgf59\") on node \"ip-10-0-128-241.ec2.internal\" DevicePath \"\""
Sep 10 15:52:59.000658 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:52:59.000620    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:52:59.098571 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:52:59.098536    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:52:59.139339 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:52:59.139308    2567 generic.go:356] "Generic (PLEG): container finished" podID="2d52e90c-ab6d-47ee-a2ba-dc945af6d473" containerID="777b6e2a8d2e8bfc35d6e21811bafe4db998ca01b6735023afcce6eafd561f48" exitCode=0
Sep 10 15:52:59.139523 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:52:59.139353    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-59c481-78ff4666df-jn22f"
Sep 10 15:52:59.139523 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:52:59.139407    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-59c481-78ff4666df-jn22f" event={"ID":"2d52e90c-ab6d-47ee-a2ba-dc945af6d473","Type":"ContainerDied","Data":"777b6e2a8d2e8bfc35d6e21811bafe4db998ca01b6735023afcce6eafd561f48"}
Sep 10 15:52:59.139523 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:52:59.139455    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-59c481-78ff4666df-jn22f" event={"ID":"2d52e90c-ab6d-47ee-a2ba-dc945af6d473","Type":"ContainerDied","Data":"6d2e9e60f59992568551c0f342acab1f26e06d512b89c71d7dec2ae2ce6fa974"}
Sep 10 15:52:59.139523 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:52:59.139479    2567 scope.go:117] "RemoveContainer" containerID="777b6e2a8d2e8bfc35d6e21811bafe4db998ca01b6735023afcce6eafd561f48"
Sep 10 15:52:59.148331 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:52:59.148319    2567 scope.go:117] "RemoveContainer" containerID="777b6e2a8d2e8bfc35d6e21811bafe4db998ca01b6735023afcce6eafd561f48"
Sep 10 15:52:59.148650 ip-10-0-128-241 kubenswrapper[2567]: E0910 15:52:59.148628    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"777b6e2a8d2e8bfc35d6e21811bafe4db998ca01b6735023afcce6eafd561f48\": container with ID starting with 777b6e2a8d2e8bfc35d6e21811bafe4db998ca01b6735023afcce6eafd561f48 not found: ID does not exist" containerID="777b6e2a8d2e8bfc35d6e21811bafe4db998ca01b6735023afcce6eafd561f48"
Sep 10 15:52:59.148697 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:52:59.148662    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"777b6e2a8d2e8bfc35d6e21811bafe4db998ca01b6735023afcce6eafd561f48"} err="failed to get container status \"777b6e2a8d2e8bfc35d6e21811bafe4db998ca01b6735023afcce6eafd561f48\": rpc error: code = NotFound desc = could not find container \"777b6e2a8d2e8bfc35d6e21811bafe4db998ca01b6735023afcce6eafd561f48\": container with ID starting with 777b6e2a8d2e8bfc35d6e21811bafe4db998ca01b6735023afcce6eafd561f48 not found: ID does not exist"
Sep 10 15:52:59.159041 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:52:59.159018    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-59c481-78ff4666df-jn22f"]
Sep 10 15:52:59.164172 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:52:59.164154    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-59c481-78ff4666df-jn22f"]
Sep 10 15:53:00.026175 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:53:00.026136    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:53:00.828853 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:53:00.828813    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:53:00.904660 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:53:00.904620    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2d52e90c-ab6d-47ee-a2ba-dc945af6d473" path="/var/lib/kubelet/pods/2d52e90c-ab6d-47ee-a2ba-dc945af6d473/volumes"
Sep 10 15:53:01.651759 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:53:01.651724    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:53:02.407458 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:53:02.407428    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:53:05.679881 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:53:05.679846    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:53:05.792891 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:53:05.792845    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:53:06.003006 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:53:06.002918    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:53:06.563359 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:53:06.563320    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:53:07.661319 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:53:07.661282    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:53:08.451088 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:53:08.451051    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:53:09.415599 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:53:09.415560    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:53:10.239076 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:53:10.239034    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:53:11.014137 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:53:11.014103    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:53:16.833791 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:53:16.833755    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-30d70c-5bbb87697b-sc5pg"]
Sep 10 15:53:16.834309 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:53:16.833956    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-30d70c-5bbb87697b-sc5pg" podUID="52c8b4d6-193b-4ee5-aeed-88a414860ef2" containerName="maas-api" containerID="cri-o://9f428e4d1779e8ca7b6ee9136ffe699132bda326088f24aaa231ef4f3128409e" gracePeriod=30
Sep 10 15:53:16.862708 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:53:16.862647    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:53:16.939541 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:53:16.939511    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:53:17.002918 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:53:17.002883    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:53:17.082929 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:53:17.082905    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-30d70c-5bbb87697b-sc5pg"
Sep 10 15:53:17.107854 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:53:17.107772    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:53:17.198435 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:53:17.198407    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/52c8b4d6-193b-4ee5-aeed-88a414860ef2-metrics-certs\") pod \"52c8b4d6-193b-4ee5-aeed-88a414860ef2\" (UID: \"52c8b4d6-193b-4ee5-aeed-88a414860ef2\") "
Sep 10 15:53:17.198612 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:53:17.198454    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7d5hn\" (UniqueName: \"kubernetes.io/projected/52c8b4d6-193b-4ee5-aeed-88a414860ef2-kube-api-access-7d5hn\") pod \"52c8b4d6-193b-4ee5-aeed-88a414860ef2\" (UID: \"52c8b4d6-193b-4ee5-aeed-88a414860ef2\") "
Sep 10 15:53:17.198612 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:53:17.198526    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/52c8b4d6-193b-4ee5-aeed-88a414860ef2-maas-api-tls\") pod \"52c8b4d6-193b-4ee5-aeed-88a414860ef2\" (UID: \"52c8b4d6-193b-4ee5-aeed-88a414860ef2\") "
Sep 10 15:53:17.198612 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:53:17.198599    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/52c8b4d6-193b-4ee5-aeed-88a414860ef2-openshift-service-ca\") pod \"52c8b4d6-193b-4ee5-aeed-88a414860ef2\" (UID: \"52c8b4d6-193b-4ee5-aeed-88a414860ef2\") "
Sep 10 15:53:17.199013 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:53:17.198981    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/52c8b4d6-193b-4ee5-aeed-88a414860ef2-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "52c8b4d6-193b-4ee5-aeed-88a414860ef2" (UID: "52c8b4d6-193b-4ee5-aeed-88a414860ef2"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 15:53:17.200714 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:53:17.200683    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/52c8b4d6-193b-4ee5-aeed-88a414860ef2-kube-api-access-7d5hn" (OuterVolumeSpecName: "kube-api-access-7d5hn") pod "52c8b4d6-193b-4ee5-aeed-88a414860ef2" (UID: "52c8b4d6-193b-4ee5-aeed-88a414860ef2"). InnerVolumeSpecName "kube-api-access-7d5hn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 15:53:17.201052 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:53:17.201012    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/52c8b4d6-193b-4ee5-aeed-88a414860ef2-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "52c8b4d6-193b-4ee5-aeed-88a414860ef2" (UID: "52c8b4d6-193b-4ee5-aeed-88a414860ef2"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 15:53:17.201148 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:53:17.201121    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/52c8b4d6-193b-4ee5-aeed-88a414860ef2-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "52c8b4d6-193b-4ee5-aeed-88a414860ef2" (UID: "52c8b4d6-193b-4ee5-aeed-88a414860ef2"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 15:53:17.201707 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:53:17.201679    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:53:17.232369 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:53:17.232340    2567 generic.go:356] "Generic (PLEG): container finished" podID="52c8b4d6-193b-4ee5-aeed-88a414860ef2" containerID="9f428e4d1779e8ca7b6ee9136ffe699132bda326088f24aaa231ef4f3128409e" exitCode=0
Sep 10 15:53:17.232548 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:53:17.232390    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-30d70c-5bbb87697b-sc5pg"
Sep 10 15:53:17.232548 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:53:17.232406    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-30d70c-5bbb87697b-sc5pg" event={"ID":"52c8b4d6-193b-4ee5-aeed-88a414860ef2","Type":"ContainerDied","Data":"9f428e4d1779e8ca7b6ee9136ffe699132bda326088f24aaa231ef4f3128409e"}
Sep 10 15:53:17.232548 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:53:17.232449    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-30d70c-5bbb87697b-sc5pg" event={"ID":"52c8b4d6-193b-4ee5-aeed-88a414860ef2","Type":"ContainerDied","Data":"c6ce4f0e43aca9407d20dc2171cce85d2bd81cf9aad3310c624ad9a5fee9d971"}
Sep 10 15:53:17.232548 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:53:17.232472    2567 scope.go:117] "RemoveContainer" containerID="9f428e4d1779e8ca7b6ee9136ffe699132bda326088f24aaa231ef4f3128409e"
Sep 10 15:53:17.242624 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:53:17.242602    2567 scope.go:117] "RemoveContainer" containerID="9f428e4d1779e8ca7b6ee9136ffe699132bda326088f24aaa231ef4f3128409e"
Sep 10 15:53:17.242889 ip-10-0-128-241 kubenswrapper[2567]: E0910 15:53:17.242862    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9f428e4d1779e8ca7b6ee9136ffe699132bda326088f24aaa231ef4f3128409e\": container with ID starting with 9f428e4d1779e8ca7b6ee9136ffe699132bda326088f24aaa231ef4f3128409e not found: ID does not exist" containerID="9f428e4d1779e8ca7b6ee9136ffe699132bda326088f24aaa231ef4f3128409e"
Sep 10 15:53:17.242958 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:53:17.242898    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9f428e4d1779e8ca7b6ee9136ffe699132bda326088f24aaa231ef4f3128409e"} err="failed to get container status \"9f428e4d1779e8ca7b6ee9136ffe699132bda326088f24aaa231ef4f3128409e\": rpc error: code = NotFound desc = could not find container \"9f428e4d1779e8ca7b6ee9136ffe699132bda326088f24aaa231ef4f3128409e\": container with ID starting with 9f428e4d1779e8ca7b6ee9136ffe699132bda326088f24aaa231ef4f3128409e not found: ID does not exist"
Sep 10 15:53:17.257692 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:53:17.257665    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-30d70c-5bbb87697b-sc5pg"]
Sep 10 15:53:17.264466 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:53:17.264439    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-30d70c-5bbb87697b-sc5pg"]
Sep 10 15:53:17.299677 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:53:17.299649    2567 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/52c8b4d6-193b-4ee5-aeed-88a414860ef2-maas-api-tls\") on node \"ip-10-0-128-241.ec2.internal\" DevicePath \"\""
Sep 10 15:53:17.299677 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:53:17.299676    2567 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/52c8b4d6-193b-4ee5-aeed-88a414860ef2-openshift-service-ca\") on node \"ip-10-0-128-241.ec2.internal\" DevicePath \"\""
Sep 10 15:53:17.299677 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:53:17.299686    2567 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/52c8b4d6-193b-4ee5-aeed-88a414860ef2-metrics-certs\") on node \"ip-10-0-128-241.ec2.internal\" DevicePath \"\""
Sep 10 15:53:17.299919 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:53:17.299696    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7d5hn\" (UniqueName: \"kubernetes.io/projected/52c8b4d6-193b-4ee5-aeed-88a414860ef2-kube-api-access-7d5hn\") on node \"ip-10-0-128-241.ec2.internal\" DevicePath \"\""
Sep 10 15:53:17.468635 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:53:17.468582    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:53:17.644857 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:53:17.644820    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:53:18.378607 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:53:18.378570    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:53:18.904164 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:53:18.904132    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="52c8b4d6-193b-4ee5-aeed-88a414860ef2" path="/var/lib/kubelet/pods/52c8b4d6-193b-4ee5-aeed-88a414860ef2/volumes"
Sep 10 15:53:19.239348 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:53:19.239320    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:53:19.980505 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:53:19.980469    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:53:23.738409 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:53:23.738375    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:53:23.838440 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:53:23.838392    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:53:24.007571 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:53:24.007487    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:53:24.658670 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:53:24.658638    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:53:25.372563 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:53:25.372522    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:53:26.281449 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:53:26.281414    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:53:27.043992 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:53:27.043955    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:53:27.697671 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:53:27.697637    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:53:27.951845 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:53:27.951752    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:53:28.500950 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:53:28.500911    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:53:28.933553 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:53:28.933517    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:53:29.256460 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:53:29.256361    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:53:30.127532 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:53:30.127496    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:53:31.847518 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:53:31.847481    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:53:32.481853 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:53:32.481798    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:53:32.585526 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:53:32.585491    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:53:32.849205 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:53:32.849103    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:53:33.759617 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:53:33.759583    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:53:34.429350 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:53:34.429313    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:53:34.458845 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:53:34.458813    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-ec08ac-68959b96f9-dnnld"]
Sep 10 15:53:34.459080 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:53:34.459050    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-ec08ac-68959b96f9-dnnld" podUID="7e4bd2b6-bf50-4289-88a6-f9e123c50409" containerName="payload-processing" containerID="cri-o://9c1f24fd9a31ebe52b9c0b957206d3dd6b753f782c50efc3d00c7737283dd203" gracePeriod=30
Sep 10 15:53:34.466585 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:53:34.466557    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-ec08ac-7d9964cf54-fk5zt"]
Sep 10 15:53:34.467466 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:53:34.467436    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-ec08ac-7d9964cf54-fk5zt" podUID="1ea2c51d-8376-4903-88e6-7db46a6f6264" containerName="payload-pre-processing" containerID="cri-o://34d9b5beda566f37f6cf20bcfb631424cbb222aaa8a44ad79fe84cef06235191" gracePeriod=30
Sep 10 15:53:34.512790 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:53:34.512748    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:53:34.586662 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:53:34.586629    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:53:34.687746 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:53:34.687657    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:53:34.717278 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:53:34.717255    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-ec08ac-68959b96f9-dnnld"
Sep 10 15:53:34.748450 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:53:34.748423    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-ec08ac-7d9964cf54-fk5zt"
Sep 10 15:53:34.769574 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:53:34.769536    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1ea2c51d-8376-4903-88e6-7db46a6f6264-plugins-config-volume\") pod \"1ea2c51d-8376-4903-88e6-7db46a6f6264\" (UID: \"1ea2c51d-8376-4903-88e6-7db46a6f6264\") "
Sep 10 15:53:34.769781 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:53:34.769653    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pwcvx\" (UniqueName: \"kubernetes.io/projected/7e4bd2b6-bf50-4289-88a6-f9e123c50409-kube-api-access-pwcvx\") pod \"7e4bd2b6-bf50-4289-88a6-f9e123c50409\" (UID: \"7e4bd2b6-bf50-4289-88a6-f9e123c50409\") "
Sep 10 15:53:34.769781 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:53:34.769737    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tvzhj\" (UniqueName: \"kubernetes.io/projected/1ea2c51d-8376-4903-88e6-7db46a6f6264-kube-api-access-tvzhj\") pod \"1ea2c51d-8376-4903-88e6-7db46a6f6264\" (UID: \"1ea2c51d-8376-4903-88e6-7db46a6f6264\") "
Sep 10 15:53:34.769781 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:53:34.769775    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7e4bd2b6-bf50-4289-88a6-f9e123c50409-plugins-config-volume\") pod \"7e4bd2b6-bf50-4289-88a6-f9e123c50409\" (UID: \"7e4bd2b6-bf50-4289-88a6-f9e123c50409\") "
Sep 10 15:53:34.769977 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:53:34.769956    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1ea2c51d-8376-4903-88e6-7db46a6f6264-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "1ea2c51d-8376-4903-88e6-7db46a6f6264" (UID: "1ea2c51d-8376-4903-88e6-7db46a6f6264"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 15:53:34.770114 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:53:34.770095    2567 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1ea2c51d-8376-4903-88e6-7db46a6f6264-plugins-config-volume\") on node \"ip-10-0-128-241.ec2.internal\" DevicePath \"\""
Sep 10 15:53:34.770398 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:53:34.770371    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7e4bd2b6-bf50-4289-88a6-f9e123c50409-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "7e4bd2b6-bf50-4289-88a6-f9e123c50409" (UID: "7e4bd2b6-bf50-4289-88a6-f9e123c50409"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 15:53:34.771789 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:53:34.771764    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7e4bd2b6-bf50-4289-88a6-f9e123c50409-kube-api-access-pwcvx" (OuterVolumeSpecName: "kube-api-access-pwcvx") pod "7e4bd2b6-bf50-4289-88a6-f9e123c50409" (UID: "7e4bd2b6-bf50-4289-88a6-f9e123c50409"). InnerVolumeSpecName "kube-api-access-pwcvx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 15:53:34.771891 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:53:34.771865    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1ea2c51d-8376-4903-88e6-7db46a6f6264-kube-api-access-tvzhj" (OuterVolumeSpecName: "kube-api-access-tvzhj") pod "1ea2c51d-8376-4903-88e6-7db46a6f6264" (UID: "1ea2c51d-8376-4903-88e6-7db46a6f6264"). InnerVolumeSpecName "kube-api-access-tvzhj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 15:53:34.782057 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:53:34.782029    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:53:34.851088 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:53:34.851053    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:53:34.871082 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:53:34.871043    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pwcvx\" (UniqueName: \"kubernetes.io/projected/7e4bd2b6-bf50-4289-88a6-f9e123c50409-kube-api-access-pwcvx\") on node \"ip-10-0-128-241.ec2.internal\" DevicePath \"\""
Sep 10 15:53:34.871082 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:53:34.871073    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tvzhj\" (UniqueName: \"kubernetes.io/projected/1ea2c51d-8376-4903-88e6-7db46a6f6264-kube-api-access-tvzhj\") on node \"ip-10-0-128-241.ec2.internal\" DevicePath \"\""
Sep 10 15:53:34.871082 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:53:34.871083    2567 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7e4bd2b6-bf50-4289-88a6-f9e123c50409-plugins-config-volume\") on node \"ip-10-0-128-241.ec2.internal\" DevicePath \"\""
Sep 10 15:53:35.066703 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:53:35.066614    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:53:35.321679 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:53:35.321579    2567 generic.go:356] "Generic (PLEG): container finished" podID="1ea2c51d-8376-4903-88e6-7db46a6f6264" containerID="34d9b5beda566f37f6cf20bcfb631424cbb222aaa8a44ad79fe84cef06235191" exitCode=0
Sep 10 15:53:35.321679 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:53:35.321630    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-ec08ac-7d9964cf54-fk5zt"
Sep 10 15:53:35.321679 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:53:35.321668    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-ec08ac-7d9964cf54-fk5zt" event={"ID":"1ea2c51d-8376-4903-88e6-7db46a6f6264","Type":"ContainerDied","Data":"34d9b5beda566f37f6cf20bcfb631424cbb222aaa8a44ad79fe84cef06235191"}
Sep 10 15:53:35.321972 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:53:35.321726    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-ec08ac-7d9964cf54-fk5zt" event={"ID":"1ea2c51d-8376-4903-88e6-7db46a6f6264","Type":"ContainerDied","Data":"fa4545909dc07d80095fc4f1ea2724f1ac7354c5f68f7ee23b1c052ec8d92ab4"}
Sep 10 15:53:35.321972 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:53:35.321755    2567 scope.go:117] "RemoveContainer" containerID="34d9b5beda566f37f6cf20bcfb631424cbb222aaa8a44ad79fe84cef06235191"
Sep 10 15:53:35.323447 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:53:35.323421    2567 generic.go:356] "Generic (PLEG): container finished" podID="7e4bd2b6-bf50-4289-88a6-f9e123c50409" containerID="9c1f24fd9a31ebe52b9c0b957206d3dd6b753f782c50efc3d00c7737283dd203" exitCode=0
Sep 10 15:53:35.323552 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:53:35.323512    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-ec08ac-68959b96f9-dnnld"
Sep 10 15:53:35.324046 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:53:35.324008    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-ec08ac-68959b96f9-dnnld" event={"ID":"7e4bd2b6-bf50-4289-88a6-f9e123c50409","Type":"ContainerDied","Data":"9c1f24fd9a31ebe52b9c0b957206d3dd6b753f782c50efc3d00c7737283dd203"}
Sep 10 15:53:35.324151 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:53:35.324083    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-ec08ac-68959b96f9-dnnld" event={"ID":"7e4bd2b6-bf50-4289-88a6-f9e123c50409","Type":"ContainerDied","Data":"cc7e579f369cf97a2efaf4831ede1af0a2711828040c41b9af2ed5e714744d54"}
Sep 10 15:53:35.331675 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:53:35.331652    2567 scope.go:117] "RemoveContainer" containerID="34d9b5beda566f37f6cf20bcfb631424cbb222aaa8a44ad79fe84cef06235191"
Sep 10 15:53:35.331976 ip-10-0-128-241 kubenswrapper[2567]: E0910 15:53:35.331955    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"34d9b5beda566f37f6cf20bcfb631424cbb222aaa8a44ad79fe84cef06235191\": container with ID starting with 34d9b5beda566f37f6cf20bcfb631424cbb222aaa8a44ad79fe84cef06235191 not found: ID does not exist" containerID="34d9b5beda566f37f6cf20bcfb631424cbb222aaa8a44ad79fe84cef06235191"
Sep 10 15:53:35.332056 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:53:35.331984    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"34d9b5beda566f37f6cf20bcfb631424cbb222aaa8a44ad79fe84cef06235191"} err="failed to get container status \"34d9b5beda566f37f6cf20bcfb631424cbb222aaa8a44ad79fe84cef06235191\": rpc error: code = NotFound desc = could not find container \"34d9b5beda566f37f6cf20bcfb631424cbb222aaa8a44ad79fe84cef06235191\": container with ID starting with 34d9b5beda566f37f6cf20bcfb631424cbb222aaa8a44ad79fe84cef06235191 not found: ID does not exist"
Sep 10 15:53:35.332056 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:53:35.332001    2567 scope.go:117] "RemoveContainer" containerID="9c1f24fd9a31ebe52b9c0b957206d3dd6b753f782c50efc3d00c7737283dd203"
Sep 10 15:53:35.341097 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:53:35.341056    2567 scope.go:117] "RemoveContainer" containerID="9c1f24fd9a31ebe52b9c0b957206d3dd6b753f782c50efc3d00c7737283dd203"
Sep 10 15:53:35.341765 ip-10-0-128-241 kubenswrapper[2567]: E0910 15:53:35.341452    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9c1f24fd9a31ebe52b9c0b957206d3dd6b753f782c50efc3d00c7737283dd203\": container with ID starting with 9c1f24fd9a31ebe52b9c0b957206d3dd6b753f782c50efc3d00c7737283dd203 not found: ID does not exist" containerID="9c1f24fd9a31ebe52b9c0b957206d3dd6b753f782c50efc3d00c7737283dd203"
Sep 10 15:53:35.341765 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:53:35.341508    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9c1f24fd9a31ebe52b9c0b957206d3dd6b753f782c50efc3d00c7737283dd203"} err="failed to get container status \"9c1f24fd9a31ebe52b9c0b957206d3dd6b753f782c50efc3d00c7737283dd203\": rpc error: code = NotFound desc = could not find container \"9c1f24fd9a31ebe52b9c0b957206d3dd6b753f782c50efc3d00c7737283dd203\": container with ID starting with 9c1f24fd9a31ebe52b9c0b957206d3dd6b753f782c50efc3d00c7737283dd203 not found: ID does not exist"
Sep 10 15:53:35.343786 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:53:35.343761    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-ec08ac-7d9964cf54-fk5zt"]
Sep 10 15:53:35.348149 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:53:35.348123    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-ec08ac-7d9964cf54-fk5zt"]
Sep 10 15:53:35.358670 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:53:35.358640    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-ec08ac-68959b96f9-dnnld"]
Sep 10 15:53:35.364912 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:53:35.364888    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-ec08ac-68959b96f9-dnnld"]
Sep 10 15:53:35.468104 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:53:35.468053    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:53:35.567867 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:53:35.567832    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:53:35.588380 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:53:35.588303    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-8cb756-785d8666ff-4pbxl"]
Sep 10 15:53:35.588677 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:53:35.588663    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1ea2c51d-8376-4903-88e6-7db46a6f6264" containerName="payload-pre-processing"
Sep 10 15:53:35.588677 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:53:35.588678    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="1ea2c51d-8376-4903-88e6-7db46a6f6264" containerName="payload-pre-processing"
Sep 10 15:53:35.588853 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:53:35.588700    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="52c8b4d6-193b-4ee5-aeed-88a414860ef2" containerName="maas-api"
Sep 10 15:53:35.588853 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:53:35.588708    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="52c8b4d6-193b-4ee5-aeed-88a414860ef2" containerName="maas-api"
Sep 10 15:53:35.588853 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:53:35.588715    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7e4bd2b6-bf50-4289-88a6-f9e123c50409" containerName="payload-processing"
Sep 10 15:53:35.588853 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:53:35.588720    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="7e4bd2b6-bf50-4289-88a6-f9e123c50409" containerName="payload-processing"
Sep 10 15:53:35.588853 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:53:35.588726    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f21e844c-809e-4b7b-a675-b12ad2accd4f" containerName="payload-pre-processing"
Sep 10 15:53:35.588853 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:53:35.588731    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="f21e844c-809e-4b7b-a675-b12ad2accd4f" containerName="payload-pre-processing"
Sep 10 15:53:35.588853 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:53:35.588738    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3837fc27-5440-4736-995c-6e0f12ba4033" containerName="maas-api"
Sep 10 15:53:35.588853 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:53:35.588743    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="3837fc27-5440-4736-995c-6e0f12ba4033" containerName="maas-api"
Sep 10 15:53:35.588853 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:53:35.588750    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c079d979-0e43-461c-b6e5-f2a99850e50c" containerName="test-curl-7752-59c482"
Sep 10 15:53:35.588853 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:53:35.588755    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="c079d979-0e43-461c-b6e5-f2a99850e50c" containerName="test-curl-7752-59c482"
Sep 10 15:53:35.588853 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:53:35.588764    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2d52e90c-ab6d-47ee-a2ba-dc945af6d473" containerName="payload-processing"
Sep 10 15:53:35.588853 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:53:35.588769    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="2d52e90c-ab6d-47ee-a2ba-dc945af6d473" containerName="payload-processing"
Sep 10 15:53:35.588853 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:53:35.588823    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="f21e844c-809e-4b7b-a675-b12ad2accd4f" containerName="payload-pre-processing"
Sep 10 15:53:35.588853 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:53:35.588831    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="3837fc27-5440-4736-995c-6e0f12ba4033" containerName="maas-api"
Sep 10 15:53:35.588853 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:53:35.588838    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="2d52e90c-ab6d-47ee-a2ba-dc945af6d473" containerName="payload-processing"
Sep 10 15:53:35.588853 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:53:35.588843    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="52c8b4d6-193b-4ee5-aeed-88a414860ef2" containerName="maas-api"
Sep 10 15:53:35.588853 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:53:35.588860    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="1ea2c51d-8376-4903-88e6-7db46a6f6264" containerName="payload-pre-processing"
Sep 10 15:53:35.589533 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:53:35.588868    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="c079d979-0e43-461c-b6e5-f2a99850e50c" containerName="test-curl-7752-59c482"
Sep 10 15:53:35.589533 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:53:35.588878    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="7e4bd2b6-bf50-4289-88a6-f9e123c50409" containerName="payload-processing"
Sep 10 15:53:35.593562 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:53:35.593545    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-8cb756-785d8666ff-4pbxl"
Sep 10 15:53:35.596171 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:53:35.596144    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-a-8cb756-dockercfg-dt2jg\""
Sep 10 15:53:35.596171 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:53:35.596162    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-a-8cb756\""
Sep 10 15:53:35.601802 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:53:35.601780    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-8cb756-785d8666ff-4pbxl"]
Sep 10 15:53:35.678187 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:53:35.678148    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qwhn2\" (UniqueName: \"kubernetes.io/projected/d2205171-a2e0-4934-8bcf-f6931285bca5-kube-api-access-qwhn2\") pod \"payload-processing-e2e-ipp-a-8cb756-785d8666ff-4pbxl\" (UID: \"d2205171-a2e0-4934-8bcf-f6931285bca5\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-8cb756-785d8666ff-4pbxl"
Sep 10 15:53:35.678411 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:53:35.678200    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d2205171-a2e0-4934-8bcf-f6931285bca5-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-8cb756-785d8666ff-4pbxl\" (UID: \"d2205171-a2e0-4934-8bcf-f6931285bca5\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-8cb756-785d8666ff-4pbxl"
Sep 10 15:53:35.779579 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:53:35.779533    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qwhn2\" (UniqueName: \"kubernetes.io/projected/d2205171-a2e0-4934-8bcf-f6931285bca5-kube-api-access-qwhn2\") pod \"payload-processing-e2e-ipp-a-8cb756-785d8666ff-4pbxl\" (UID: \"d2205171-a2e0-4934-8bcf-f6931285bca5\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-8cb756-785d8666ff-4pbxl"
Sep 10 15:53:35.779579 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:53:35.779580    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d2205171-a2e0-4934-8bcf-f6931285bca5-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-8cb756-785d8666ff-4pbxl\" (UID: \"d2205171-a2e0-4934-8bcf-f6931285bca5\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-8cb756-785d8666ff-4pbxl"
Sep 10 15:53:35.780267 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:53:35.780219    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d2205171-a2e0-4934-8bcf-f6931285bca5-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-8cb756-785d8666ff-4pbxl\" (UID: \"d2205171-a2e0-4934-8bcf-f6931285bca5\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-8cb756-785d8666ff-4pbxl"
Sep 10 15:53:35.790104 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:53:35.790074    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qwhn2\" (UniqueName: \"kubernetes.io/projected/d2205171-a2e0-4934-8bcf-f6931285bca5-kube-api-access-qwhn2\") pod \"payload-processing-e2e-ipp-a-8cb756-785d8666ff-4pbxl\" (UID: \"d2205171-a2e0-4934-8bcf-f6931285bca5\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-8cb756-785d8666ff-4pbxl"
Sep 10 15:53:35.883403 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:53:35.883312    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:53:35.905346 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:53:35.905308    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-8cb756-785d8666ff-4pbxl"
Sep 10 15:53:36.176628 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:53:36.176488    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-8cb756-785d8666ff-4pbxl"]
Sep 10 15:53:36.179027 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:53:36.179000    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd2205171_a2e0_4934_8bcf_f6931285bca5.slice/crio-499b48bb343bbae64c7928db5605cb24ab747a23659aaf4e5bed66150de73ac4 WatchSource:0}: Error finding container 499b48bb343bbae64c7928db5605cb24ab747a23659aaf4e5bed66150de73ac4: Status 404 returned error can't find the container with id 499b48bb343bbae64c7928db5605cb24ab747a23659aaf4e5bed66150de73ac4
Sep 10 15:53:36.332319 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:53:36.332281    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-8cb756-785d8666ff-4pbxl" event={"ID":"d2205171-a2e0-4934-8bcf-f6931285bca5","Type":"ContainerStarted","Data":"ffa08bd68998a1bf957b51b83d40a68f93e5c840297f3923784a8e80875dfab2"}
Sep 10 15:53:36.332505 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:53:36.332332    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-8cb756-785d8666ff-4pbxl" event={"ID":"d2205171-a2e0-4934-8bcf-f6931285bca5","Type":"ContainerStarted","Data":"499b48bb343bbae64c7928db5605cb24ab747a23659aaf4e5bed66150de73ac4"}
Sep 10 15:53:36.332505 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:53:36.332365    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-8cb756-785d8666ff-4pbxl"
Sep 10 15:53:36.351554 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:53:36.351501    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-a-8cb756-785d8666ff-4pbxl" podStartSLOduration=1.35148432 podStartE2EDuration="1.35148432s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 15:53:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 15:53:36.350720297 +0000 UTC m=+1601.918952030" watchObservedRunningTime="2026-09-10 15:53:36.35148432 +0000 UTC m=+1601.919716050"
Sep 10 15:53:36.762167 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:53:36.762131    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:53:36.904707 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:53:36.904677    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1ea2c51d-8376-4903-88e6-7db46a6f6264" path="/var/lib/kubelet/pods/1ea2c51d-8376-4903-88e6-7db46a6f6264/volumes"
Sep 10 15:53:36.905042 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:53:36.905029    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7e4bd2b6-bf50-4289-88a6-f9e123c50409" path="/var/lib/kubelet/pods/7e4bd2b6-bf50-4289-88a6-f9e123c50409/volumes"
Sep 10 15:53:37.542216 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:53:37.542182    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:53:38.419903 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:53:38.419867    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:53:41.018072 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:53:41.018032    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:53:41.118141 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:53:41.118105    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:53:41.293547 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:53:41.293439    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:53:47.339473 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:53:47.339391    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-8cb756-785d8666ff-4pbxl"
Sep 10 15:53:56.939859 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:53:56.939817    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:53:57.527625 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:53:57.527583    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:53:57.868619 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:53:57.868523    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:53:58.184524 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:53:58.184490    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:53:58.500970 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:53:58.500878    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:53:58.673512 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:53:58.673481    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:53:58.773526 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:53:58.773439    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:53:59.402594 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:53:59.402554    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:54:00.667518 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:54:00.667478    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:54:01.378815 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:54:01.378766    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:54:01.478056 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:54:01.478021    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:54:01.869498 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:54:01.869461    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:54:02.651337 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:54:02.651305    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:54:03.555483 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:54:03.555449    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:54:04.386782 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:54:04.386740    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:54:04.487385 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:54:04.487350    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:54:05.006921 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:54:05.006887    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:54:05.106621 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:54:05.106582    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:54:05.133581 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:54:05.133544    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-6d929c-6dd9df4dcc-mgrwh"]
Sep 10 15:54:05.137030 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:54:05.137013    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-6d929c-6dd9df4dcc-mgrwh"
Sep 10 15:54:05.139678 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:54:05.139654    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-b-6d929c\""
Sep 10 15:54:05.139780 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:54:05.139714    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-b-6d929c-dockercfg-q778d\""
Sep 10 15:54:05.147607 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:54:05.147581    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-6d929c-6dd9df4dcc-mgrwh"]
Sep 10 15:54:05.192773 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:54:05.192738    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:54:05.193625 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:54:05.193600    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-6d929c-5f6d4fd6f4-mlxst"]
Sep 10 15:54:05.196866 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:54:05.196846    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-6d929c-5f6d4fd6f4-mlxst"
Sep 10 15:54:05.206938 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:54:05.206912    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-6d929c-5f6d4fd6f4-mlxst"]
Sep 10 15:54:05.266196 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:54:05.266105    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d40f9e24-8020-4a0b-96db-e18cf3b4c09b-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-6d929c-6dd9df4dcc-mgrwh\" (UID: \"d40f9e24-8020-4a0b-96db-e18cf3b4c09b\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-6d929c-6dd9df4dcc-mgrwh"
Sep 10 15:54:05.266196 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:54:05.266137    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bllzp\" (UniqueName: \"kubernetes.io/projected/d40f9e24-8020-4a0b-96db-e18cf3b4c09b-kube-api-access-bllzp\") pod \"payload-processing-e2e-ipp-b-6d929c-6dd9df4dcc-mgrwh\" (UID: \"d40f9e24-8020-4a0b-96db-e18cf3b4c09b\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-6d929c-6dd9df4dcc-mgrwh"
Sep 10 15:54:05.367547 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:54:05.367514    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d40f9e24-8020-4a0b-96db-e18cf3b4c09b-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-6d929c-6dd9df4dcc-mgrwh\" (UID: \"d40f9e24-8020-4a0b-96db-e18cf3b4c09b\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-6d929c-6dd9df4dcc-mgrwh"
Sep 10 15:54:05.367547 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:54:05.367548    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bllzp\" (UniqueName: \"kubernetes.io/projected/d40f9e24-8020-4a0b-96db-e18cf3b4c09b-kube-api-access-bllzp\") pod \"payload-processing-e2e-ipp-b-6d929c-6dd9df4dcc-mgrwh\" (UID: \"d40f9e24-8020-4a0b-96db-e18cf3b4c09b\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-6d929c-6dd9df4dcc-mgrwh"
Sep 10 15:54:05.367797 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:54:05.367638    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-56bj6\" (UniqueName: \"kubernetes.io/projected/ae4f3225-d4d9-4e87-ba9b-fc9ce03cf3df-kube-api-access-56bj6\") pod \"payload-pre-processing-e2e-ipp-b-6d929c-5f6d4fd6f4-mlxst\" (UID: \"ae4f3225-d4d9-4e87-ba9b-fc9ce03cf3df\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-6d929c-5f6d4fd6f4-mlxst"
Sep 10 15:54:05.367797 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:54:05.367685    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ae4f3225-d4d9-4e87-ba9b-fc9ce03cf3df-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-6d929c-5f6d4fd6f4-mlxst\" (UID: \"ae4f3225-d4d9-4e87-ba9b-fc9ce03cf3df\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-6d929c-5f6d4fd6f4-mlxst"
Sep 10 15:54:05.368306 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:54:05.368286    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d40f9e24-8020-4a0b-96db-e18cf3b4c09b-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-6d929c-6dd9df4dcc-mgrwh\" (UID: \"d40f9e24-8020-4a0b-96db-e18cf3b4c09b\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-6d929c-6dd9df4dcc-mgrwh"
Sep 10 15:54:05.377114 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:54:05.377088    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bllzp\" (UniqueName: \"kubernetes.io/projected/d40f9e24-8020-4a0b-96db-e18cf3b4c09b-kube-api-access-bllzp\") pod \"payload-processing-e2e-ipp-b-6d929c-6dd9df4dcc-mgrwh\" (UID: \"d40f9e24-8020-4a0b-96db-e18cf3b4c09b\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-6d929c-6dd9df4dcc-mgrwh"
Sep 10 15:54:05.448694 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:54:05.448661    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-6d929c-6dd9df4dcc-mgrwh"
Sep 10 15:54:05.468456 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:54:05.468423    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-56bj6\" (UniqueName: \"kubernetes.io/projected/ae4f3225-d4d9-4e87-ba9b-fc9ce03cf3df-kube-api-access-56bj6\") pod \"payload-pre-processing-e2e-ipp-b-6d929c-5f6d4fd6f4-mlxst\" (UID: \"ae4f3225-d4d9-4e87-ba9b-fc9ce03cf3df\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-6d929c-5f6d4fd6f4-mlxst"
Sep 10 15:54:05.468596 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:54:05.468475    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ae4f3225-d4d9-4e87-ba9b-fc9ce03cf3df-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-6d929c-5f6d4fd6f4-mlxst\" (UID: \"ae4f3225-d4d9-4e87-ba9b-fc9ce03cf3df\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-6d929c-5f6d4fd6f4-mlxst"
Sep 10 15:54:05.469371 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:54:05.469345    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ae4f3225-d4d9-4e87-ba9b-fc9ce03cf3df-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-6d929c-5f6d4fd6f4-mlxst\" (UID: \"ae4f3225-d4d9-4e87-ba9b-fc9ce03cf3df\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-6d929c-5f6d4fd6f4-mlxst"
Sep 10 15:54:05.477701 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:54:05.477666    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-56bj6\" (UniqueName: \"kubernetes.io/projected/ae4f3225-d4d9-4e87-ba9b-fc9ce03cf3df-kube-api-access-56bj6\") pod \"payload-pre-processing-e2e-ipp-b-6d929c-5f6d4fd6f4-mlxst\" (UID: \"ae4f3225-d4d9-4e87-ba9b-fc9ce03cf3df\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-6d929c-5f6d4fd6f4-mlxst"
Sep 10 15:54:05.507604 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:54:05.507574    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-6d929c-5f6d4fd6f4-mlxst"
Sep 10 15:54:05.721087 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:54:05.721053    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-6d929c-6dd9df4dcc-mgrwh"]
Sep 10 15:54:05.723483 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:54:05.723454    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd40f9e24_8020_4a0b_96db_e18cf3b4c09b.slice/crio-3e3dda241c3174f4034a29a831d305ca0b279296a7943fef957b2f4eabe667b4 WatchSource:0}: Error finding container 3e3dda241c3174f4034a29a831d305ca0b279296a7943fef957b2f4eabe667b4: Status 404 returned error can't find the container with id 3e3dda241c3174f4034a29a831d305ca0b279296a7943fef957b2f4eabe667b4
Sep 10 15:54:05.799255 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:54:05.799214    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-6d929c-5f6d4fd6f4-mlxst"]
Sep 10 15:54:05.801120 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:54:05.801095    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podae4f3225_d4d9_4e87_ba9b_fc9ce03cf3df.slice/crio-07ed660b984a1301941dab0455233e9f903219778babc37207b808ac91f6cef5 WatchSource:0}: Error finding container 07ed660b984a1301941dab0455233e9f903219778babc37207b808ac91f6cef5: Status 404 returned error can't find the container with id 07ed660b984a1301941dab0455233e9f903219778babc37207b808ac91f6cef5
Sep 10 15:54:06.492280 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:54:06.492227    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-6d929c-5f6d4fd6f4-mlxst" event={"ID":"ae4f3225-d4d9-4e87-ba9b-fc9ce03cf3df","Type":"ContainerStarted","Data":"6ba678849e371e6a1390dea8019c9d0af422996bf30ef6e889631bc6b2d03cc4"}
Sep 10 15:54:06.492739 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:54:06.492301    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-6d929c-5f6d4fd6f4-mlxst" event={"ID":"ae4f3225-d4d9-4e87-ba9b-fc9ce03cf3df","Type":"ContainerStarted","Data":"07ed660b984a1301941dab0455233e9f903219778babc37207b808ac91f6cef5"}
Sep 10 15:54:06.492739 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:54:06.492446    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-6d929c-5f6d4fd6f4-mlxst"
Sep 10 15:54:06.493871 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:54:06.493842    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-6d929c-6dd9df4dcc-mgrwh" event={"ID":"d40f9e24-8020-4a0b-96db-e18cf3b4c09b","Type":"ContainerStarted","Data":"50167c682ad75f7179e899a3ab52d40828aee9cb66f16980771b93f772bad597"}
Sep 10 15:54:06.493975 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:54:06.493897    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-b-6d929c-6dd9df4dcc-mgrwh"
Sep 10 15:54:06.493975 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:54:06.493915    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-6d929c-6dd9df4dcc-mgrwh" event={"ID":"d40f9e24-8020-4a0b-96db-e18cf3b4c09b","Type":"ContainerStarted","Data":"3e3dda241c3174f4034a29a831d305ca0b279296a7943fef957b2f4eabe667b4"}
Sep 10 15:54:06.510217 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:54:06.510166    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-6d929c-5f6d4fd6f4-mlxst" podStartSLOduration=1.510151429 podStartE2EDuration="1.510151429s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 15:54:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 15:54:06.50887614 +0000 UTC m=+1632.077107861" watchObservedRunningTime="2026-09-10 15:54:06.510151429 +0000 UTC m=+1632.078383230"
Sep 10 15:54:06.524979 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:54:06.524926    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-b-6d929c-6dd9df4dcc-mgrwh" podStartSLOduration=1.524909741 podStartE2EDuration="1.524909741s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 15:54:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 15:54:06.523994941 +0000 UTC m=+1632.092226673" watchObservedRunningTime="2026-09-10 15:54:06.524909741 +0000 UTC m=+1632.093141471"
Sep 10 15:54:17.501786 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:54:17.501744    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-b-6d929c-6dd9df4dcc-mgrwh"
Sep 10 15:54:17.502213 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:54:17.501916    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-6d929c-5f6d4fd6f4-mlxst"
Sep 10 15:54:26.399505 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:54:26.399471    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:54:26.756530 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:54:26.756492    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:54:27.646188 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:54:27.646147    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:54:28.360207 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:54:28.360175    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:54:29.172576 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:54:29.172541    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:54:29.932459 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:54:29.932426    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:54:31.638307 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:54:31.638273    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-b-6d929c-6dd9df4dcc-mgrwh_d40f9e24-8020-4a0b-96db-e18cf3b4c09b/payload-processing/0.log"
Sep 10 15:54:33.032324 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:54:33.032286    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:54:33.241449 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:54:33.241414    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:54:50.815393 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:54:50.815355    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:54:50.869813 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:54:50.869773    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:55:08.213157 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:55:08.213121    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-a-8cb756-785d8666ff-4pbxl_d2205171-a2e0-4934-8bcf-f6931285bca5/payload-processing/0.log"
Sep 10 15:55:08.467718 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:55:08.467640    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-b-6d929c-6dd9df4dcc-mgrwh_d40f9e24-8020-4a0b-96db-e18cf3b4c09b/payload-processing/0.log"
Sep 10 15:55:08.739798 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:55:08.739713    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:55:09.083003 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:55:09.082900    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:55:09.559164 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:55:09.559129    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:55:11.144949 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:55:11.144913    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:55:11.799440 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:55:11.799405    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:55:11.891580 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:55:11.891547    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:55:11.991834 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:55:11.991798    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:55:12.850665 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:55:12.850628    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:55:13.651801 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:55:13.651766    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:55:14.394789 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:55:14.394753    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:55:15.118369 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:55:15.118334    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-e48e16-779bdbcc5f-rvscb"]
Sep 10 15:55:15.123276 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:55:15.123251    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-e48e16-779bdbcc5f-rvscb"
Sep 10 15:55:15.125697 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:55:15.125673    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-cleanup-e48e16-serving-cert\""
Sep 10 15:55:15.131736 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:55:15.131163    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-e48e16-779bdbcc5f-rvscb"]
Sep 10 15:55:15.156674 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:55:15.156639    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:55:15.219667 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:55:15.219632    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:55:15.221820 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:55:15.221790    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b25a0acb-f46d-4e62-9c5f-b47d0710cc4a-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-e48e16-779bdbcc5f-rvscb\" (UID: \"b25a0acb-f46d-4e62-9c5f-b47d0710cc4a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-e48e16-779bdbcc5f-rvscb"
Sep 10 15:55:15.221954 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:55:15.221856    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gmkqf\" (UniqueName: \"kubernetes.io/projected/b25a0acb-f46d-4e62-9c5f-b47d0710cc4a-kube-api-access-gmkqf\") pod \"maas-api-e2e-ipp-cleanup-e48e16-779bdbcc5f-rvscb\" (UID: \"b25a0acb-f46d-4e62-9c5f-b47d0710cc4a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-e48e16-779bdbcc5f-rvscb"
Sep 10 15:55:15.221954 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:55:15.221937    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b25a0acb-f46d-4e62-9c5f-b47d0710cc4a-metrics-certs\") pod \"maas-api-e2e-ipp-cleanup-e48e16-779bdbcc5f-rvscb\" (UID: \"b25a0acb-f46d-4e62-9c5f-b47d0710cc4a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-e48e16-779bdbcc5f-rvscb"
Sep 10 15:55:15.222078 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:55:15.221988    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b25a0acb-f46d-4e62-9c5f-b47d0710cc4a-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-e48e16-779bdbcc5f-rvscb\" (UID: \"b25a0acb-f46d-4e62-9c5f-b47d0710cc4a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-e48e16-779bdbcc5f-rvscb"
Sep 10 15:55:15.322489 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:55:15.322454    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b25a0acb-f46d-4e62-9c5f-b47d0710cc4a-metrics-certs\") pod \"maas-api-e2e-ipp-cleanup-e48e16-779bdbcc5f-rvscb\" (UID: \"b25a0acb-f46d-4e62-9c5f-b47d0710cc4a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-e48e16-779bdbcc5f-rvscb"
Sep 10 15:55:15.322688 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:55:15.322495    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b25a0acb-f46d-4e62-9c5f-b47d0710cc4a-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-e48e16-779bdbcc5f-rvscb\" (UID: \"b25a0acb-f46d-4e62-9c5f-b47d0710cc4a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-e48e16-779bdbcc5f-rvscb"
Sep 10 15:55:15.322688 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:55:15.322581    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b25a0acb-f46d-4e62-9c5f-b47d0710cc4a-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-e48e16-779bdbcc5f-rvscb\" (UID: \"b25a0acb-f46d-4e62-9c5f-b47d0710cc4a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-e48e16-779bdbcc5f-rvscb"
Sep 10 15:55:15.322688 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:55:15.322607    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gmkqf\" (UniqueName: \"kubernetes.io/projected/b25a0acb-f46d-4e62-9c5f-b47d0710cc4a-kube-api-access-gmkqf\") pod \"maas-api-e2e-ipp-cleanup-e48e16-779bdbcc5f-rvscb\" (UID: \"b25a0acb-f46d-4e62-9c5f-b47d0710cc4a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-e48e16-779bdbcc5f-rvscb"
Sep 10 15:55:15.322856 ip-10-0-128-241 kubenswrapper[2567]: E0910 15:55:15.322705    2567 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-e48e16-serving-cert: secret "maas-api-e2e-ipp-cleanup-e48e16-serving-cert" not found
Sep 10 15:55:15.322856 ip-10-0-128-241 kubenswrapper[2567]: E0910 15:55:15.322816    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b25a0acb-f46d-4e62-9c5f-b47d0710cc4a-maas-api-tls podName:b25a0acb-f46d-4e62-9c5f-b47d0710cc4a nodeName:}" failed. No retries permitted until 2026-09-10 15:55:15.822788829 +0000 UTC m=+1701.391020559 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/b25a0acb-f46d-4e62-9c5f-b47d0710cc4a-maas-api-tls") pod "maas-api-e2e-ipp-cleanup-e48e16-779bdbcc5f-rvscb" (UID: "b25a0acb-f46d-4e62-9c5f-b47d0710cc4a") : secret "maas-api-e2e-ipp-cleanup-e48e16-serving-cert" not found
Sep 10 15:55:15.323433 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:55:15.323407    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b25a0acb-f46d-4e62-9c5f-b47d0710cc4a-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-e48e16-779bdbcc5f-rvscb\" (UID: \"b25a0acb-f46d-4e62-9c5f-b47d0710cc4a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-e48e16-779bdbcc5f-rvscb"
Sep 10 15:55:15.326072 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:55:15.326048    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b25a0acb-f46d-4e62-9c5f-b47d0710cc4a-metrics-certs\") pod \"maas-api-e2e-ipp-cleanup-e48e16-779bdbcc5f-rvscb\" (UID: \"b25a0acb-f46d-4e62-9c5f-b47d0710cc4a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-e48e16-779bdbcc5f-rvscb"
Sep 10 15:55:15.331563 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:55:15.331541    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gmkqf\" (UniqueName: \"kubernetes.io/projected/b25a0acb-f46d-4e62-9c5f-b47d0710cc4a-kube-api-access-gmkqf\") pod \"maas-api-e2e-ipp-cleanup-e48e16-779bdbcc5f-rvscb\" (UID: \"b25a0acb-f46d-4e62-9c5f-b47d0710cc4a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-e48e16-779bdbcc5f-rvscb"
Sep 10 15:55:15.418518 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:55:15.418485    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:55:15.650122 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:55:15.650080    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:55:15.748595 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:55:15.748494    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:55:15.827125 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:55:15.827085    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b25a0acb-f46d-4e62-9c5f-b47d0710cc4a-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-e48e16-779bdbcc5f-rvscb\" (UID: \"b25a0acb-f46d-4e62-9c5f-b47d0710cc4a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-e48e16-779bdbcc5f-rvscb"
Sep 10 15:55:15.829509 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:55:15.829484    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b25a0acb-f46d-4e62-9c5f-b47d0710cc4a-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-e48e16-779bdbcc5f-rvscb\" (UID: \"b25a0acb-f46d-4e62-9c5f-b47d0710cc4a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-e48e16-779bdbcc5f-rvscb"
Sep 10 15:55:16.036343 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:55:16.036263    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-e48e16-779bdbcc5f-rvscb"
Sep 10 15:55:16.310478 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:55:16.310452    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-e48e16-779bdbcc5f-rvscb"]
Sep 10 15:55:16.312421 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:55:16.312391    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb25a0acb_f46d_4e62_9c5f_b47d0710cc4a.slice/crio-dbc7611fbb13521ad1c525fd35686e1ec46d5fcd530de93a792392f24830be84 WatchSource:0}: Error finding container dbc7611fbb13521ad1c525fd35686e1ec46d5fcd530de93a792392f24830be84: Status 404 returned error can't find the container with id dbc7611fbb13521ad1c525fd35686e1ec46d5fcd530de93a792392f24830be84
Sep 10 15:55:16.829405 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:55:16.829368    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-e48e16-779bdbcc5f-rvscb" event={"ID":"b25a0acb-f46d-4e62-9c5f-b47d0710cc4a","Type":"ContainerStarted","Data":"dbc7611fbb13521ad1c525fd35686e1ec46d5fcd530de93a792392f24830be84"}
Sep 10 15:55:17.835459 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:55:17.835417    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-e48e16-779bdbcc5f-rvscb" event={"ID":"b25a0acb-f46d-4e62-9c5f-b47d0710cc4a","Type":"ContainerStarted","Data":"2bacbe04649ddbd073c95d4d443af635cccb2b57ffe8965e3d23981231bf22c0"}
Sep 10 15:55:17.835936 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:55:17.835582    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-e48e16-779bdbcc5f-rvscb"
Sep 10 15:55:17.854316 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:55:17.854264    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-e48e16-779bdbcc5f-rvscb" podStartSLOduration=2.255313599 podStartE2EDuration="2.854218189s" totalImagesPullingTime="598.90459ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 15:55:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 15:55:17.853181923 +0000 UTC m=+1703.421413653" watchObservedRunningTime="2026-09-10 15:55:17.854218189 +0000 UTC m=+1703.422449919"
Sep 10 15:55:23.845806 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:55:23.845773    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-e48e16-779bdbcc5f-rvscb"
Sep 10 15:55:37.214952 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:55:37.214912    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:55:38.071446 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:55:38.071413    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:55:38.854469 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:55:38.854437    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:55:39.783829 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:55:39.783789    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:55:40.709056 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:55:40.709018    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:55:41.546812 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:55:41.546782    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:55:41.950769 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:55:41.950735    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:55:43.473691 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:55:43.473655    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:55:44.358192 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:55:44.358153    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:55:45.103985 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:55:45.103945    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:55:45.879568 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:55:45.879531    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:55:46.847565 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:55:46.847497    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:55:47.805581 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:55:47.805545    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:55:53.591958 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:55:53.591921    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-e48e16-779bdbcc5f-rvscb"]
Sep 10 15:55:53.592426 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:55:53.592175    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-e48e16-779bdbcc5f-rvscb" podUID="b25a0acb-f46d-4e62-9c5f-b47d0710cc4a" containerName="maas-api" containerID="cri-o://2bacbe04649ddbd073c95d4d443af635cccb2b57ffe8965e3d23981231bf22c0" gracePeriod=30
Sep 10 15:55:53.616616 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:55:53.616588    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:55:53.698752 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:55:53.698712    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:55:53.765818 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:55:53.765787    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:55:53.841892 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:55:53.841865    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-e48e16-779bdbcc5f-rvscb"
Sep 10 15:55:53.865825 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:55:53.865680    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:55:53.896932 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:55:53.896897    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b25a0acb-f46d-4e62-9c5f-b47d0710cc4a-metrics-certs\") pod \"b25a0acb-f46d-4e62-9c5f-b47d0710cc4a\" (UID: \"b25a0acb-f46d-4e62-9c5f-b47d0710cc4a\") "
Sep 10 15:55:53.896932 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:55:53.896934    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b25a0acb-f46d-4e62-9c5f-b47d0710cc4a-maas-api-tls\") pod \"b25a0acb-f46d-4e62-9c5f-b47d0710cc4a\" (UID: \"b25a0acb-f46d-4e62-9c5f-b47d0710cc4a\") "
Sep 10 15:55:53.897169 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:55:53.896964    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gmkqf\" (UniqueName: \"kubernetes.io/projected/b25a0acb-f46d-4e62-9c5f-b47d0710cc4a-kube-api-access-gmkqf\") pod \"b25a0acb-f46d-4e62-9c5f-b47d0710cc4a\" (UID: \"b25a0acb-f46d-4e62-9c5f-b47d0710cc4a\") "
Sep 10 15:55:53.897216 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:55:53.897172    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b25a0acb-f46d-4e62-9c5f-b47d0710cc4a-openshift-service-ca\") pod \"b25a0acb-f46d-4e62-9c5f-b47d0710cc4a\" (UID: \"b25a0acb-f46d-4e62-9c5f-b47d0710cc4a\") "
Sep 10 15:55:53.897573 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:55:53.897543    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b25a0acb-f46d-4e62-9c5f-b47d0710cc4a-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "b25a0acb-f46d-4e62-9c5f-b47d0710cc4a" (UID: "b25a0acb-f46d-4e62-9c5f-b47d0710cc4a"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 15:55:53.899325 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:55:53.899290    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b25a0acb-f46d-4e62-9c5f-b47d0710cc4a-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "b25a0acb-f46d-4e62-9c5f-b47d0710cc4a" (UID: "b25a0acb-f46d-4e62-9c5f-b47d0710cc4a"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 15:55:53.899453 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:55:53.899382    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b25a0acb-f46d-4e62-9c5f-b47d0710cc4a-kube-api-access-gmkqf" (OuterVolumeSpecName: "kube-api-access-gmkqf") pod "b25a0acb-f46d-4e62-9c5f-b47d0710cc4a" (UID: "b25a0acb-f46d-4e62-9c5f-b47d0710cc4a"). InnerVolumeSpecName "kube-api-access-gmkqf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 15:55:53.899453 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:55:53.899396    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b25a0acb-f46d-4e62-9c5f-b47d0710cc4a-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "b25a0acb-f46d-4e62-9c5f-b47d0710cc4a" (UID: "b25a0acb-f46d-4e62-9c5f-b47d0710cc4a"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 15:55:53.961759 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:55:53.961728    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:55:53.998184 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:55:53.998153    2567 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b25a0acb-f46d-4e62-9c5f-b47d0710cc4a-openshift-service-ca\") on node \"ip-10-0-128-241.ec2.internal\" DevicePath \"\""
Sep 10 15:55:53.998184 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:55:53.998183    2567 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b25a0acb-f46d-4e62-9c5f-b47d0710cc4a-metrics-certs\") on node \"ip-10-0-128-241.ec2.internal\" DevicePath \"\""
Sep 10 15:55:53.998426 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:55:53.998195    2567 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b25a0acb-f46d-4e62-9c5f-b47d0710cc4a-maas-api-tls\") on node \"ip-10-0-128-241.ec2.internal\" DevicePath \"\""
Sep 10 15:55:53.998426 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:55:53.998208    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gmkqf\" (UniqueName: \"kubernetes.io/projected/b25a0acb-f46d-4e62-9c5f-b47d0710cc4a-kube-api-access-gmkqf\") on node \"ip-10-0-128-241.ec2.internal\" DevicePath \"\""
Sep 10 15:55:54.007628 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:55:54.007593    2567 generic.go:356] "Generic (PLEG): container finished" podID="b25a0acb-f46d-4e62-9c5f-b47d0710cc4a" containerID="2bacbe04649ddbd073c95d4d443af635cccb2b57ffe8965e3d23981231bf22c0" exitCode=0
Sep 10 15:55:54.007818 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:55:54.007648    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-e48e16-779bdbcc5f-rvscb"
Sep 10 15:55:54.007818 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:55:54.007685    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-e48e16-779bdbcc5f-rvscb" event={"ID":"b25a0acb-f46d-4e62-9c5f-b47d0710cc4a","Type":"ContainerDied","Data":"2bacbe04649ddbd073c95d4d443af635cccb2b57ffe8965e3d23981231bf22c0"}
Sep 10 15:55:54.007818 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:55:54.007758    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-e48e16-779bdbcc5f-rvscb" event={"ID":"b25a0acb-f46d-4e62-9c5f-b47d0710cc4a","Type":"ContainerDied","Data":"dbc7611fbb13521ad1c525fd35686e1ec46d5fcd530de93a792392f24830be84"}
Sep 10 15:55:54.007818 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:55:54.007791    2567 scope.go:117] "RemoveContainer" containerID="2bacbe04649ddbd073c95d4d443af635cccb2b57ffe8965e3d23981231bf22c0"
Sep 10 15:55:54.017010 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:55:54.016991    2567 scope.go:117] "RemoveContainer" containerID="2bacbe04649ddbd073c95d4d443af635cccb2b57ffe8965e3d23981231bf22c0"
Sep 10 15:55:54.017313 ip-10-0-128-241 kubenswrapper[2567]: E0910 15:55:54.017292    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2bacbe04649ddbd073c95d4d443af635cccb2b57ffe8965e3d23981231bf22c0\": container with ID starting with 2bacbe04649ddbd073c95d4d443af635cccb2b57ffe8965e3d23981231bf22c0 not found: ID does not exist" containerID="2bacbe04649ddbd073c95d4d443af635cccb2b57ffe8965e3d23981231bf22c0"
Sep 10 15:55:54.017399 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:55:54.017324    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2bacbe04649ddbd073c95d4d443af635cccb2b57ffe8965e3d23981231bf22c0"} err="failed to get container status \"2bacbe04649ddbd073c95d4d443af635cccb2b57ffe8965e3d23981231bf22c0\": rpc error: code = NotFound desc = could not find container \"2bacbe04649ddbd073c95d4d443af635cccb2b57ffe8965e3d23981231bf22c0\": container with ID starting with 2bacbe04649ddbd073c95d4d443af635cccb2b57ffe8965e3d23981231bf22c0 not found: ID does not exist"
Sep 10 15:55:54.030275 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:55:54.030244    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-e48e16-779bdbcc5f-rvscb"]
Sep 10 15:55:54.032375 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:55:54.032353    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-e48e16-779bdbcc5f-rvscb"]
Sep 10 15:55:54.267667 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:55:54.267628    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:55:54.368089 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:55:54.368048    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:55:54.904423 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:55:54.904393    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b25a0acb-f46d-4e62-9c5f-b47d0710cc4a" path="/var/lib/kubelet/pods/b25a0acb-f46d-4e62-9c5f-b47d0710cc4a/volumes"
Sep 10 15:55:54.971344 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:55:54.971312    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:55:55.729174 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:55:55.729139    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:55:56.615562 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:55:56.615533    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:55:57.447156 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:55:57.447122    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:56:00.647375 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:56:00.647343    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:56:00.728684 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:56:00.728653    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:56:00.929433 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:56:00.929391    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:56:05.055465 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:56:05.055429    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:56:05.968153 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:56:05.968123    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:56:06.725028 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:56:06.724980    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:56:07.786419 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:56:07.786379    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:56:08.538331 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:56:08.538292    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:56:09.445415 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:56:09.445380    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:56:20.238741 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:56:20.238708    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:56:20.273746 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:56:20.273711    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-8cb756-785d8666ff-4pbxl"]
Sep 10 15:56:20.274007 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:56:20.273978    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-a-8cb756-785d8666ff-4pbxl" podUID="d2205171-a2e0-4934-8bcf-f6931285bca5" containerName="payload-processing" containerID="cri-o://ffa08bd68998a1bf957b51b83d40a68f93e5c840297f3923784a8e80875dfab2" gracePeriod=30
Sep 10 15:56:20.340222 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:56:20.340190    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:56:20.485286 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:56:20.485226    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:56:20.538044 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:56:20.538017    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-8cb756-785d8666ff-4pbxl"
Sep 10 15:56:20.552712 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:56:20.552680    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:56:20.635178 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:56:20.635146    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d2205171-a2e0-4934-8bcf-f6931285bca5-plugins-config-volume\") pod \"d2205171-a2e0-4934-8bcf-f6931285bca5\" (UID: \"d2205171-a2e0-4934-8bcf-f6931285bca5\") "
Sep 10 15:56:20.635392 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:56:20.635198    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qwhn2\" (UniqueName: \"kubernetes.io/projected/d2205171-a2e0-4934-8bcf-f6931285bca5-kube-api-access-qwhn2\") pod \"d2205171-a2e0-4934-8bcf-f6931285bca5\" (UID: \"d2205171-a2e0-4934-8bcf-f6931285bca5\") "
Sep 10 15:56:20.635540 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:56:20.635518    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d2205171-a2e0-4934-8bcf-f6931285bca5-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "d2205171-a2e0-4934-8bcf-f6931285bca5" (UID: "d2205171-a2e0-4934-8bcf-f6931285bca5"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 15:56:20.637245 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:56:20.637211    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d2205171-a2e0-4934-8bcf-f6931285bca5-kube-api-access-qwhn2" (OuterVolumeSpecName: "kube-api-access-qwhn2") pod "d2205171-a2e0-4934-8bcf-f6931285bca5" (UID: "d2205171-a2e0-4934-8bcf-f6931285bca5"). InnerVolumeSpecName "kube-api-access-qwhn2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 15:56:20.736722 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:56:20.736683    2567 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d2205171-a2e0-4934-8bcf-f6931285bca5-plugins-config-volume\") on node \"ip-10-0-128-241.ec2.internal\" DevicePath \"\""
Sep 10 15:56:20.736722 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:56:20.736714    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qwhn2\" (UniqueName: \"kubernetes.io/projected/d2205171-a2e0-4934-8bcf-f6931285bca5-kube-api-access-qwhn2\") on node \"ip-10-0-128-241.ec2.internal\" DevicePath \"\""
Sep 10 15:56:20.849192 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:56:20.849094    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:56:20.949967 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:56:20.949937    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:56:21.140716 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:56:21.140635    2567 generic.go:356] "Generic (PLEG): container finished" podID="d2205171-a2e0-4934-8bcf-f6931285bca5" containerID="ffa08bd68998a1bf957b51b83d40a68f93e5c840297f3923784a8e80875dfab2" exitCode=0
Sep 10 15:56:21.140716 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:56:21.140684    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-8cb756-785d8666ff-4pbxl"
Sep 10 15:56:21.140893 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:56:21.140719    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-8cb756-785d8666ff-4pbxl" event={"ID":"d2205171-a2e0-4934-8bcf-f6931285bca5","Type":"ContainerDied","Data":"ffa08bd68998a1bf957b51b83d40a68f93e5c840297f3923784a8e80875dfab2"}
Sep 10 15:56:21.140893 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:56:21.140777    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-8cb756-785d8666ff-4pbxl" event={"ID":"d2205171-a2e0-4934-8bcf-f6931285bca5","Type":"ContainerDied","Data":"499b48bb343bbae64c7928db5605cb24ab747a23659aaf4e5bed66150de73ac4"}
Sep 10 15:56:21.140893 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:56:21.140804    2567 scope.go:117] "RemoveContainer" containerID="ffa08bd68998a1bf957b51b83d40a68f93e5c840297f3923784a8e80875dfab2"
Sep 10 15:56:21.149985 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:56:21.149964    2567 scope.go:117] "RemoveContainer" containerID="ffa08bd68998a1bf957b51b83d40a68f93e5c840297f3923784a8e80875dfab2"
Sep 10 15:56:21.150321 ip-10-0-128-241 kubenswrapper[2567]: E0910 15:56:21.150302    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ffa08bd68998a1bf957b51b83d40a68f93e5c840297f3923784a8e80875dfab2\": container with ID starting with ffa08bd68998a1bf957b51b83d40a68f93e5c840297f3923784a8e80875dfab2 not found: ID does not exist" containerID="ffa08bd68998a1bf957b51b83d40a68f93e5c840297f3923784a8e80875dfab2"
Sep 10 15:56:21.150379 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:56:21.150328    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ffa08bd68998a1bf957b51b83d40a68f93e5c840297f3923784a8e80875dfab2"} err="failed to get container status \"ffa08bd68998a1bf957b51b83d40a68f93e5c840297f3923784a8e80875dfab2\": rpc error: code = NotFound desc = could not find container \"ffa08bd68998a1bf957b51b83d40a68f93e5c840297f3923784a8e80875dfab2\": container with ID starting with ffa08bd68998a1bf957b51b83d40a68f93e5c840297f3923784a8e80875dfab2 not found: ID does not exist"
Sep 10 15:56:21.158225 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:56:21.158200    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-8cb756-785d8666ff-4pbxl"]
Sep 10 15:56:21.164358 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:56:21.164336    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-8cb756-785d8666ff-4pbxl"]
Sep 10 15:56:21.750632 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:56:21.750603    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:56:22.577138 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:56:22.577096    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:56:22.905900 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:56:22.905821    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d2205171-a2e0-4934-8bcf-f6931285bca5" path="/var/lib/kubelet/pods/d2205171-a2e0-4934-8bcf-f6931285bca5/volumes"
Sep 10 15:56:23.269108 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:56:23.269073    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:56:25.865573 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:56:25.865540    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:56:54.973891 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:56:54.973855    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-t96pm_0b9d2cda-73e3-4e7c-9e17-1a3d903d95cf/console-operator/2.log"
Sep 10 15:56:54.979453 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:56:54.979430    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-t96pm_0b9d2cda-73e3-4e7c-9e17-1a3d903d95cf/console-operator/2.log"
Sep 10 15:57:03.052865 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:57:03.052818    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:57:03.146710 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:57:03.146678    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:57:03.247427 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:57:03.247391    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:57:06.451375 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:57:06.451343    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:57:09.357420 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:57:09.357387    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:57:15.273594 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:57:15.273562    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:57:15.374082 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:57:15.374044    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:57:15.558946 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:57:15.558861    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:57:16.259587 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:57:16.259553    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:57:17.334412 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:57:17.334380    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:57:18.132433 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:57:18.132395    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:57:19.040398 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:57:19.040367    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:57:19.797106 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:57:19.797073    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:57:24.991774 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:57:24.991741    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:57:26.076556 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:57:26.076519    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:57:26.840894 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:57:26.840861    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:57:26.882099 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:57:26.882070    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-6d929c-6dd9df4dcc-mgrwh"]
Sep 10 15:57:26.882381 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:57:26.882357    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-b-6d929c-6dd9df4dcc-mgrwh" podUID="d40f9e24-8020-4a0b-96db-e18cf3b4c09b" containerName="payload-processing" containerID="cri-o://50167c682ad75f7179e899a3ab52d40828aee9cb66f16980771b93f772bad597" gracePeriod=30
Sep 10 15:57:26.891381 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:57:26.891353    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-6d929c-5f6d4fd6f4-mlxst"]
Sep 10 15:57:26.891622 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:57:26.891596    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-6d929c-5f6d4fd6f4-mlxst" podUID="ae4f3225-d4d9-4e87-ba9b-fc9ce03cf3df" containerName="payload-pre-processing" containerID="cri-o://6ba678849e371e6a1390dea8019c9d0af422996bf30ef6e889631bc6b2d03cc4" gracePeriod=30
Sep 10 15:57:26.934815 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:57:26.934783    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:57:27.006189 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:57:27.006160    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:57:27.096049 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:57:27.095912    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:57:27.162866 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:57:27.162844    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-6d929c-6dd9df4dcc-mgrwh"
Sep 10 15:57:27.165755 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:57:27.165737    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-6d929c-5f6d4fd6f4-mlxst"
Sep 10 15:57:27.177971 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:57:27.177950    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:57:27.235442 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:57:27.235411    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ae4f3225-d4d9-4e87-ba9b-fc9ce03cf3df-plugins-config-volume\") pod \"ae4f3225-d4d9-4e87-ba9b-fc9ce03cf3df\" (UID: \"ae4f3225-d4d9-4e87-ba9b-fc9ce03cf3df\") "
Sep 10 15:57:27.235442 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:57:27.235447    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-56bj6\" (UniqueName: \"kubernetes.io/projected/ae4f3225-d4d9-4e87-ba9b-fc9ce03cf3df-kube-api-access-56bj6\") pod \"ae4f3225-d4d9-4e87-ba9b-fc9ce03cf3df\" (UID: \"ae4f3225-d4d9-4e87-ba9b-fc9ce03cf3df\") "
Sep 10 15:57:27.235638 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:57:27.235484    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d40f9e24-8020-4a0b-96db-e18cf3b4c09b-plugins-config-volume\") pod \"d40f9e24-8020-4a0b-96db-e18cf3b4c09b\" (UID: \"d40f9e24-8020-4a0b-96db-e18cf3b4c09b\") "
Sep 10 15:57:27.235638 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:57:27.235534    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bllzp\" (UniqueName: \"kubernetes.io/projected/d40f9e24-8020-4a0b-96db-e18cf3b4c09b-kube-api-access-bllzp\") pod \"d40f9e24-8020-4a0b-96db-e18cf3b4c09b\" (UID: \"d40f9e24-8020-4a0b-96db-e18cf3b4c09b\") "
Sep 10 15:57:27.235878 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:57:27.235853    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ae4f3225-d4d9-4e87-ba9b-fc9ce03cf3df-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "ae4f3225-d4d9-4e87-ba9b-fc9ce03cf3df" (UID: "ae4f3225-d4d9-4e87-ba9b-fc9ce03cf3df"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 15:57:27.235915 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:57:27.235853    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d40f9e24-8020-4a0b-96db-e18cf3b4c09b-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "d40f9e24-8020-4a0b-96db-e18cf3b4c09b" (UID: "d40f9e24-8020-4a0b-96db-e18cf3b4c09b"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 15:57:27.237644 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:57:27.237623    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ae4f3225-d4d9-4e87-ba9b-fc9ce03cf3df-kube-api-access-56bj6" (OuterVolumeSpecName: "kube-api-access-56bj6") pod "ae4f3225-d4d9-4e87-ba9b-fc9ce03cf3df" (UID: "ae4f3225-d4d9-4e87-ba9b-fc9ce03cf3df"). InnerVolumeSpecName "kube-api-access-56bj6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 15:57:27.237708 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:57:27.237653    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d40f9e24-8020-4a0b-96db-e18cf3b4c09b-kube-api-access-bllzp" (OuterVolumeSpecName: "kube-api-access-bllzp") pod "d40f9e24-8020-4a0b-96db-e18cf3b4c09b" (UID: "d40f9e24-8020-4a0b-96db-e18cf3b4c09b"). InnerVolumeSpecName "kube-api-access-bllzp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 15:57:27.336345 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:57:27.336307    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bllzp\" (UniqueName: \"kubernetes.io/projected/d40f9e24-8020-4a0b-96db-e18cf3b4c09b-kube-api-access-bllzp\") on node \"ip-10-0-128-241.ec2.internal\" DevicePath \"\""
Sep 10 15:57:27.336345 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:57:27.336337    2567 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ae4f3225-d4d9-4e87-ba9b-fc9ce03cf3df-plugins-config-volume\") on node \"ip-10-0-128-241.ec2.internal\" DevicePath \"\""
Sep 10 15:57:27.336345 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:57:27.336348    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-56bj6\" (UniqueName: \"kubernetes.io/projected/ae4f3225-d4d9-4e87-ba9b-fc9ce03cf3df-kube-api-access-56bj6\") on node \"ip-10-0-128-241.ec2.internal\" DevicePath \"\""
Sep 10 15:57:27.336579 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:57:27.336357    2567 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d40f9e24-8020-4a0b-96db-e18cf3b4c09b-plugins-config-volume\") on node \"ip-10-0-128-241.ec2.internal\" DevicePath \"\""
Sep 10 15:57:27.455352 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:57:27.455321    2567 generic.go:356] "Generic (PLEG): container finished" podID="ae4f3225-d4d9-4e87-ba9b-fc9ce03cf3df" containerID="6ba678849e371e6a1390dea8019c9d0af422996bf30ef6e889631bc6b2d03cc4" exitCode=0
Sep 10 15:57:27.455550 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:57:27.455376    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-6d929c-5f6d4fd6f4-mlxst"
Sep 10 15:57:27.455550 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:57:27.455409    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-6d929c-5f6d4fd6f4-mlxst" event={"ID":"ae4f3225-d4d9-4e87-ba9b-fc9ce03cf3df","Type":"ContainerDied","Data":"6ba678849e371e6a1390dea8019c9d0af422996bf30ef6e889631bc6b2d03cc4"}
Sep 10 15:57:27.455550 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:57:27.455473    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-6d929c-5f6d4fd6f4-mlxst" event={"ID":"ae4f3225-d4d9-4e87-ba9b-fc9ce03cf3df","Type":"ContainerDied","Data":"07ed660b984a1301941dab0455233e9f903219778babc37207b808ac91f6cef5"}
Sep 10 15:57:27.455550 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:57:27.455503    2567 scope.go:117] "RemoveContainer" containerID="6ba678849e371e6a1390dea8019c9d0af422996bf30ef6e889631bc6b2d03cc4"
Sep 10 15:57:27.456909 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:57:27.456884    2567 generic.go:356] "Generic (PLEG): container finished" podID="d40f9e24-8020-4a0b-96db-e18cf3b4c09b" containerID="50167c682ad75f7179e899a3ab52d40828aee9cb66f16980771b93f772bad597" exitCode=0
Sep 10 15:57:27.457048 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:57:27.456939    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-6d929c-6dd9df4dcc-mgrwh"
Sep 10 15:57:27.457048 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:57:27.456943    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-6d929c-6dd9df4dcc-mgrwh" event={"ID":"d40f9e24-8020-4a0b-96db-e18cf3b4c09b","Type":"ContainerDied","Data":"50167c682ad75f7179e899a3ab52d40828aee9cb66f16980771b93f772bad597"}
Sep 10 15:57:27.457048 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:57:27.456985    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-6d929c-6dd9df4dcc-mgrwh" event={"ID":"d40f9e24-8020-4a0b-96db-e18cf3b4c09b","Type":"ContainerDied","Data":"3e3dda241c3174f4034a29a831d305ca0b279296a7943fef957b2f4eabe667b4"}
Sep 10 15:57:27.466587 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:57:27.466508    2567 scope.go:117] "RemoveContainer" containerID="6ba678849e371e6a1390dea8019c9d0af422996bf30ef6e889631bc6b2d03cc4"
Sep 10 15:57:27.466904 ip-10-0-128-241 kubenswrapper[2567]: E0910 15:57:27.466880    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6ba678849e371e6a1390dea8019c9d0af422996bf30ef6e889631bc6b2d03cc4\": container with ID starting with 6ba678849e371e6a1390dea8019c9d0af422996bf30ef6e889631bc6b2d03cc4 not found: ID does not exist" containerID="6ba678849e371e6a1390dea8019c9d0af422996bf30ef6e889631bc6b2d03cc4"
Sep 10 15:57:27.467001 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:57:27.466947    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6ba678849e371e6a1390dea8019c9d0af422996bf30ef6e889631bc6b2d03cc4"} err="failed to get container status \"6ba678849e371e6a1390dea8019c9d0af422996bf30ef6e889631bc6b2d03cc4\": rpc error: code = NotFound desc = could not find container \"6ba678849e371e6a1390dea8019c9d0af422996bf30ef6e889631bc6b2d03cc4\": container with ID starting with 6ba678849e371e6a1390dea8019c9d0af422996bf30ef6e889631bc6b2d03cc4 not found: ID does not exist"
Sep 10 15:57:27.467001 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:57:27.466975    2567 scope.go:117] "RemoveContainer" containerID="50167c682ad75f7179e899a3ab52d40828aee9cb66f16980771b93f772bad597"
Sep 10 15:57:27.475080 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:57:27.475063    2567 scope.go:117] "RemoveContainer" containerID="50167c682ad75f7179e899a3ab52d40828aee9cb66f16980771b93f772bad597"
Sep 10 15:57:27.475434 ip-10-0-128-241 kubenswrapper[2567]: E0910 15:57:27.475404    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"50167c682ad75f7179e899a3ab52d40828aee9cb66f16980771b93f772bad597\": container with ID starting with 50167c682ad75f7179e899a3ab52d40828aee9cb66f16980771b93f772bad597 not found: ID does not exist" containerID="50167c682ad75f7179e899a3ab52d40828aee9cb66f16980771b93f772bad597"
Sep 10 15:57:27.475519 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:57:27.475438    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"50167c682ad75f7179e899a3ab52d40828aee9cb66f16980771b93f772bad597"} err="failed to get container status \"50167c682ad75f7179e899a3ab52d40828aee9cb66f16980771b93f772bad597\": rpc error: code = NotFound desc = could not find container \"50167c682ad75f7179e899a3ab52d40828aee9cb66f16980771b93f772bad597\": container with ID starting with 50167c682ad75f7179e899a3ab52d40828aee9cb66f16980771b93f772bad597 not found: ID does not exist"
Sep 10 15:57:27.486695 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:57:27.484246    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-6d929c-5f6d4fd6f4-mlxst"]
Sep 10 15:57:27.491153 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:57:27.491120    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-6d929c-5f6d4fd6f4-mlxst"]
Sep 10 15:57:27.501109 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:57:27.501088    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-6d929c-6dd9df4dcc-mgrwh"]
Sep 10 15:57:27.505449 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:57:27.505425    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:57:27.505923 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:57:27.505907    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-6d929c-6dd9df4dcc-mgrwh"]
Sep 10 15:57:27.605468 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:57:27.605440    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:57:28.360179 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:57:28.360146    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:57:28.903745 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:57:28.903714    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ae4f3225-d4d9-4e87-ba9b-fc9ce03cf3df" path="/var/lib/kubelet/pods/ae4f3225-d4d9-4e87-ba9b-fc9ce03cf3df/volumes"
Sep 10 15:57:28.904083 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:57:28.904070    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d40f9e24-8020-4a0b-96db-e18cf3b4c09b" path="/var/lib/kubelet/pods/d40f9e24-8020-4a0b-96db-e18cf3b4c09b/volumes"
Sep 10 15:57:29.297036 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:57:29.296998    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:57:32.025262 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:57:32.025214    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:57:34.213625 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:57:34.213589    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:57:34.285398 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:57:34.285361    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:57:34.478711 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:57:34.478625    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:57:35.636289 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:57:35.636247    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:57:36.524441 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:57:36.524411    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:57:37.307742 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:57:37.307697    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:57:45.462372 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:57:45.462340    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-c7257b-7bcfbc88b8-lkjp7"]
Sep 10 15:57:45.462827 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:57:45.462563    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-c7257b-7bcfbc88b8-lkjp7" podUID="ee2c75eb-0049-4111-94f7-39cc5e7896d1" containerName="maas-api" containerID="cri-o://41726d1bc8d00639351e629004e5c78f8f0b705edda01d693deb4f56ffa2b3e5" gracePeriod=30
Sep 10 15:57:45.507099 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:57:45.507061    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:57:45.570638 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:57:45.570607    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-c7257b-5569d4b646-7pn87"]
Sep 10 15:57:45.571668 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:57:45.571058    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-c7257b-5569d4b646-7pn87" podUID="9deb994d-b35c-4651-bb7b-aba81f779366" containerName="payload-pre-processing" containerID="cri-o://c400a209fcada5676853c2ec9d2a389f9a20fe63679baa4a3ae1d16eb5508b4d" gracePeriod=30
Sep 10 15:57:45.714842 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:57:45.714777    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-c7257b-7bcfbc88b8-lkjp7"
Sep 10 15:57:45.804972 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:57:45.804933    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ee2c75eb-0049-4111-94f7-39cc5e7896d1-maas-api-tls\") pod \"ee2c75eb-0049-4111-94f7-39cc5e7896d1\" (UID: \"ee2c75eb-0049-4111-94f7-39cc5e7896d1\") "
Sep 10 15:57:45.805173 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:57:45.805016    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vh8tf\" (UniqueName: \"kubernetes.io/projected/ee2c75eb-0049-4111-94f7-39cc5e7896d1-kube-api-access-vh8tf\") pod \"ee2c75eb-0049-4111-94f7-39cc5e7896d1\" (UID: \"ee2c75eb-0049-4111-94f7-39cc5e7896d1\") "
Sep 10 15:57:45.805173 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:57:45.805045    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ee2c75eb-0049-4111-94f7-39cc5e7896d1-openshift-service-ca\") pod \"ee2c75eb-0049-4111-94f7-39cc5e7896d1\" (UID: \"ee2c75eb-0049-4111-94f7-39cc5e7896d1\") "
Sep 10 15:57:45.805173 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:57:45.805128    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ee2c75eb-0049-4111-94f7-39cc5e7896d1-metrics-certs\") pod \"ee2c75eb-0049-4111-94f7-39cc5e7896d1\" (UID: \"ee2c75eb-0049-4111-94f7-39cc5e7896d1\") "
Sep 10 15:57:45.805547 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:57:45.805518    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ee2c75eb-0049-4111-94f7-39cc5e7896d1-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "ee2c75eb-0049-4111-94f7-39cc5e7896d1" (UID: "ee2c75eb-0049-4111-94f7-39cc5e7896d1"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 15:57:45.807499 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:57:45.807469    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ee2c75eb-0049-4111-94f7-39cc5e7896d1-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "ee2c75eb-0049-4111-94f7-39cc5e7896d1" (UID: "ee2c75eb-0049-4111-94f7-39cc5e7896d1"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 15:57:45.807619 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:57:45.807547    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ee2c75eb-0049-4111-94f7-39cc5e7896d1-kube-api-access-vh8tf" (OuterVolumeSpecName: "kube-api-access-vh8tf") pod "ee2c75eb-0049-4111-94f7-39cc5e7896d1" (UID: "ee2c75eb-0049-4111-94f7-39cc5e7896d1"). InnerVolumeSpecName "kube-api-access-vh8tf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 15:57:45.807715 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:57:45.807687    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ee2c75eb-0049-4111-94f7-39cc5e7896d1-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "ee2c75eb-0049-4111-94f7-39cc5e7896d1" (UID: "ee2c75eb-0049-4111-94f7-39cc5e7896d1"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 15:57:45.825658 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:57:45.825633    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-c7257b-5569d4b646-7pn87"
Sep 10 15:57:45.906494 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:57:45.906462    2567 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ee2c75eb-0049-4111-94f7-39cc5e7896d1-metrics-certs\") on node \"ip-10-0-128-241.ec2.internal\" DevicePath \"\""
Sep 10 15:57:45.906494 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:57:45.906491    2567 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ee2c75eb-0049-4111-94f7-39cc5e7896d1-maas-api-tls\") on node \"ip-10-0-128-241.ec2.internal\" DevicePath \"\""
Sep 10 15:57:45.906494 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:57:45.906502    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vh8tf\" (UniqueName: \"kubernetes.io/projected/ee2c75eb-0049-4111-94f7-39cc5e7896d1-kube-api-access-vh8tf\") on node \"ip-10-0-128-241.ec2.internal\" DevicePath \"\""
Sep 10 15:57:45.906725 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:57:45.906513    2567 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ee2c75eb-0049-4111-94f7-39cc5e7896d1-openshift-service-ca\") on node \"ip-10-0-128-241.ec2.internal\" DevicePath \"\""
Sep 10 15:57:46.007558 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:57:46.007474    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-865hl\" (UniqueName: \"kubernetes.io/projected/9deb994d-b35c-4651-bb7b-aba81f779366-kube-api-access-865hl\") pod \"9deb994d-b35c-4651-bb7b-aba81f779366\" (UID: \"9deb994d-b35c-4651-bb7b-aba81f779366\") "
Sep 10 15:57:46.007558 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:57:46.007519    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9deb994d-b35c-4651-bb7b-aba81f779366-plugins-config-volume\") pod \"9deb994d-b35c-4651-bb7b-aba81f779366\" (UID: \"9deb994d-b35c-4651-bb7b-aba81f779366\") "
Sep 10 15:57:46.007913 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:57:46.007882    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9deb994d-b35c-4651-bb7b-aba81f779366-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "9deb994d-b35c-4651-bb7b-aba81f779366" (UID: "9deb994d-b35c-4651-bb7b-aba81f779366"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 15:57:46.009578 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:57:46.009555    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9deb994d-b35c-4651-bb7b-aba81f779366-kube-api-access-865hl" (OuterVolumeSpecName: "kube-api-access-865hl") pod "9deb994d-b35c-4651-bb7b-aba81f779366" (UID: "9deb994d-b35c-4651-bb7b-aba81f779366"). InnerVolumeSpecName "kube-api-access-865hl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 15:57:46.108893 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:57:46.108856    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-865hl\" (UniqueName: \"kubernetes.io/projected/9deb994d-b35c-4651-bb7b-aba81f779366-kube-api-access-865hl\") on node \"ip-10-0-128-241.ec2.internal\" DevicePath \"\""
Sep 10 15:57:46.108893 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:57:46.108882    2567 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9deb994d-b35c-4651-bb7b-aba81f779366-plugins-config-volume\") on node \"ip-10-0-128-241.ec2.internal\" DevicePath \"\""
Sep 10 15:57:46.545863 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:57:46.545824    2567 generic.go:356] "Generic (PLEG): container finished" podID="9deb994d-b35c-4651-bb7b-aba81f779366" containerID="c400a209fcada5676853c2ec9d2a389f9a20fe63679baa4a3ae1d16eb5508b4d" exitCode=0
Sep 10 15:57:46.546361 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:57:46.545869    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-c7257b-5569d4b646-7pn87"
Sep 10 15:57:46.546361 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:57:46.545907    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-c7257b-5569d4b646-7pn87" event={"ID":"9deb994d-b35c-4651-bb7b-aba81f779366","Type":"ContainerDied","Data":"c400a209fcada5676853c2ec9d2a389f9a20fe63679baa4a3ae1d16eb5508b4d"}
Sep 10 15:57:46.546361 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:57:46.545969    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-c7257b-5569d4b646-7pn87" event={"ID":"9deb994d-b35c-4651-bb7b-aba81f779366","Type":"ContainerDied","Data":"c444c98fd37cd0c9da1e437fd96bcde12dff23234c78be97342875cc24f2a7bf"}
Sep 10 15:57:46.546361 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:57:46.546001    2567 scope.go:117] "RemoveContainer" containerID="c400a209fcada5676853c2ec9d2a389f9a20fe63679baa4a3ae1d16eb5508b4d"
Sep 10 15:57:46.547335 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:57:46.547262    2567 generic.go:356] "Generic (PLEG): container finished" podID="ee2c75eb-0049-4111-94f7-39cc5e7896d1" containerID="41726d1bc8d00639351e629004e5c78f8f0b705edda01d693deb4f56ffa2b3e5" exitCode=0
Sep 10 15:57:46.547335 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:57:46.547316    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-c7257b-7bcfbc88b8-lkjp7"
Sep 10 15:57:46.547467 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:57:46.547360    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-c7257b-7bcfbc88b8-lkjp7" event={"ID":"ee2c75eb-0049-4111-94f7-39cc5e7896d1","Type":"ContainerDied","Data":"41726d1bc8d00639351e629004e5c78f8f0b705edda01d693deb4f56ffa2b3e5"}
Sep 10 15:57:46.547467 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:57:46.547412    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-c7257b-7bcfbc88b8-lkjp7" event={"ID":"ee2c75eb-0049-4111-94f7-39cc5e7896d1","Type":"ContainerDied","Data":"80486dbb2b6aff80f3a1eb9e9f775bcf1c3891ec4581425eefeec1458fb21df1"}
Sep 10 15:57:46.554824 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:57:46.554807    2567 scope.go:117] "RemoveContainer" containerID="c400a209fcada5676853c2ec9d2a389f9a20fe63679baa4a3ae1d16eb5508b4d"
Sep 10 15:57:46.555206 ip-10-0-128-241 kubenswrapper[2567]: E0910 15:57:46.555087    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c400a209fcada5676853c2ec9d2a389f9a20fe63679baa4a3ae1d16eb5508b4d\": container with ID starting with c400a209fcada5676853c2ec9d2a389f9a20fe63679baa4a3ae1d16eb5508b4d not found: ID does not exist" containerID="c400a209fcada5676853c2ec9d2a389f9a20fe63679baa4a3ae1d16eb5508b4d"
Sep 10 15:57:46.555206 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:57:46.555118    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c400a209fcada5676853c2ec9d2a389f9a20fe63679baa4a3ae1d16eb5508b4d"} err="failed to get container status \"c400a209fcada5676853c2ec9d2a389f9a20fe63679baa4a3ae1d16eb5508b4d\": rpc error: code = NotFound desc = could not find container \"c400a209fcada5676853c2ec9d2a389f9a20fe63679baa4a3ae1d16eb5508b4d\": container with ID starting with c400a209fcada5676853c2ec9d2a389f9a20fe63679baa4a3ae1d16eb5508b4d not found: ID does not exist"
Sep 10 15:57:46.555206 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:57:46.555140    2567 scope.go:117] "RemoveContainer" containerID="41726d1bc8d00639351e629004e5c78f8f0b705edda01d693deb4f56ffa2b3e5"
Sep 10 15:57:46.563524 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:57:46.563510    2567 scope.go:117] "RemoveContainer" containerID="41726d1bc8d00639351e629004e5c78f8f0b705edda01d693deb4f56ffa2b3e5"
Sep 10 15:57:46.563786 ip-10-0-128-241 kubenswrapper[2567]: E0910 15:57:46.563769    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"41726d1bc8d00639351e629004e5c78f8f0b705edda01d693deb4f56ffa2b3e5\": container with ID starting with 41726d1bc8d00639351e629004e5c78f8f0b705edda01d693deb4f56ffa2b3e5 not found: ID does not exist" containerID="41726d1bc8d00639351e629004e5c78f8f0b705edda01d693deb4f56ffa2b3e5"
Sep 10 15:57:46.563826 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:57:46.563794    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"41726d1bc8d00639351e629004e5c78f8f0b705edda01d693deb4f56ffa2b3e5"} err="failed to get container status \"41726d1bc8d00639351e629004e5c78f8f0b705edda01d693deb4f56ffa2b3e5\": rpc error: code = NotFound desc = could not find container \"41726d1bc8d00639351e629004e5c78f8f0b705edda01d693deb4f56ffa2b3e5\": container with ID starting with 41726d1bc8d00639351e629004e5c78f8f0b705edda01d693deb4f56ffa2b3e5 not found: ID does not exist"
Sep 10 15:57:46.579320 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:57:46.579301    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-c7257b-7bcfbc88b8-lkjp7"]
Sep 10 15:57:46.582509 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:57:46.582487    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-c7257b-7bcfbc88b8-lkjp7"]
Sep 10 15:57:46.598406 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:57:46.598383    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-c7257b-5569d4b646-7pn87"]
Sep 10 15:57:46.607539 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:57:46.607519    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-c7257b-5569d4b646-7pn87"]
Sep 10 15:57:46.827031 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:57:46.826934    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:57:46.904500 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:57:46.904469    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9deb994d-b35c-4651-bb7b-aba81f779366" path="/var/lib/kubelet/pods/9deb994d-b35c-4651-bb7b-aba81f779366/volumes"
Sep 10 15:57:46.904863 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:57:46.904851    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ee2c75eb-0049-4111-94f7-39cc5e7896d1" path="/var/lib/kubelet/pods/ee2c75eb-0049-4111-94f7-39cc5e7896d1/volumes"
Sep 10 15:57:47.347493 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:57:47.347457    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:57:48.344870 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:57:48.344834    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:57:53.037219 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:57:53.037169    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:57:53.103584 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:57:53.103554    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:57:53.318967 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:57:53.318881    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:57:53.849538 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:57:53.849507    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:57:54.770020 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:57:54.769986    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:57:55.664514 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:57:55.664419    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:58:04.182694 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:58:04.182631    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-3650b0-5bb8755959-pqcth"]
Sep 10 15:58:04.183280 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:58:04.182895    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-3650b0-5bb8755959-pqcth" podUID="524402b2-7893-418c-8e4b-2364a39cad2c" containerName="maas-api" containerID="cri-o://0a554680fc168b8aaf434aae533a085c84d8d9db61d543e2ed7151982f3a5da1" gracePeriod=30
Sep 10 15:58:04.213507 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:58:04.213477    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:58:04.255702 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:58:04.255669    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-3650b0-599b79859d-w2qxr"]
Sep 10 15:58:04.255946 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:58:04.255922    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-3650b0-599b79859d-w2qxr" podUID="738e3d0d-8d31-4cec-bfe3-efa3c955d119" containerName="payload-pre-processing" containerID="cri-o://1f52ae7592e282c35b9586da976b93583c9ed17c66c1eb5883c6fe45afa02c8d" gracePeriod=30
Sep 10 15:58:04.314794 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:58:04.314767    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:58:04.332926 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:58:04.332890    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-3650b0-599b79859d-w2qxr" podUID="738e3d0d-8d31-4cec-bfe3-efa3c955d119" containerName="payload-pre-processing" probeResult="failure" output="dial tcp 10.132.0.59:9004: connect: connection refused"
Sep 10 15:58:04.413962 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:58:04.413910    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:58:04.479654 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:58:04.479544    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:58:04.498984 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:58:04.498957    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-3650b0-5bb8755959-pqcth"
Sep 10 15:58:04.529541 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:58:04.529496    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-3650b0-599b79859d-w2qxr"
Sep 10 15:58:04.570355 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:58:04.570316    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9lnkt\" (UniqueName: \"kubernetes.io/projected/738e3d0d-8d31-4cec-bfe3-efa3c955d119-kube-api-access-9lnkt\") pod \"738e3d0d-8d31-4cec-bfe3-efa3c955d119\" (UID: \"738e3d0d-8d31-4cec-bfe3-efa3c955d119\") "
Sep 10 15:58:04.570543 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:58:04.570362    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/524402b2-7893-418c-8e4b-2364a39cad2c-maas-api-tls\") pod \"524402b2-7893-418c-8e4b-2364a39cad2c\" (UID: \"524402b2-7893-418c-8e4b-2364a39cad2c\") "
Sep 10 15:58:04.570543 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:58:04.570388    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/524402b2-7893-418c-8e4b-2364a39cad2c-openshift-service-ca\") pod \"524402b2-7893-418c-8e4b-2364a39cad2c\" (UID: \"524402b2-7893-418c-8e4b-2364a39cad2c\") "
Sep 10 15:58:04.570543 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:58:04.570443    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/738e3d0d-8d31-4cec-bfe3-efa3c955d119-plugins-config-volume\") pod \"738e3d0d-8d31-4cec-bfe3-efa3c955d119\" (UID: \"738e3d0d-8d31-4cec-bfe3-efa3c955d119\") "
Sep 10 15:58:04.570543 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:58:04.570470    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/524402b2-7893-418c-8e4b-2364a39cad2c-metrics-certs\") pod \"524402b2-7893-418c-8e4b-2364a39cad2c\" (UID: \"524402b2-7893-418c-8e4b-2364a39cad2c\") "
Sep 10 15:58:04.570774 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:58:04.570559    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-782j9\" (UniqueName: \"kubernetes.io/projected/524402b2-7893-418c-8e4b-2364a39cad2c-kube-api-access-782j9\") pod \"524402b2-7893-418c-8e4b-2364a39cad2c\" (UID: \"524402b2-7893-418c-8e4b-2364a39cad2c\") "
Sep 10 15:58:04.571124 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:58:04.571089    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/738e3d0d-8d31-4cec-bfe3-efa3c955d119-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "738e3d0d-8d31-4cec-bfe3-efa3c955d119" (UID: "738e3d0d-8d31-4cec-bfe3-efa3c955d119"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 15:58:04.571494 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:58:04.571458    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/524402b2-7893-418c-8e4b-2364a39cad2c-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "524402b2-7893-418c-8e4b-2364a39cad2c" (UID: "524402b2-7893-418c-8e4b-2364a39cad2c"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 15:58:04.573088 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:58:04.573054    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/524402b2-7893-418c-8e4b-2364a39cad2c-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "524402b2-7893-418c-8e4b-2364a39cad2c" (UID: "524402b2-7893-418c-8e4b-2364a39cad2c"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 15:58:04.573190 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:58:04.573064    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/738e3d0d-8d31-4cec-bfe3-efa3c955d119-kube-api-access-9lnkt" (OuterVolumeSpecName: "kube-api-access-9lnkt") pod "738e3d0d-8d31-4cec-bfe3-efa3c955d119" (UID: "738e3d0d-8d31-4cec-bfe3-efa3c955d119"). InnerVolumeSpecName "kube-api-access-9lnkt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 15:58:04.573190 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:58:04.573091    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/524402b2-7893-418c-8e4b-2364a39cad2c-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "524402b2-7893-418c-8e4b-2364a39cad2c" (UID: "524402b2-7893-418c-8e4b-2364a39cad2c"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 15:58:04.573468 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:58:04.573443    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/524402b2-7893-418c-8e4b-2364a39cad2c-kube-api-access-782j9" (OuterVolumeSpecName: "kube-api-access-782j9") pod "524402b2-7893-418c-8e4b-2364a39cad2c" (UID: "524402b2-7893-418c-8e4b-2364a39cad2c"). InnerVolumeSpecName "kube-api-access-782j9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 15:58:04.631787 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:58:04.631756    2567 generic.go:356] "Generic (PLEG): container finished" podID="738e3d0d-8d31-4cec-bfe3-efa3c955d119" containerID="1f52ae7592e282c35b9586da976b93583c9ed17c66c1eb5883c6fe45afa02c8d" exitCode=0
Sep 10 15:58:04.631916 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:58:04.631799    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-3650b0-599b79859d-w2qxr" event={"ID":"738e3d0d-8d31-4cec-bfe3-efa3c955d119","Type":"ContainerDied","Data":"1f52ae7592e282c35b9586da976b93583c9ed17c66c1eb5883c6fe45afa02c8d"}
Sep 10 15:58:04.631916 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:58:04.631859    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-3650b0-599b79859d-w2qxr" event={"ID":"738e3d0d-8d31-4cec-bfe3-efa3c955d119","Type":"ContainerDied","Data":"7c7511aed79574d8fda1a6b43518aec12f40e58d1c42c197aa8c61e25afccaf5"}
Sep 10 15:58:04.631916 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:58:04.631803    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-3650b0-599b79859d-w2qxr"
Sep 10 15:58:04.632044 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:58:04.631890    2567 scope.go:117] "RemoveContainer" containerID="1f52ae7592e282c35b9586da976b93583c9ed17c66c1eb5883c6fe45afa02c8d"
Sep 10 15:58:04.633456 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:58:04.633430    2567 generic.go:356] "Generic (PLEG): container finished" podID="524402b2-7893-418c-8e4b-2364a39cad2c" containerID="0a554680fc168b8aaf434aae533a085c84d8d9db61d543e2ed7151982f3a5da1" exitCode=0
Sep 10 15:58:04.633579 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:58:04.633473    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-3650b0-5bb8755959-pqcth"
Sep 10 15:58:04.633579 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:58:04.633483    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-3650b0-5bb8755959-pqcth" event={"ID":"524402b2-7893-418c-8e4b-2364a39cad2c","Type":"ContainerDied","Data":"0a554680fc168b8aaf434aae533a085c84d8d9db61d543e2ed7151982f3a5da1"}
Sep 10 15:58:04.633579 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:58:04.633522    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-3650b0-5bb8755959-pqcth" event={"ID":"524402b2-7893-418c-8e4b-2364a39cad2c","Type":"ContainerDied","Data":"fa77b53ff966b53912124c1993a520217080666f7f2d906c1392fa443b098e4a"}
Sep 10 15:58:04.640997 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:58:04.640971    2567 scope.go:117] "RemoveContainer" containerID="1f52ae7592e282c35b9586da976b93583c9ed17c66c1eb5883c6fe45afa02c8d"
Sep 10 15:58:04.641305 ip-10-0-128-241 kubenswrapper[2567]: E0910 15:58:04.641285    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1f52ae7592e282c35b9586da976b93583c9ed17c66c1eb5883c6fe45afa02c8d\": container with ID starting with 1f52ae7592e282c35b9586da976b93583c9ed17c66c1eb5883c6fe45afa02c8d not found: ID does not exist" containerID="1f52ae7592e282c35b9586da976b93583c9ed17c66c1eb5883c6fe45afa02c8d"
Sep 10 15:58:04.641376 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:58:04.641312    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1f52ae7592e282c35b9586da976b93583c9ed17c66c1eb5883c6fe45afa02c8d"} err="failed to get container status \"1f52ae7592e282c35b9586da976b93583c9ed17c66c1eb5883c6fe45afa02c8d\": rpc error: code = NotFound desc = could not find container \"1f52ae7592e282c35b9586da976b93583c9ed17c66c1eb5883c6fe45afa02c8d\": container with ID starting with 1f52ae7592e282c35b9586da976b93583c9ed17c66c1eb5883c6fe45afa02c8d not found: ID does not exist"
Sep 10 15:58:04.641376 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:58:04.641329    2567 scope.go:117] "RemoveContainer" containerID="0a554680fc168b8aaf434aae533a085c84d8d9db61d543e2ed7151982f3a5da1"
Sep 10 15:58:04.649075 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:58:04.649059    2567 scope.go:117] "RemoveContainer" containerID="0a554680fc168b8aaf434aae533a085c84d8d9db61d543e2ed7151982f3a5da1"
Sep 10 15:58:04.649389 ip-10-0-128-241 kubenswrapper[2567]: E0910 15:58:04.649366    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0a554680fc168b8aaf434aae533a085c84d8d9db61d543e2ed7151982f3a5da1\": container with ID starting with 0a554680fc168b8aaf434aae533a085c84d8d9db61d543e2ed7151982f3a5da1 not found: ID does not exist" containerID="0a554680fc168b8aaf434aae533a085c84d8d9db61d543e2ed7151982f3a5da1"
Sep 10 15:58:04.649446 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:58:04.649400    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0a554680fc168b8aaf434aae533a085c84d8d9db61d543e2ed7151982f3a5da1"} err="failed to get container status \"0a554680fc168b8aaf434aae533a085c84d8d9db61d543e2ed7151982f3a5da1\": rpc error: code = NotFound desc = could not find container \"0a554680fc168b8aaf434aae533a085c84d8d9db61d543e2ed7151982f3a5da1\": container with ID starting with 0a554680fc168b8aaf434aae533a085c84d8d9db61d543e2ed7151982f3a5da1 not found: ID does not exist"
Sep 10 15:58:04.658038 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:58:04.658013    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-3650b0-5bb8755959-pqcth"]
Sep 10 15:58:04.665117 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:58:04.665096    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-3650b0-5bb8755959-pqcth"]
Sep 10 15:58:04.671439 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:58:04.671421    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9lnkt\" (UniqueName: \"kubernetes.io/projected/738e3d0d-8d31-4cec-bfe3-efa3c955d119-kube-api-access-9lnkt\") on node \"ip-10-0-128-241.ec2.internal\" DevicePath \"\""
Sep 10 15:58:04.671521 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:58:04.671440    2567 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/524402b2-7893-418c-8e4b-2364a39cad2c-maas-api-tls\") on node \"ip-10-0-128-241.ec2.internal\" DevicePath \"\""
Sep 10 15:58:04.671521 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:58:04.671451    2567 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/524402b2-7893-418c-8e4b-2364a39cad2c-openshift-service-ca\") on node \"ip-10-0-128-241.ec2.internal\" DevicePath \"\""
Sep 10 15:58:04.671521 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:58:04.671460    2567 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/738e3d0d-8d31-4cec-bfe3-efa3c955d119-plugins-config-volume\") on node \"ip-10-0-128-241.ec2.internal\" DevicePath \"\""
Sep 10 15:58:04.671521 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:58:04.671468    2567 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/524402b2-7893-418c-8e4b-2364a39cad2c-metrics-certs\") on node \"ip-10-0-128-241.ec2.internal\" DevicePath \"\""
Sep 10 15:58:04.671521 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:58:04.671476    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-782j9\" (UniqueName: \"kubernetes.io/projected/524402b2-7893-418c-8e4b-2364a39cad2c-kube-api-access-782j9\") on node \"ip-10-0-128-241.ec2.internal\" DevicePath \"\""
Sep 10 15:58:04.677725 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:58:04.677705    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-3650b0-599b79859d-w2qxr"]
Sep 10 15:58:04.685140 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:58:04.685112    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-3650b0-599b79859d-w2qxr"]
Sep 10 15:58:04.820201 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:58:04.820107    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:58:04.903730 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:58:04.903697    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="524402b2-7893-418c-8e4b-2364a39cad2c" path="/var/lib/kubelet/pods/524402b2-7893-418c-8e4b-2364a39cad2c/volumes"
Sep 10 15:58:04.904069 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:58:04.904055    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="738e3d0d-8d31-4cec-bfe3-efa3c955d119" path="/var/lib/kubelet/pods/738e3d0d-8d31-4cec-bfe3-efa3c955d119/volumes"
Sep 10 15:58:04.919944 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:58:04.919920    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:58:06.135359 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:58:06.135325    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:58:11.789922 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:58:11.789889    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:58:11.889101 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:58:11.889052    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:58:12.083572 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:58:12.083473    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:58:14.782309 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:58:14.782269    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-68569f6669-754p8"]
Sep 10 15:58:14.782755 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:58:14.782530    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-68569f6669-754p8" podUID="0181721a-e1e2-4991-8970-8073aebda063" containerName="manager" containerID="cri-o://b6dce6c6bfa1cca606aa45b3058bf43600088ab879ea6f93b005c794f5a94ead" gracePeriod=30
Sep 10 15:58:15.026551 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:58:15.026523    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-68569f6669-754p8"
Sep 10 15:58:15.161014 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:58:15.160925    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/0181721a-e1e2-4991-8970-8073aebda063-webhook-cert\") pod \"0181721a-e1e2-4991-8970-8073aebda063\" (UID: \"0181721a-e1e2-4991-8970-8073aebda063\") "
Sep 10 15:58:15.161014 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:58:15.161002    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0181721a-e1e2-4991-8970-8073aebda063-metrics-certs\") pod \"0181721a-e1e2-4991-8970-8073aebda063\" (UID: \"0181721a-e1e2-4991-8970-8073aebda063\") "
Sep 10 15:58:15.161204 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:58:15.161033    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b6wbq\" (UniqueName: \"kubernetes.io/projected/0181721a-e1e2-4991-8970-8073aebda063-kube-api-access-b6wbq\") pod \"0181721a-e1e2-4991-8970-8073aebda063\" (UID: \"0181721a-e1e2-4991-8970-8073aebda063\") "
Sep 10 15:58:15.163038 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:58:15.163003    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0181721a-e1e2-4991-8970-8073aebda063-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "0181721a-e1e2-4991-8970-8073aebda063" (UID: "0181721a-e1e2-4991-8970-8073aebda063"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 15:58:15.163177 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:58:15.163157    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0181721a-e1e2-4991-8970-8073aebda063-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "0181721a-e1e2-4991-8970-8073aebda063" (UID: "0181721a-e1e2-4991-8970-8073aebda063"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 15:58:15.163226 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:58:15.163180    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0181721a-e1e2-4991-8970-8073aebda063-kube-api-access-b6wbq" (OuterVolumeSpecName: "kube-api-access-b6wbq") pod "0181721a-e1e2-4991-8970-8073aebda063" (UID: "0181721a-e1e2-4991-8970-8073aebda063"). InnerVolumeSpecName "kube-api-access-b6wbq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 15:58:15.262323 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:58:15.262299    2567 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/0181721a-e1e2-4991-8970-8073aebda063-webhook-cert\") on node \"ip-10-0-128-241.ec2.internal\" DevicePath \"\""
Sep 10 15:58:15.262323 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:58:15.262320    2567 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0181721a-e1e2-4991-8970-8073aebda063-metrics-certs\") on node \"ip-10-0-128-241.ec2.internal\" DevicePath \"\""
Sep 10 15:58:15.262323 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:58:15.262330    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b6wbq\" (UniqueName: \"kubernetes.io/projected/0181721a-e1e2-4991-8970-8073aebda063-kube-api-access-b6wbq\") on node \"ip-10-0-128-241.ec2.internal\" DevicePath \"\""
Sep 10 15:58:15.684941 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:58:15.684904    2567 generic.go:356] "Generic (PLEG): container finished" podID="0181721a-e1e2-4991-8970-8073aebda063" containerID="b6dce6c6bfa1cca606aa45b3058bf43600088ab879ea6f93b005c794f5a94ead" exitCode=0
Sep 10 15:58:15.685118 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:58:15.684952    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-68569f6669-754p8"
Sep 10 15:58:15.685118 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:58:15.684956    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-68569f6669-754p8" event={"ID":"0181721a-e1e2-4991-8970-8073aebda063","Type":"ContainerDied","Data":"b6dce6c6bfa1cca606aa45b3058bf43600088ab879ea6f93b005c794f5a94ead"}
Sep 10 15:58:15.685118 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:58:15.685074    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-68569f6669-754p8" event={"ID":"0181721a-e1e2-4991-8970-8073aebda063","Type":"ContainerDied","Data":"6f50448d35d6e03baec64ff8911265e5f473372242f18e9c372ce92725533bf1"}
Sep 10 15:58:15.685118 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:58:15.685099    2567 scope.go:117] "RemoveContainer" containerID="b6dce6c6bfa1cca606aa45b3058bf43600088ab879ea6f93b005c794f5a94ead"
Sep 10 15:58:15.694430 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:58:15.694414    2567 scope.go:117] "RemoveContainer" containerID="b6dce6c6bfa1cca606aa45b3058bf43600088ab879ea6f93b005c794f5a94ead"
Sep 10 15:58:15.694692 ip-10-0-128-241 kubenswrapper[2567]: E0910 15:58:15.694673    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b6dce6c6bfa1cca606aa45b3058bf43600088ab879ea6f93b005c794f5a94ead\": container with ID starting with b6dce6c6bfa1cca606aa45b3058bf43600088ab879ea6f93b005c794f5a94ead not found: ID does not exist" containerID="b6dce6c6bfa1cca606aa45b3058bf43600088ab879ea6f93b005c794f5a94ead"
Sep 10 15:58:15.694734 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:58:15.694701    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b6dce6c6bfa1cca606aa45b3058bf43600088ab879ea6f93b005c794f5a94ead"} err="failed to get container status \"b6dce6c6bfa1cca606aa45b3058bf43600088ab879ea6f93b005c794f5a94ead\": rpc error: code = NotFound desc = could not find container \"b6dce6c6bfa1cca606aa45b3058bf43600088ab879ea6f93b005c794f5a94ead\": container with ID starting with b6dce6c6bfa1cca606aa45b3058bf43600088ab879ea6f93b005c794f5a94ead not found: ID does not exist"
Sep 10 15:58:15.708312 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:58:15.708290    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-68569f6669-754p8"]
Sep 10 15:58:15.717157 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:58:15.717135    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-68569f6669-754p8"]
Sep 10 15:58:16.908937 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:58:16.908895    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0181721a-e1e2-4991-8970-8073aebda063" path="/var/lib/kubelet/pods/0181721a-e1e2-4991-8970-8073aebda063/volumes"
Sep 10 15:59:02.469982 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:59:02.469946    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:59:02.575783 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:59:02.575749    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:59:09.441880 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:59:09.441846    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:59:16.524026 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:59:16.523988    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-68569f6669-zvk8l"]
Sep 10 15:59:16.524427 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:59:16.524319    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d2205171-a2e0-4934-8bcf-f6931285bca5" containerName="payload-processing"
Sep 10 15:59:16.524427 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:59:16.524330    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="d2205171-a2e0-4934-8bcf-f6931285bca5" containerName="payload-processing"
Sep 10 15:59:16.524427 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:59:16.524341    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d40f9e24-8020-4a0b-96db-e18cf3b4c09b" containerName="payload-processing"
Sep 10 15:59:16.524427 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:59:16.524346    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="d40f9e24-8020-4a0b-96db-e18cf3b4c09b" containerName="payload-processing"
Sep 10 15:59:16.524427 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:59:16.524360    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="738e3d0d-8d31-4cec-bfe3-efa3c955d119" containerName="payload-pre-processing"
Sep 10 15:59:16.524427 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:59:16.524366    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="738e3d0d-8d31-4cec-bfe3-efa3c955d119" containerName="payload-pre-processing"
Sep 10 15:59:16.524427 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:59:16.524373    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="524402b2-7893-418c-8e4b-2364a39cad2c" containerName="maas-api"
Sep 10 15:59:16.524427 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:59:16.524378    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="524402b2-7893-418c-8e4b-2364a39cad2c" containerName="maas-api"
Sep 10 15:59:16.524427 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:59:16.524385    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b25a0acb-f46d-4e62-9c5f-b47d0710cc4a" containerName="maas-api"
Sep 10 15:59:16.524427 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:59:16.524390    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="b25a0acb-f46d-4e62-9c5f-b47d0710cc4a" containerName="maas-api"
Sep 10 15:59:16.524427 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:59:16.524397    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0181721a-e1e2-4991-8970-8073aebda063" containerName="manager"
Sep 10 15:59:16.524427 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:59:16.524402    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="0181721a-e1e2-4991-8970-8073aebda063" containerName="manager"
Sep 10 15:59:16.524427 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:59:16.524415    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ae4f3225-d4d9-4e87-ba9b-fc9ce03cf3df" containerName="payload-pre-processing"
Sep 10 15:59:16.524427 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:59:16.524420    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="ae4f3225-d4d9-4e87-ba9b-fc9ce03cf3df" containerName="payload-pre-processing"
Sep 10 15:59:16.524427 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:59:16.524426    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9deb994d-b35c-4651-bb7b-aba81f779366" containerName="payload-pre-processing"
Sep 10 15:59:16.524427 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:59:16.524431    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="9deb994d-b35c-4651-bb7b-aba81f779366" containerName="payload-pre-processing"
Sep 10 15:59:16.524942 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:59:16.524438    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ee2c75eb-0049-4111-94f7-39cc5e7896d1" containerName="maas-api"
Sep 10 15:59:16.524942 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:59:16.524443    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="ee2c75eb-0049-4111-94f7-39cc5e7896d1" containerName="maas-api"
Sep 10 15:59:16.524942 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:59:16.524489    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="d40f9e24-8020-4a0b-96db-e18cf3b4c09b" containerName="payload-processing"
Sep 10 15:59:16.524942 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:59:16.524496    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="ae4f3225-d4d9-4e87-ba9b-fc9ce03cf3df" containerName="payload-pre-processing"
Sep 10 15:59:16.524942 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:59:16.524502    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="9deb994d-b35c-4651-bb7b-aba81f779366" containerName="payload-pre-processing"
Sep 10 15:59:16.524942 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:59:16.524506    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="b25a0acb-f46d-4e62-9c5f-b47d0710cc4a" containerName="maas-api"
Sep 10 15:59:16.524942 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:59:16.524513    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="0181721a-e1e2-4991-8970-8073aebda063" containerName="manager"
Sep 10 15:59:16.524942 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:59:16.524519    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="524402b2-7893-418c-8e4b-2364a39cad2c" containerName="maas-api"
Sep 10 15:59:16.524942 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:59:16.524523    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="ee2c75eb-0049-4111-94f7-39cc5e7896d1" containerName="maas-api"
Sep 10 15:59:16.524942 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:59:16.524529    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="738e3d0d-8d31-4cec-bfe3-efa3c955d119" containerName="payload-pre-processing"
Sep 10 15:59:16.524942 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:59:16.524536    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="d2205171-a2e0-4934-8bcf-f6931285bca5" containerName="payload-processing"
Sep 10 15:59:16.527532 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:59:16.527514    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-68569f6669-zvk8l"
Sep 10 15:59:16.531573 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:59:16.531550    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Sep 10 15:59:16.531573 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:59:16.531561    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-dsv9b\""
Sep 10 15:59:16.531734 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:59:16.531552    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Sep 10 15:59:16.531734 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:59:16.531604    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\""
Sep 10 15:59:16.538496 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:59:16.538476    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-68569f6669-zvk8l"]
Sep 10 15:59:16.671290 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:59:16.671252    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h6fqz\" (UniqueName: \"kubernetes.io/projected/32eaa2da-8244-4158-b55a-29d32ec876d4-kube-api-access-h6fqz\") pod \"maas-controller-68569f6669-zvk8l\" (UID: \"32eaa2da-8244-4158-b55a-29d32ec876d4\") " pod="opendatahub/maas-controller-68569f6669-zvk8l"
Sep 10 15:59:16.671486 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:59:16.671298    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/32eaa2da-8244-4158-b55a-29d32ec876d4-webhook-cert\") pod \"maas-controller-68569f6669-zvk8l\" (UID: \"32eaa2da-8244-4158-b55a-29d32ec876d4\") " pod="opendatahub/maas-controller-68569f6669-zvk8l"
Sep 10 15:59:16.671486 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:59:16.671406    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/32eaa2da-8244-4158-b55a-29d32ec876d4-metrics-certs\") pod \"maas-controller-68569f6669-zvk8l\" (UID: \"32eaa2da-8244-4158-b55a-29d32ec876d4\") " pod="opendatahub/maas-controller-68569f6669-zvk8l"
Sep 10 15:59:16.772446 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:59:16.772411    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h6fqz\" (UniqueName: \"kubernetes.io/projected/32eaa2da-8244-4158-b55a-29d32ec876d4-kube-api-access-h6fqz\") pod \"maas-controller-68569f6669-zvk8l\" (UID: \"32eaa2da-8244-4158-b55a-29d32ec876d4\") " pod="opendatahub/maas-controller-68569f6669-zvk8l"
Sep 10 15:59:16.772604 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:59:16.772458    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/32eaa2da-8244-4158-b55a-29d32ec876d4-webhook-cert\") pod \"maas-controller-68569f6669-zvk8l\" (UID: \"32eaa2da-8244-4158-b55a-29d32ec876d4\") " pod="opendatahub/maas-controller-68569f6669-zvk8l"
Sep 10 15:59:16.772604 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:59:16.772493    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/32eaa2da-8244-4158-b55a-29d32ec876d4-metrics-certs\") pod \"maas-controller-68569f6669-zvk8l\" (UID: \"32eaa2da-8244-4158-b55a-29d32ec876d4\") " pod="opendatahub/maas-controller-68569f6669-zvk8l"
Sep 10 15:59:16.774803 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:59:16.774748    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/32eaa2da-8244-4158-b55a-29d32ec876d4-metrics-certs\") pod \"maas-controller-68569f6669-zvk8l\" (UID: \"32eaa2da-8244-4158-b55a-29d32ec876d4\") " pod="opendatahub/maas-controller-68569f6669-zvk8l"
Sep 10 15:59:16.774911 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:59:16.774877    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/32eaa2da-8244-4158-b55a-29d32ec876d4-webhook-cert\") pod \"maas-controller-68569f6669-zvk8l\" (UID: \"32eaa2da-8244-4158-b55a-29d32ec876d4\") " pod="opendatahub/maas-controller-68569f6669-zvk8l"
Sep 10 15:59:16.783537 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:59:16.783514    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h6fqz\" (UniqueName: \"kubernetes.io/projected/32eaa2da-8244-4158-b55a-29d32ec876d4-kube-api-access-h6fqz\") pod \"maas-controller-68569f6669-zvk8l\" (UID: \"32eaa2da-8244-4158-b55a-29d32ec876d4\") " pod="opendatahub/maas-controller-68569f6669-zvk8l"
Sep 10 15:59:16.838271 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:59:16.838246    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-68569f6669-zvk8l"
Sep 10 15:59:17.111563 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:59:17.111531    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-68569f6669-zvk8l"]
Sep 10 15:59:17.113710 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:59:17.113684    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod32eaa2da_8244_4158_b55a_29d32ec876d4.slice/crio-47f248e52a615a106f91f4cf39bc485f82724b5647129b62ff518b84455f108f WatchSource:0}: Error finding container 47f248e52a615a106f91f4cf39bc485f82724b5647129b62ff518b84455f108f: Status 404 returned error can't find the container with id 47f248e52a615a106f91f4cf39bc485f82724b5647129b62ff518b84455f108f
Sep 10 15:59:17.115067 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:59:17.115048    2567 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 10 15:59:17.953588 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:59:17.953552    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-68569f6669-zvk8l" event={"ID":"32eaa2da-8244-4158-b55a-29d32ec876d4","Type":"ContainerStarted","Data":"2ab4113b772e1d3a13a7ff64daabe3950f8ab7165ceb6a9adac31ae84ef5f3ac"}
Sep 10 15:59:17.953987 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:59:17.953600    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-68569f6669-zvk8l" event={"ID":"32eaa2da-8244-4158-b55a-29d32ec876d4","Type":"ContainerStarted","Data":"47f248e52a615a106f91f4cf39bc485f82724b5647129b62ff518b84455f108f"}
Sep 10 15:59:17.953987 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:59:17.953684    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-68569f6669-zvk8l"
Sep 10 15:59:17.972475 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:59:17.972431    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-68569f6669-zvk8l" podStartSLOduration=1.299119412 podStartE2EDuration="1.972416284s" totalImagesPullingTime="673.296872ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 15:59:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 15:59:17.971455424 +0000 UTC m=+1943.539687154" watchObservedRunningTime="2026-09-10 15:59:17.972416284 +0000 UTC m=+1943.540648014"
Sep 10 15:59:28.959685 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:59:28.959652    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-68569f6669-zvk8l"
Sep 10 15:59:38.202024 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:59:38.201990    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:59:38.294493 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:59:38.294458    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-68569f6669-zvk8l"]
Sep 10 15:59:38.294689 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:59:38.294663    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-68569f6669-zvk8l" podUID="32eaa2da-8244-4158-b55a-29d32ec876d4" containerName="manager" containerID="cri-o://2ab4113b772e1d3a13a7ff64daabe3950f8ab7165ceb6a9adac31ae84ef5f3ac" gracePeriod=30
Sep 10 15:59:39.031621 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:59:39.031595    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-68569f6669-zvk8l"
Sep 10 15:59:39.055072 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:59:39.055035    2567 generic.go:356] "Generic (PLEG): container finished" podID="32eaa2da-8244-4158-b55a-29d32ec876d4" containerID="2ab4113b772e1d3a13a7ff64daabe3950f8ab7165ceb6a9adac31ae84ef5f3ac" exitCode=0
Sep 10 15:59:39.055224 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:59:39.055083    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-68569f6669-zvk8l"
Sep 10 15:59:39.055224 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:59:39.055116    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-68569f6669-zvk8l" event={"ID":"32eaa2da-8244-4158-b55a-29d32ec876d4","Type":"ContainerDied","Data":"2ab4113b772e1d3a13a7ff64daabe3950f8ab7165ceb6a9adac31ae84ef5f3ac"}
Sep 10 15:59:39.055224 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:59:39.055165    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-68569f6669-zvk8l" event={"ID":"32eaa2da-8244-4158-b55a-29d32ec876d4","Type":"ContainerDied","Data":"47f248e52a615a106f91f4cf39bc485f82724b5647129b62ff518b84455f108f"}
Sep 10 15:59:39.055224 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:59:39.055188    2567 scope.go:117] "RemoveContainer" containerID="2ab4113b772e1d3a13a7ff64daabe3950f8ab7165ceb6a9adac31ae84ef5f3ac"
Sep 10 15:59:39.064379 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:59:39.064358    2567 scope.go:117] "RemoveContainer" containerID="2ab4113b772e1d3a13a7ff64daabe3950f8ab7165ceb6a9adac31ae84ef5f3ac"
Sep 10 15:59:39.064657 ip-10-0-128-241 kubenswrapper[2567]: E0910 15:59:39.064637    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2ab4113b772e1d3a13a7ff64daabe3950f8ab7165ceb6a9adac31ae84ef5f3ac\": container with ID starting with 2ab4113b772e1d3a13a7ff64daabe3950f8ab7165ceb6a9adac31ae84ef5f3ac not found: ID does not exist" containerID="2ab4113b772e1d3a13a7ff64daabe3950f8ab7165ceb6a9adac31ae84ef5f3ac"
Sep 10 15:59:39.064725 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:59:39.064668    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2ab4113b772e1d3a13a7ff64daabe3950f8ab7165ceb6a9adac31ae84ef5f3ac"} err="failed to get container status \"2ab4113b772e1d3a13a7ff64daabe3950f8ab7165ceb6a9adac31ae84ef5f3ac\": rpc error: code = NotFound desc = could not find container \"2ab4113b772e1d3a13a7ff64daabe3950f8ab7165ceb6a9adac31ae84ef5f3ac\": container with ID starting with 2ab4113b772e1d3a13a7ff64daabe3950f8ab7165ceb6a9adac31ae84ef5f3ac not found: ID does not exist"
Sep 10 15:59:39.071992 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:59:39.071972    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h6fqz\" (UniqueName: \"kubernetes.io/projected/32eaa2da-8244-4158-b55a-29d32ec876d4-kube-api-access-h6fqz\") pod \"32eaa2da-8244-4158-b55a-29d32ec876d4\" (UID: \"32eaa2da-8244-4158-b55a-29d32ec876d4\") "
Sep 10 15:59:39.072107 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:59:39.072033    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/32eaa2da-8244-4158-b55a-29d32ec876d4-webhook-cert\") pod \"32eaa2da-8244-4158-b55a-29d32ec876d4\" (UID: \"32eaa2da-8244-4158-b55a-29d32ec876d4\") "
Sep 10 15:59:39.072107 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:59:39.072079    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/32eaa2da-8244-4158-b55a-29d32ec876d4-metrics-certs\") pod \"32eaa2da-8244-4158-b55a-29d32ec876d4\" (UID: \"32eaa2da-8244-4158-b55a-29d32ec876d4\") "
Sep 10 15:59:39.074209 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:59:39.074180    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/32eaa2da-8244-4158-b55a-29d32ec876d4-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "32eaa2da-8244-4158-b55a-29d32ec876d4" (UID: "32eaa2da-8244-4158-b55a-29d32ec876d4"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 15:59:39.074546 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:59:39.074519    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/32eaa2da-8244-4158-b55a-29d32ec876d4-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "32eaa2da-8244-4158-b55a-29d32ec876d4" (UID: "32eaa2da-8244-4158-b55a-29d32ec876d4"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 15:59:39.074660 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:59:39.074626    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/32eaa2da-8244-4158-b55a-29d32ec876d4-kube-api-access-h6fqz" (OuterVolumeSpecName: "kube-api-access-h6fqz") pod "32eaa2da-8244-4158-b55a-29d32ec876d4" (UID: "32eaa2da-8244-4158-b55a-29d32ec876d4"). InnerVolumeSpecName "kube-api-access-h6fqz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 15:59:39.172733 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:59:39.172708    2567 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/32eaa2da-8244-4158-b55a-29d32ec876d4-metrics-certs\") on node \"ip-10-0-128-241.ec2.internal\" DevicePath \"\""
Sep 10 15:59:39.172873 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:59:39.172735    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h6fqz\" (UniqueName: \"kubernetes.io/projected/32eaa2da-8244-4158-b55a-29d32ec876d4-kube-api-access-h6fqz\") on node \"ip-10-0-128-241.ec2.internal\" DevicePath \"\""
Sep 10 15:59:39.172873 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:59:39.172747    2567 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/32eaa2da-8244-4158-b55a-29d32ec876d4-webhook-cert\") on node \"ip-10-0-128-241.ec2.internal\" DevicePath \"\""
Sep 10 15:59:39.384770 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:59:39.384741    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-68569f6669-zvk8l"]
Sep 10 15:59:39.389952 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:59:39.389926    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-68569f6669-zvk8l"]
Sep 10 15:59:39.958974 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:59:39.958930    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="opendatahub/maas-controller-68569f6669-zvk8l" podUID="32eaa2da-8244-4158-b55a-29d32ec876d4" containerName="manager" probeResult="failure" output="Get \"http://10.132.0.75:8081/readyz\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Sep 10 15:59:40.058992 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:59:40.058956    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 15:59:40.199815 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:59:40.199783    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-68569f6669-xb22r"]
Sep 10 15:59:40.200209 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:59:40.200190    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="32eaa2da-8244-4158-b55a-29d32ec876d4" containerName="manager"
Sep 10 15:59:40.200335 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:59:40.200213    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="32eaa2da-8244-4158-b55a-29d32ec876d4" containerName="manager"
Sep 10 15:59:40.200390 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:59:40.200343    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="32eaa2da-8244-4158-b55a-29d32ec876d4" containerName="manager"
Sep 10 15:59:40.204693 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:59:40.204669    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-68569f6669-xb22r"
Sep 10 15:59:40.207644 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:59:40.207626    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\""
Sep 10 15:59:40.208699 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:59:40.208679    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Sep 10 15:59:40.208800 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:59:40.208698    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Sep 10 15:59:40.208800 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:59:40.208684    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-dsv9b\""
Sep 10 15:59:40.219193 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:59:40.219171    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-68569f6669-xb22r"]
Sep 10 15:59:40.278846 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:59:40.278820    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6ndg9\" (UniqueName: \"kubernetes.io/projected/286af2a2-bbd2-415c-96dc-fa96daa937b4-kube-api-access-6ndg9\") pod \"maas-controller-68569f6669-xb22r\" (UID: \"286af2a2-bbd2-415c-96dc-fa96daa937b4\") " pod="opendatahub/maas-controller-68569f6669-xb22r"
Sep 10 15:59:40.278985 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:59:40.278869    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/286af2a2-bbd2-415c-96dc-fa96daa937b4-webhook-cert\") pod \"maas-controller-68569f6669-xb22r\" (UID: \"286af2a2-bbd2-415c-96dc-fa96daa937b4\") " pod="opendatahub/maas-controller-68569f6669-xb22r"
Sep 10 15:59:40.278985 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:59:40.278945    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/286af2a2-bbd2-415c-96dc-fa96daa937b4-metrics-certs\") pod \"maas-controller-68569f6669-xb22r\" (UID: \"286af2a2-bbd2-415c-96dc-fa96daa937b4\") " pod="opendatahub/maas-controller-68569f6669-xb22r"
Sep 10 15:59:40.380286 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:59:40.380251    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6ndg9\" (UniqueName: \"kubernetes.io/projected/286af2a2-bbd2-415c-96dc-fa96daa937b4-kube-api-access-6ndg9\") pod \"maas-controller-68569f6669-xb22r\" (UID: \"286af2a2-bbd2-415c-96dc-fa96daa937b4\") " pod="opendatahub/maas-controller-68569f6669-xb22r"
Sep 10 15:59:40.380467 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:59:40.380304    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/286af2a2-bbd2-415c-96dc-fa96daa937b4-webhook-cert\") pod \"maas-controller-68569f6669-xb22r\" (UID: \"286af2a2-bbd2-415c-96dc-fa96daa937b4\") " pod="opendatahub/maas-controller-68569f6669-xb22r"
Sep 10 15:59:40.380467 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:59:40.380338    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/286af2a2-bbd2-415c-96dc-fa96daa937b4-metrics-certs\") pod \"maas-controller-68569f6669-xb22r\" (UID: \"286af2a2-bbd2-415c-96dc-fa96daa937b4\") " pod="opendatahub/maas-controller-68569f6669-xb22r"
Sep 10 15:59:40.382731 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:59:40.382698    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/286af2a2-bbd2-415c-96dc-fa96daa937b4-webhook-cert\") pod \"maas-controller-68569f6669-xb22r\" (UID: \"286af2a2-bbd2-415c-96dc-fa96daa937b4\") " pod="opendatahub/maas-controller-68569f6669-xb22r"
Sep 10 15:59:40.382856 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:59:40.382835    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/286af2a2-bbd2-415c-96dc-fa96daa937b4-metrics-certs\") pod \"maas-controller-68569f6669-xb22r\" (UID: \"286af2a2-bbd2-415c-96dc-fa96daa937b4\") " pod="opendatahub/maas-controller-68569f6669-xb22r"
Sep 10 15:59:40.392468 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:59:40.392445    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6ndg9\" (UniqueName: \"kubernetes.io/projected/286af2a2-bbd2-415c-96dc-fa96daa937b4-kube-api-access-6ndg9\") pod \"maas-controller-68569f6669-xb22r\" (UID: \"286af2a2-bbd2-415c-96dc-fa96daa937b4\") " pod="opendatahub/maas-controller-68569f6669-xb22r"
Sep 10 15:59:40.514986 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:59:40.514913    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-68569f6669-xb22r"
Sep 10 15:59:40.783994 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:59:40.783897    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-68569f6669-xb22r"]
Sep 10 15:59:40.786618 ip-10-0-128-241 kubenswrapper[2567]: W0910 15:59:40.786594    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod286af2a2_bbd2_415c_96dc_fa96daa937b4.slice/crio-65fa2e65e51fd0adec3a201c4b8381094f6089b6d5942b2646ace1c6b9b4726e WatchSource:0}: Error finding container 65fa2e65e51fd0adec3a201c4b8381094f6089b6d5942b2646ace1c6b9b4726e: Status 404 returned error can't find the container with id 65fa2e65e51fd0adec3a201c4b8381094f6089b6d5942b2646ace1c6b9b4726e
Sep 10 15:59:40.903741 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:59:40.903708    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="32eaa2da-8244-4158-b55a-29d32ec876d4" path="/var/lib/kubelet/pods/32eaa2da-8244-4158-b55a-29d32ec876d4/volumes"
Sep 10 15:59:41.067272 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:59:41.067179    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-68569f6669-xb22r" event={"ID":"286af2a2-bbd2-415c-96dc-fa96daa937b4","Type":"ContainerStarted","Data":"65fa2e65e51fd0adec3a201c4b8381094f6089b6d5942b2646ace1c6b9b4726e"}
Sep 10 15:59:42.073530 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:59:42.073490    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-68569f6669-xb22r" event={"ID":"286af2a2-bbd2-415c-96dc-fa96daa937b4","Type":"ContainerStarted","Data":"40757ce4953dc82750331e38b886b343a52c492b887f82bdb1e6802f9001908b"}
Sep 10 15:59:42.073919 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:59:42.073583    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-68569f6669-xb22r"
Sep 10 15:59:42.093732 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:59:42.093680    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-68569f6669-xb22r" podStartSLOduration=1.7539208099999999 podStartE2EDuration="2.093664068s" totalImagesPullingTime="339.743258ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 15:59:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 15:59:42.091254349 +0000 UTC m=+1967.659486077" watchObservedRunningTime="2026-09-10 15:59:42.093664068 +0000 UTC m=+1967.661895807"
Sep 10 15:59:53.080538 ip-10-0-128-241 kubenswrapper[2567]: I0910 15:59:53.080455    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-68569f6669-xb22r"
Sep 10 16:01:55.001047 ip-10-0-128-241 kubenswrapper[2567]: I0910 16:01:55.000956    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-t96pm_0b9d2cda-73e3-4e7c-9e17-1a3d903d95cf/console-operator/2.log"
Sep 10 16:01:55.011319 ip-10-0-128-241 kubenswrapper[2567]: I0910 16:01:55.011290    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-t96pm_0b9d2cda-73e3-4e7c-9e17-1a3d903d95cf/console-operator/2.log"
Sep 10 16:02:25.341294 ip-10-0-128-241 kubenswrapper[2567]: I0910 16:02:25.341252    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 16:02:25.430349 ip-10-0-128-241 kubenswrapper[2567]: I0910 16:02:25.430316    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 16:02:35.531608 ip-10-0-128-241 kubenswrapper[2567]: I0910 16:02:35.531574    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 16:02:46.308601 ip-10-0-128-241 kubenswrapper[2567]: I0910 16:02:46.308565    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-b-130b9555-kserve-69b45fc775-hfzg8"]
Sep 10 16:02:46.312044 ip-10-0-128-241 kubenswrapper[2567]: I0910 16:02:46.312024    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-130b9555-kserve-69b45fc775-hfzg8"
Sep 10 16:02:46.314693 ip-10-0-128-241 kubenswrapper[2567]: I0910 16:02:46.314672    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-b-130b9555-kserve-self-signed-certs\""
Sep 10 16:02:46.322178 ip-10-0-128-241 kubenswrapper[2567]: I0910 16:02:46.322156    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-130b9555-kserve-69b45fc775-hfzg8"]
Sep 10 16:02:46.351106 ip-10-0-128-241 kubenswrapper[2567]: I0910 16:02:46.351076    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 16:02:46.381990 ip-10-0-128-241 kubenswrapper[2567]: I0910 16:02:46.381961    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/13ae729b-fa06-400f-b2d3-be211867a90f-tmp-dir\") pod \"e2e-same-id-b-130b9555-kserve-69b45fc775-hfzg8\" (UID: \"13ae729b-fa06-400f-b2d3-be211867a90f\") " pod="llm/e2e-same-id-b-130b9555-kserve-69b45fc775-hfzg8"
Sep 10 16:02:46.382159 ip-10-0-128-241 kubenswrapper[2567]: I0910 16:02:46.382130    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/13ae729b-fa06-400f-b2d3-be211867a90f-model-cache\") pod \"e2e-same-id-b-130b9555-kserve-69b45fc775-hfzg8\" (UID: \"13ae729b-fa06-400f-b2d3-be211867a90f\") " pod="llm/e2e-same-id-b-130b9555-kserve-69b45fc775-hfzg8"
Sep 10 16:02:46.382255 ip-10-0-128-241 kubenswrapper[2567]: I0910 16:02:46.382183    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/13ae729b-fa06-400f-b2d3-be211867a90f-dshm\") pod \"e2e-same-id-b-130b9555-kserve-69b45fc775-hfzg8\" (UID: \"13ae729b-fa06-400f-b2d3-be211867a90f\") " pod="llm/e2e-same-id-b-130b9555-kserve-69b45fc775-hfzg8"
Sep 10 16:02:46.382255 ip-10-0-128-241 kubenswrapper[2567]: I0910 16:02:46.382213    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/13ae729b-fa06-400f-b2d3-be211867a90f-home\") pod \"e2e-same-id-b-130b9555-kserve-69b45fc775-hfzg8\" (UID: \"13ae729b-fa06-400f-b2d3-be211867a90f\") " pod="llm/e2e-same-id-b-130b9555-kserve-69b45fc775-hfzg8"
Sep 10 16:02:46.382376 ip-10-0-128-241 kubenswrapper[2567]: I0910 16:02:46.382313    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-crchv\" (UniqueName: \"kubernetes.io/projected/13ae729b-fa06-400f-b2d3-be211867a90f-kube-api-access-crchv\") pod \"e2e-same-id-b-130b9555-kserve-69b45fc775-hfzg8\" (UID: \"13ae729b-fa06-400f-b2d3-be211867a90f\") " pod="llm/e2e-same-id-b-130b9555-kserve-69b45fc775-hfzg8"
Sep 10 16:02:46.382376 ip-10-0-128-241 kubenswrapper[2567]: I0910 16:02:46.382361    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/13ae729b-fa06-400f-b2d3-be211867a90f-tls-certs\") pod \"e2e-same-id-b-130b9555-kserve-69b45fc775-hfzg8\" (UID: \"13ae729b-fa06-400f-b2d3-be211867a90f\") " pod="llm/e2e-same-id-b-130b9555-kserve-69b45fc775-hfzg8"
Sep 10 16:02:46.483791 ip-10-0-128-241 kubenswrapper[2567]: I0910 16:02:46.483756    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/13ae729b-fa06-400f-b2d3-be211867a90f-dshm\") pod \"e2e-same-id-b-130b9555-kserve-69b45fc775-hfzg8\" (UID: \"13ae729b-fa06-400f-b2d3-be211867a90f\") " pod="llm/e2e-same-id-b-130b9555-kserve-69b45fc775-hfzg8"
Sep 10 16:02:46.483791 ip-10-0-128-241 kubenswrapper[2567]: I0910 16:02:46.483790    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/13ae729b-fa06-400f-b2d3-be211867a90f-home\") pod \"e2e-same-id-b-130b9555-kserve-69b45fc775-hfzg8\" (UID: \"13ae729b-fa06-400f-b2d3-be211867a90f\") " pod="llm/e2e-same-id-b-130b9555-kserve-69b45fc775-hfzg8"
Sep 10 16:02:46.484034 ip-10-0-128-241 kubenswrapper[2567]: I0910 16:02:46.483922    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-crchv\" (UniqueName: \"kubernetes.io/projected/13ae729b-fa06-400f-b2d3-be211867a90f-kube-api-access-crchv\") pod \"e2e-same-id-b-130b9555-kserve-69b45fc775-hfzg8\" (UID: \"13ae729b-fa06-400f-b2d3-be211867a90f\") " pod="llm/e2e-same-id-b-130b9555-kserve-69b45fc775-hfzg8"
Sep 10 16:02:46.484034 ip-10-0-128-241 kubenswrapper[2567]: I0910 16:02:46.483971    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/13ae729b-fa06-400f-b2d3-be211867a90f-tls-certs\") pod \"e2e-same-id-b-130b9555-kserve-69b45fc775-hfzg8\" (UID: \"13ae729b-fa06-400f-b2d3-be211867a90f\") " pod="llm/e2e-same-id-b-130b9555-kserve-69b45fc775-hfzg8"
Sep 10 16:02:46.484034 ip-10-0-128-241 kubenswrapper[2567]: I0910 16:02:46.484005    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/13ae729b-fa06-400f-b2d3-be211867a90f-tmp-dir\") pod \"e2e-same-id-b-130b9555-kserve-69b45fc775-hfzg8\" (UID: \"13ae729b-fa06-400f-b2d3-be211867a90f\") " pod="llm/e2e-same-id-b-130b9555-kserve-69b45fc775-hfzg8"
Sep 10 16:02:46.484198 ip-10-0-128-241 kubenswrapper[2567]: I0910 16:02:46.484068    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/13ae729b-fa06-400f-b2d3-be211867a90f-model-cache\") pod \"e2e-same-id-b-130b9555-kserve-69b45fc775-hfzg8\" (UID: \"13ae729b-fa06-400f-b2d3-be211867a90f\") " pod="llm/e2e-same-id-b-130b9555-kserve-69b45fc775-hfzg8"
Sep 10 16:02:46.484293 ip-10-0-128-241 kubenswrapper[2567]: I0910 16:02:46.484205    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/13ae729b-fa06-400f-b2d3-be211867a90f-home\") pod \"e2e-same-id-b-130b9555-kserve-69b45fc775-hfzg8\" (UID: \"13ae729b-fa06-400f-b2d3-be211867a90f\") " pod="llm/e2e-same-id-b-130b9555-kserve-69b45fc775-hfzg8"
Sep 10 16:02:46.484421 ip-10-0-128-241 kubenswrapper[2567]: I0910 16:02:46.484391    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/13ae729b-fa06-400f-b2d3-be211867a90f-tmp-dir\") pod \"e2e-same-id-b-130b9555-kserve-69b45fc775-hfzg8\" (UID: \"13ae729b-fa06-400f-b2d3-be211867a90f\") " pod="llm/e2e-same-id-b-130b9555-kserve-69b45fc775-hfzg8"
Sep 10 16:02:46.484493 ip-10-0-128-241 kubenswrapper[2567]: I0910 16:02:46.484438    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/13ae729b-fa06-400f-b2d3-be211867a90f-model-cache\") pod \"e2e-same-id-b-130b9555-kserve-69b45fc775-hfzg8\" (UID: \"13ae729b-fa06-400f-b2d3-be211867a90f\") " pod="llm/e2e-same-id-b-130b9555-kserve-69b45fc775-hfzg8"
Sep 10 16:02:46.486363 ip-10-0-128-241 kubenswrapper[2567]: I0910 16:02:46.486334    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/13ae729b-fa06-400f-b2d3-be211867a90f-dshm\") pod \"e2e-same-id-b-130b9555-kserve-69b45fc775-hfzg8\" (UID: \"13ae729b-fa06-400f-b2d3-be211867a90f\") " pod="llm/e2e-same-id-b-130b9555-kserve-69b45fc775-hfzg8"
Sep 10 16:02:46.486550 ip-10-0-128-241 kubenswrapper[2567]: I0910 16:02:46.486531    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/13ae729b-fa06-400f-b2d3-be211867a90f-tls-certs\") pod \"e2e-same-id-b-130b9555-kserve-69b45fc775-hfzg8\" (UID: \"13ae729b-fa06-400f-b2d3-be211867a90f\") " pod="llm/e2e-same-id-b-130b9555-kserve-69b45fc775-hfzg8"
Sep 10 16:02:46.494594 ip-10-0-128-241 kubenswrapper[2567]: I0910 16:02:46.494574    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-crchv\" (UniqueName: \"kubernetes.io/projected/13ae729b-fa06-400f-b2d3-be211867a90f-kube-api-access-crchv\") pod \"e2e-same-id-b-130b9555-kserve-69b45fc775-hfzg8\" (UID: \"13ae729b-fa06-400f-b2d3-be211867a90f\") " pod="llm/e2e-same-id-b-130b9555-kserve-69b45fc775-hfzg8"
Sep 10 16:02:46.621950 ip-10-0-128-241 kubenswrapper[2567]: I0910 16:02:46.621865    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-130b9555-kserve-69b45fc775-hfzg8"
Sep 10 16:02:46.894186 ip-10-0-128-241 kubenswrapper[2567]: I0910 16:02:46.894164    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-130b9555-kserve-69b45fc775-hfzg8"]
Sep 10 16:02:46.896695 ip-10-0-128-241 kubenswrapper[2567]: W0910 16:02:46.896667    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod13ae729b_fa06_400f_b2d3_be211867a90f.slice/crio-09d22dfc72b18e1bdf339cead2c9a9d60535dbaf0e5746aa192bee01ad9839df WatchSource:0}: Error finding container 09d22dfc72b18e1bdf339cead2c9a9d60535dbaf0e5746aa192bee01ad9839df: Status 404 returned error can't find the container with id 09d22dfc72b18e1bdf339cead2c9a9d60535dbaf0e5746aa192bee01ad9839df
Sep 10 16:02:47.899765 ip-10-0-128-241 kubenswrapper[2567]: I0910 16:02:47.899730    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-130b9555-kserve-69b45fc775-hfzg8" event={"ID":"13ae729b-fa06-400f-b2d3-be211867a90f","Type":"ContainerStarted","Data":"78d7bafecfde31ee342737795b2f695fd668c13d41d62ae5d04a07d41507cbda"}
Sep 10 16:02:47.899765 ip-10-0-128-241 kubenswrapper[2567]: I0910 16:02:47.899773    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-130b9555-kserve-69b45fc775-hfzg8" event={"ID":"13ae729b-fa06-400f-b2d3-be211867a90f","Type":"ContainerStarted","Data":"09d22dfc72b18e1bdf339cead2c9a9d60535dbaf0e5746aa192bee01ad9839df"}
Sep 10 16:02:47.917183 ip-10-0-128-241 kubenswrapper[2567]: I0910 16:02:47.917137    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-b-130b9555-kserve-69b45fc775-hfzg8" podStartSLOduration=1.9171228550000001 podStartE2EDuration="1.917122855s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 16:02:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 16:02:47.916929001 +0000 UTC m=+2153.485160731" watchObservedRunningTime="2026-09-10 16:02:47.917122855 +0000 UTC m=+2153.485354586"
Sep 10 16:02:56.622706 ip-10-0-128-241 kubenswrapper[2567]: I0910 16:02:56.622670    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-b-130b9555-kserve-69b45fc775-hfzg8"
Sep 10 16:02:56.623206 ip-10-0-128-241 kubenswrapper[2567]: I0910 16:02:56.622719    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-b-130b9555-kserve-69b45fc775-hfzg8"
Sep 10 16:02:56.634862 ip-10-0-128-241 kubenswrapper[2567]: I0910 16:02:56.634843    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-b-130b9555-kserve-69b45fc775-hfzg8"
Sep 10 16:02:56.950514 ip-10-0-128-241 kubenswrapper[2567]: I0910 16:02:56.950488    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-b-130b9555-kserve-69b45fc775-hfzg8"
Sep 10 16:03:17.496185 ip-10-0-128-241 kubenswrapper[2567]: I0910 16:03:17.496143    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 16:03:26.699767 ip-10-0-128-241 kubenswrapper[2567]: I0910 16:03:26.699731    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 16:03:28.904486 ip-10-0-128-241 kubenswrapper[2567]: I0910 16:03:28.904451    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-130b9555-kserve-69b45fc775-hfzg8"]
Sep 10 16:03:28.945971 ip-10-0-128-241 kubenswrapper[2567]: I0910 16:03:28.945939    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 16:03:40.016041 ip-10-0-128-241 kubenswrapper[2567]: I0910 16:03:40.015999    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 16:03:43.923390 ip-10-0-128-241 kubenswrapper[2567]: I0910 16:03:43.923347    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-b-130b9555-kserve-69b45fc775-hfzg8" podUID="13ae729b-fa06-400f-b2d3-be211867a90f" containerName="main" containerID="cri-o://78d7bafecfde31ee342737795b2f695fd668c13d41d62ae5d04a07d41507cbda" gracePeriod=45
Sep 10 16:03:44.148722 ip-10-0-128-241 kubenswrapper[2567]: I0910 16:03:44.148570    2567 generic.go:356] "Generic (PLEG): container finished" podID="13ae729b-fa06-400f-b2d3-be211867a90f" containerID="78d7bafecfde31ee342737795b2f695fd668c13d41d62ae5d04a07d41507cbda" exitCode=0
Sep 10 16:03:44.148722 ip-10-0-128-241 kubenswrapper[2567]: I0910 16:03:44.148614    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-130b9555-kserve-69b45fc775-hfzg8" event={"ID":"13ae729b-fa06-400f-b2d3-be211867a90f","Type":"ContainerDied","Data":"78d7bafecfde31ee342737795b2f695fd668c13d41d62ae5d04a07d41507cbda"}
Sep 10 16:03:44.167489 ip-10-0-128-241 kubenswrapper[2567]: I0910 16:03:44.167464    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-130b9555-kserve-69b45fc775-hfzg8"
Sep 10 16:03:44.306058 ip-10-0-128-241 kubenswrapper[2567]: I0910 16:03:44.305968    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/13ae729b-fa06-400f-b2d3-be211867a90f-home\") pod \"13ae729b-fa06-400f-b2d3-be211867a90f\" (UID: \"13ae729b-fa06-400f-b2d3-be211867a90f\") "
Sep 10 16:03:44.306058 ip-10-0-128-241 kubenswrapper[2567]: I0910 16:03:44.306038    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/13ae729b-fa06-400f-b2d3-be211867a90f-dshm\") pod \"13ae729b-fa06-400f-b2d3-be211867a90f\" (UID: \"13ae729b-fa06-400f-b2d3-be211867a90f\") "
Sep 10 16:03:44.306322 ip-10-0-128-241 kubenswrapper[2567]: I0910 16:03:44.306063    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-crchv\" (UniqueName: \"kubernetes.io/projected/13ae729b-fa06-400f-b2d3-be211867a90f-kube-api-access-crchv\") pod \"13ae729b-fa06-400f-b2d3-be211867a90f\" (UID: \"13ae729b-fa06-400f-b2d3-be211867a90f\") "
Sep 10 16:03:44.306322 ip-10-0-128-241 kubenswrapper[2567]: I0910 16:03:44.306087    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/13ae729b-fa06-400f-b2d3-be211867a90f-tmp-dir\") pod \"13ae729b-fa06-400f-b2d3-be211867a90f\" (UID: \"13ae729b-fa06-400f-b2d3-be211867a90f\") "
Sep 10 16:03:44.306322 ip-10-0-128-241 kubenswrapper[2567]: I0910 16:03:44.306137    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/13ae729b-fa06-400f-b2d3-be211867a90f-model-cache\") pod \"13ae729b-fa06-400f-b2d3-be211867a90f\" (UID: \"13ae729b-fa06-400f-b2d3-be211867a90f\") "
Sep 10 16:03:44.306322 ip-10-0-128-241 kubenswrapper[2567]: I0910 16:03:44.306180    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/13ae729b-fa06-400f-b2d3-be211867a90f-tls-certs\") pod \"13ae729b-fa06-400f-b2d3-be211867a90f\" (UID: \"13ae729b-fa06-400f-b2d3-be211867a90f\") "
Sep 10 16:03:44.306322 ip-10-0-128-241 kubenswrapper[2567]: I0910 16:03:44.306263    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/13ae729b-fa06-400f-b2d3-be211867a90f-home" (OuterVolumeSpecName: "home") pod "13ae729b-fa06-400f-b2d3-be211867a90f" (UID: "13ae729b-fa06-400f-b2d3-be211867a90f"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 16:03:44.306567 ip-10-0-128-241 kubenswrapper[2567]: I0910 16:03:44.306394    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/13ae729b-fa06-400f-b2d3-be211867a90f-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "13ae729b-fa06-400f-b2d3-be211867a90f" (UID: "13ae729b-fa06-400f-b2d3-be211867a90f"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 16:03:44.306567 ip-10-0-128-241 kubenswrapper[2567]: I0910 16:03:44.306456    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/13ae729b-fa06-400f-b2d3-be211867a90f-model-cache" (OuterVolumeSpecName: "model-cache") pod "13ae729b-fa06-400f-b2d3-be211867a90f" (UID: "13ae729b-fa06-400f-b2d3-be211867a90f"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 16:03:44.306567 ip-10-0-128-241 kubenswrapper[2567]: I0910 16:03:44.306476    2567 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/13ae729b-fa06-400f-b2d3-be211867a90f-home\") on node \"ip-10-0-128-241.ec2.internal\" DevicePath \"\""
Sep 10 16:03:44.308276 ip-10-0-128-241 kubenswrapper[2567]: I0910 16:03:44.308251    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/13ae729b-fa06-400f-b2d3-be211867a90f-dshm" (OuterVolumeSpecName: "dshm") pod "13ae729b-fa06-400f-b2d3-be211867a90f" (UID: "13ae729b-fa06-400f-b2d3-be211867a90f"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 16:03:44.308365 ip-10-0-128-241 kubenswrapper[2567]: I0910 16:03:44.308296    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/13ae729b-fa06-400f-b2d3-be211867a90f-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "13ae729b-fa06-400f-b2d3-be211867a90f" (UID: "13ae729b-fa06-400f-b2d3-be211867a90f"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 16:03:44.308365 ip-10-0-128-241 kubenswrapper[2567]: I0910 16:03:44.308328    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/13ae729b-fa06-400f-b2d3-be211867a90f-kube-api-access-crchv" (OuterVolumeSpecName: "kube-api-access-crchv") pod "13ae729b-fa06-400f-b2d3-be211867a90f" (UID: "13ae729b-fa06-400f-b2d3-be211867a90f"). InnerVolumeSpecName "kube-api-access-crchv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 16:03:44.407186 ip-10-0-128-241 kubenswrapper[2567]: I0910 16:03:44.407143    2567 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/13ae729b-fa06-400f-b2d3-be211867a90f-model-cache\") on node \"ip-10-0-128-241.ec2.internal\" DevicePath \"\""
Sep 10 16:03:44.407186 ip-10-0-128-241 kubenswrapper[2567]: I0910 16:03:44.407177    2567 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/13ae729b-fa06-400f-b2d3-be211867a90f-tls-certs\") on node \"ip-10-0-128-241.ec2.internal\" DevicePath \"\""
Sep 10 16:03:44.407186 ip-10-0-128-241 kubenswrapper[2567]: I0910 16:03:44.407189    2567 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/13ae729b-fa06-400f-b2d3-be211867a90f-dshm\") on node \"ip-10-0-128-241.ec2.internal\" DevicePath \"\""
Sep 10 16:03:44.407461 ip-10-0-128-241 kubenswrapper[2567]: I0910 16:03:44.407200    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-crchv\" (UniqueName: \"kubernetes.io/projected/13ae729b-fa06-400f-b2d3-be211867a90f-kube-api-access-crchv\") on node \"ip-10-0-128-241.ec2.internal\" DevicePath \"\""
Sep 10 16:03:44.407461 ip-10-0-128-241 kubenswrapper[2567]: I0910 16:03:44.407213    2567 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/13ae729b-fa06-400f-b2d3-be211867a90f-tmp-dir\") on node \"ip-10-0-128-241.ec2.internal\" DevicePath \"\""
Sep 10 16:03:45.153608 ip-10-0-128-241 kubenswrapper[2567]: I0910 16:03:45.153577    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-130b9555-kserve-69b45fc775-hfzg8" event={"ID":"13ae729b-fa06-400f-b2d3-be211867a90f","Type":"ContainerDied","Data":"09d22dfc72b18e1bdf339cead2c9a9d60535dbaf0e5746aa192bee01ad9839df"}
Sep 10 16:03:45.153608 ip-10-0-128-241 kubenswrapper[2567]: I0910 16:03:45.153610    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-130b9555-kserve-69b45fc775-hfzg8"
Sep 10 16:03:45.154082 ip-10-0-128-241 kubenswrapper[2567]: I0910 16:03:45.153628    2567 scope.go:117] "RemoveContainer" containerID="78d7bafecfde31ee342737795b2f695fd668c13d41d62ae5d04a07d41507cbda"
Sep 10 16:03:45.172755 ip-10-0-128-241 kubenswrapper[2567]: I0910 16:03:45.172727    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-130b9555-kserve-69b45fc775-hfzg8"]
Sep 10 16:03:45.177513 ip-10-0-128-241 kubenswrapper[2567]: I0910 16:03:45.177488    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-b-130b9555-kserve-69b45fc775-hfzg8"]
Sep 10 16:03:46.903774 ip-10-0-128-241 kubenswrapper[2567]: I0910 16:03:46.903738    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="13ae729b-fa06-400f-b2d3-be211867a90f" path="/var/lib/kubelet/pods/13ae729b-fa06-400f-b2d3-be211867a90f/volumes"
Sep 10 16:03:47.959330 ip-10-0-128-241 kubenswrapper[2567]: I0910 16:03:47.959299    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-5df4475445-6hj8l"]
Sep 10 16:03:47.959700 ip-10-0-128-241 kubenswrapper[2567]: I0910 16:03:47.959660    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="13ae729b-fa06-400f-b2d3-be211867a90f" containerName="main"
Sep 10 16:03:47.959700 ip-10-0-128-241 kubenswrapper[2567]: I0910 16:03:47.959671    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="13ae729b-fa06-400f-b2d3-be211867a90f" containerName="main"
Sep 10 16:03:47.959777 ip-10-0-128-241 kubenswrapper[2567]: I0910 16:03:47.959721    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="13ae729b-fa06-400f-b2d3-be211867a90f" containerName="main"
Sep 10 16:03:47.963808 ip-10-0-128-241 kubenswrapper[2567]: I0910 16:03:47.963782    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-5df4475445-6hj8l"
Sep 10 16:03:47.975866 ip-10-0-128-241 kubenswrapper[2567]: I0910 16:03:47.975840    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-5df4475445-6hj8l"]
Sep 10 16:03:48.142203 ip-10-0-128-241 kubenswrapper[2567]: I0910 16:03:48.142163    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4e163a97-b983-4e07-82c8-d5b092212d26-maas-api-tls\") pod \"maas-api-5df4475445-6hj8l\" (UID: \"4e163a97-b983-4e07-82c8-d5b092212d26\") " pod="odh-ai-gateway-infra/maas-api-5df4475445-6hj8l"
Sep 10 16:03:48.142415 ip-10-0-128-241 kubenswrapper[2567]: I0910 16:03:48.142226    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4e163a97-b983-4e07-82c8-d5b092212d26-openshift-service-ca\") pod \"maas-api-5df4475445-6hj8l\" (UID: \"4e163a97-b983-4e07-82c8-d5b092212d26\") " pod="odh-ai-gateway-infra/maas-api-5df4475445-6hj8l"
Sep 10 16:03:48.142415 ip-10-0-128-241 kubenswrapper[2567]: I0910 16:03:48.142293    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s2rvg\" (UniqueName: \"kubernetes.io/projected/4e163a97-b983-4e07-82c8-d5b092212d26-kube-api-access-s2rvg\") pod \"maas-api-5df4475445-6hj8l\" (UID: \"4e163a97-b983-4e07-82c8-d5b092212d26\") " pod="odh-ai-gateway-infra/maas-api-5df4475445-6hj8l"
Sep 10 16:03:48.142415 ip-10-0-128-241 kubenswrapper[2567]: I0910 16:03:48.142320    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4e163a97-b983-4e07-82c8-d5b092212d26-metrics-certs\") pod \"maas-api-5df4475445-6hj8l\" (UID: \"4e163a97-b983-4e07-82c8-d5b092212d26\") " pod="odh-ai-gateway-infra/maas-api-5df4475445-6hj8l"
Sep 10 16:03:48.243356 ip-10-0-128-241 kubenswrapper[2567]: I0910 16:03:48.243225    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4e163a97-b983-4e07-82c8-d5b092212d26-openshift-service-ca\") pod \"maas-api-5df4475445-6hj8l\" (UID: \"4e163a97-b983-4e07-82c8-d5b092212d26\") " pod="odh-ai-gateway-infra/maas-api-5df4475445-6hj8l"
Sep 10 16:03:48.243356 ip-10-0-128-241 kubenswrapper[2567]: I0910 16:03:48.243312    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s2rvg\" (UniqueName: \"kubernetes.io/projected/4e163a97-b983-4e07-82c8-d5b092212d26-kube-api-access-s2rvg\") pod \"maas-api-5df4475445-6hj8l\" (UID: \"4e163a97-b983-4e07-82c8-d5b092212d26\") " pod="odh-ai-gateway-infra/maas-api-5df4475445-6hj8l"
Sep 10 16:03:48.243599 ip-10-0-128-241 kubenswrapper[2567]: I0910 16:03:48.243412    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4e163a97-b983-4e07-82c8-d5b092212d26-metrics-certs\") pod \"maas-api-5df4475445-6hj8l\" (UID: \"4e163a97-b983-4e07-82c8-d5b092212d26\") " pod="odh-ai-gateway-infra/maas-api-5df4475445-6hj8l"
Sep 10 16:03:48.243599 ip-10-0-128-241 kubenswrapper[2567]: I0910 16:03:48.243533    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4e163a97-b983-4e07-82c8-d5b092212d26-maas-api-tls\") pod \"maas-api-5df4475445-6hj8l\" (UID: \"4e163a97-b983-4e07-82c8-d5b092212d26\") " pod="odh-ai-gateway-infra/maas-api-5df4475445-6hj8l"
Sep 10 16:03:48.243994 ip-10-0-128-241 kubenswrapper[2567]: I0910 16:03:48.243968    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4e163a97-b983-4e07-82c8-d5b092212d26-openshift-service-ca\") pod \"maas-api-5df4475445-6hj8l\" (UID: \"4e163a97-b983-4e07-82c8-d5b092212d26\") " pod="odh-ai-gateway-infra/maas-api-5df4475445-6hj8l"
Sep 10 16:03:48.245892 ip-10-0-128-241 kubenswrapper[2567]: I0910 16:03:48.245872    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4e163a97-b983-4e07-82c8-d5b092212d26-maas-api-tls\") pod \"maas-api-5df4475445-6hj8l\" (UID: \"4e163a97-b983-4e07-82c8-d5b092212d26\") " pod="odh-ai-gateway-infra/maas-api-5df4475445-6hj8l"
Sep 10 16:03:48.245981 ip-10-0-128-241 kubenswrapper[2567]: I0910 16:03:48.245926    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4e163a97-b983-4e07-82c8-d5b092212d26-metrics-certs\") pod \"maas-api-5df4475445-6hj8l\" (UID: \"4e163a97-b983-4e07-82c8-d5b092212d26\") " pod="odh-ai-gateway-infra/maas-api-5df4475445-6hj8l"
Sep 10 16:03:48.258219 ip-10-0-128-241 kubenswrapper[2567]: I0910 16:03:48.258186    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s2rvg\" (UniqueName: \"kubernetes.io/projected/4e163a97-b983-4e07-82c8-d5b092212d26-kube-api-access-s2rvg\") pod \"maas-api-5df4475445-6hj8l\" (UID: \"4e163a97-b983-4e07-82c8-d5b092212d26\") " pod="odh-ai-gateway-infra/maas-api-5df4475445-6hj8l"
Sep 10 16:03:48.276052 ip-10-0-128-241 kubenswrapper[2567]: I0910 16:03:48.276025    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-5df4475445-6hj8l"
Sep 10 16:03:48.560534 ip-10-0-128-241 kubenswrapper[2567]: I0910 16:03:48.560445    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-5df4475445-6hj8l"]
Sep 10 16:03:48.565938 ip-10-0-128-241 kubenswrapper[2567]: W0910 16:03:48.565906    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4e163a97_b983_4e07_82c8_d5b092212d26.slice/crio-f33867580ec80256a9444a81ea3c9a49bee61d1d809b4f8bcbe56933004465fd WatchSource:0}: Error finding container f33867580ec80256a9444a81ea3c9a49bee61d1d809b4f8bcbe56933004465fd: Status 404 returned error can't find the container with id f33867580ec80256a9444a81ea3c9a49bee61d1d809b4f8bcbe56933004465fd
Sep 10 16:03:49.173859 ip-10-0-128-241 kubenswrapper[2567]: I0910 16:03:49.173823    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-5df4475445-6hj8l" event={"ID":"4e163a97-b983-4e07-82c8-d5b092212d26","Type":"ContainerStarted","Data":"f33867580ec80256a9444a81ea3c9a49bee61d1d809b4f8bcbe56933004465fd"}
Sep 10 16:03:50.181124 ip-10-0-128-241 kubenswrapper[2567]: I0910 16:03:50.181085    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-5df4475445-6hj8l" event={"ID":"4e163a97-b983-4e07-82c8-d5b092212d26","Type":"ContainerStarted","Data":"d1643ee367d1f40d90d00d475da4187bdf981c6d28db86be09a26a58ac6e79a2"}
Sep 10 16:03:50.181569 ip-10-0-128-241 kubenswrapper[2567]: I0910 16:03:50.181191    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-5df4475445-6hj8l"
Sep 10 16:03:50.210579 ip-10-0-128-241 kubenswrapper[2567]: I0910 16:03:50.207413    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-5df4475445-6hj8l" podStartSLOduration=1.772425951 podStartE2EDuration="3.207387379s" totalImagesPullingTime="1.434961428s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 16:03:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 16:03:50.203813549 +0000 UTC m=+2215.772045271" watchObservedRunningTime="2026-09-10 16:03:50.207387379 +0000 UTC m=+2215.775619115"
Sep 10 16:03:56.193073 ip-10-0-128-241 kubenswrapper[2567]: I0910 16:03:56.193041    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-5df4475445-6hj8l"
Sep 10 16:03:56.265512 ip-10-0-128-241 kubenswrapper[2567]: I0910 16:03:56.265479    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-66f7f9db8f-s6ktb"]
Sep 10 16:03:56.265723 ip-10-0-128-241 kubenswrapper[2567]: I0910 16:03:56.265687    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-66f7f9db8f-s6ktb" podUID="5cb4993c-6ce8-49d6-9f34-8aa7c46200f3" containerName="maas-api" containerID="cri-o://850d1cc58c4a3e2213ecdfb075389845c3bf5a17694a06badcf22be5cb9df6e2" gracePeriod=30
Sep 10 16:03:57.509040 ip-10-0-128-241 kubenswrapper[2567]: I0910 16:03:57.509016    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-66f7f9db8f-s6ktb"
Sep 10 16:03:57.520152 ip-10-0-128-241 kubenswrapper[2567]: I0910 16:03:57.519988    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7x9gh\" (UniqueName: \"kubernetes.io/projected/5cb4993c-6ce8-49d6-9f34-8aa7c46200f3-kube-api-access-7x9gh\") pod \"5cb4993c-6ce8-49d6-9f34-8aa7c46200f3\" (UID: \"5cb4993c-6ce8-49d6-9f34-8aa7c46200f3\") "
Sep 10 16:03:57.520304 ip-10-0-128-241 kubenswrapper[2567]: I0910 16:03:57.520194    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5cb4993c-6ce8-49d6-9f34-8aa7c46200f3-metrics-certs\") pod \"5cb4993c-6ce8-49d6-9f34-8aa7c46200f3\" (UID: \"5cb4993c-6ce8-49d6-9f34-8aa7c46200f3\") "
Sep 10 16:03:57.522388 ip-10-0-128-241 kubenswrapper[2567]: I0910 16:03:57.522363    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5cb4993c-6ce8-49d6-9f34-8aa7c46200f3-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "5cb4993c-6ce8-49d6-9f34-8aa7c46200f3" (UID: "5cb4993c-6ce8-49d6-9f34-8aa7c46200f3"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 16:03:57.522591 ip-10-0-128-241 kubenswrapper[2567]: I0910 16:03:57.522571    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5cb4993c-6ce8-49d6-9f34-8aa7c46200f3-kube-api-access-7x9gh" (OuterVolumeSpecName: "kube-api-access-7x9gh") pod "5cb4993c-6ce8-49d6-9f34-8aa7c46200f3" (UID: "5cb4993c-6ce8-49d6-9f34-8aa7c46200f3"). InnerVolumeSpecName "kube-api-access-7x9gh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 16:03:57.621059 ip-10-0-128-241 kubenswrapper[2567]: I0910 16:03:57.620962    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5cb4993c-6ce8-49d6-9f34-8aa7c46200f3-openshift-service-ca\") pod \"5cb4993c-6ce8-49d6-9f34-8aa7c46200f3\" (UID: \"5cb4993c-6ce8-49d6-9f34-8aa7c46200f3\") "
Sep 10 16:03:57.621059 ip-10-0-128-241 kubenswrapper[2567]: I0910 16:03:57.621027    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5cb4993c-6ce8-49d6-9f34-8aa7c46200f3-maas-api-tls\") pod \"5cb4993c-6ce8-49d6-9f34-8aa7c46200f3\" (UID: \"5cb4993c-6ce8-49d6-9f34-8aa7c46200f3\") "
Sep 10 16:03:57.621344 ip-10-0-128-241 kubenswrapper[2567]: I0910 16:03:57.621199    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7x9gh\" (UniqueName: \"kubernetes.io/projected/5cb4993c-6ce8-49d6-9f34-8aa7c46200f3-kube-api-access-7x9gh\") on node \"ip-10-0-128-241.ec2.internal\" DevicePath \"\""
Sep 10 16:03:57.621344 ip-10-0-128-241 kubenswrapper[2567]: I0910 16:03:57.621216    2567 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5cb4993c-6ce8-49d6-9f34-8aa7c46200f3-metrics-certs\") on node \"ip-10-0-128-241.ec2.internal\" DevicePath \"\""
Sep 10 16:03:57.621344 ip-10-0-128-241 kubenswrapper[2567]: I0910 16:03:57.621313    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5cb4993c-6ce8-49d6-9f34-8aa7c46200f3-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "5cb4993c-6ce8-49d6-9f34-8aa7c46200f3" (UID: "5cb4993c-6ce8-49d6-9f34-8aa7c46200f3"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 16:03:57.623128 ip-10-0-128-241 kubenswrapper[2567]: I0910 16:03:57.623102    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5cb4993c-6ce8-49d6-9f34-8aa7c46200f3-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "5cb4993c-6ce8-49d6-9f34-8aa7c46200f3" (UID: "5cb4993c-6ce8-49d6-9f34-8aa7c46200f3"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 16:03:57.721897 ip-10-0-128-241 kubenswrapper[2567]: I0910 16:03:57.721857    2567 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5cb4993c-6ce8-49d6-9f34-8aa7c46200f3-openshift-service-ca\") on node \"ip-10-0-128-241.ec2.internal\" DevicePath \"\""
Sep 10 16:03:57.721897 ip-10-0-128-241 kubenswrapper[2567]: I0910 16:03:57.721892    2567 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5cb4993c-6ce8-49d6-9f34-8aa7c46200f3-maas-api-tls\") on node \"ip-10-0-128-241.ec2.internal\" DevicePath \"\""
Sep 10 16:03:58.219055 ip-10-0-128-241 kubenswrapper[2567]: I0910 16:03:58.219016    2567 generic.go:356] "Generic (PLEG): container finished" podID="5cb4993c-6ce8-49d6-9f34-8aa7c46200f3" containerID="850d1cc58c4a3e2213ecdfb075389845c3bf5a17694a06badcf22be5cb9df6e2" exitCode=0
Sep 10 16:03:58.219249 ip-10-0-128-241 kubenswrapper[2567]: I0910 16:03:58.219079    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-66f7f9db8f-s6ktb"
Sep 10 16:03:58.219249 ip-10-0-128-241 kubenswrapper[2567]: I0910 16:03:58.219109    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-66f7f9db8f-s6ktb" event={"ID":"5cb4993c-6ce8-49d6-9f34-8aa7c46200f3","Type":"ContainerDied","Data":"850d1cc58c4a3e2213ecdfb075389845c3bf5a17694a06badcf22be5cb9df6e2"}
Sep 10 16:03:58.219249 ip-10-0-128-241 kubenswrapper[2567]: I0910 16:03:58.219165    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-66f7f9db8f-s6ktb" event={"ID":"5cb4993c-6ce8-49d6-9f34-8aa7c46200f3","Type":"ContainerDied","Data":"a41ca198eaef2fa9cb4d7e3685092f1a9914cb1cee05748925e3849a794129d8"}
Sep 10 16:03:58.219249 ip-10-0-128-241 kubenswrapper[2567]: I0910 16:03:58.219189    2567 scope.go:117] "RemoveContainer" containerID="850d1cc58c4a3e2213ecdfb075389845c3bf5a17694a06badcf22be5cb9df6e2"
Sep 10 16:03:58.227896 ip-10-0-128-241 kubenswrapper[2567]: I0910 16:03:58.227880    2567 scope.go:117] "RemoveContainer" containerID="850d1cc58c4a3e2213ecdfb075389845c3bf5a17694a06badcf22be5cb9df6e2"
Sep 10 16:03:58.228149 ip-10-0-128-241 kubenswrapper[2567]: E0910 16:03:58.228130    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"850d1cc58c4a3e2213ecdfb075389845c3bf5a17694a06badcf22be5cb9df6e2\": container with ID starting with 850d1cc58c4a3e2213ecdfb075389845c3bf5a17694a06badcf22be5cb9df6e2 not found: ID does not exist" containerID="850d1cc58c4a3e2213ecdfb075389845c3bf5a17694a06badcf22be5cb9df6e2"
Sep 10 16:03:58.228203 ip-10-0-128-241 kubenswrapper[2567]: I0910 16:03:58.228157    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"850d1cc58c4a3e2213ecdfb075389845c3bf5a17694a06badcf22be5cb9df6e2"} err="failed to get container status \"850d1cc58c4a3e2213ecdfb075389845c3bf5a17694a06badcf22be5cb9df6e2\": rpc error: code = NotFound desc = could not find container \"850d1cc58c4a3e2213ecdfb075389845c3bf5a17694a06badcf22be5cb9df6e2\": container with ID starting with 850d1cc58c4a3e2213ecdfb075389845c3bf5a17694a06badcf22be5cb9df6e2 not found: ID does not exist"
Sep 10 16:03:58.245783 ip-10-0-128-241 kubenswrapper[2567]: I0910 16:03:58.245760    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-66f7f9db8f-s6ktb"]
Sep 10 16:03:58.259885 ip-10-0-128-241 kubenswrapper[2567]: I0910 16:03:58.259855    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-66f7f9db8f-s6ktb"]
Sep 10 16:03:58.904653 ip-10-0-128-241 kubenswrapper[2567]: I0910 16:03:58.904619    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5cb4993c-6ce8-49d6-9f34-8aa7c46200f3" path="/var/lib/kubelet/pods/5cb4993c-6ce8-49d6-9f34-8aa7c46200f3/volumes"
Sep 10 16:04:18.068522 ip-10-0-128-241 kubenswrapper[2567]: I0910 16:04:18.068435    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-557cbbbb8c-hn8vs_82379701-1582-41e4-b444-129c64f9ee97/authorino/0.log"
Sep 10 16:04:23.875879 ip-10-0-128-241 kubenswrapper[2567]: I0910 16:04:23.875846    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-68569f6669-xb22r_286af2a2-bbd2-415c-96dc-fa96daa937b4/manager/0.log"
Sep 10 16:04:24.373522 ip-10-0-128-241 kubenswrapper[2567]: I0910 16:04:24.373491    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-797ffb6d9c-sws7t_7fb99e31-3fe0-462b-af8e-cc5d038b6377/manager/0.log"
Sep 10 16:04:24.380543 ip-10-0-128-241 kubenswrapper[2567]: I0910 16:04:24.380522    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-797ffb6d9c-sws7t_7fb99e31-3fe0-462b-af8e-cc5d038b6377/copy-manifests/0.log"
Sep 10 16:04:24.852646 ip-10-0-128-241 kubenswrapper[2567]: I0910 16:04:24.852618    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-5df4475445-6hj8l_4e163a97-b983-4e07-82c8-d5b092212d26/maas-api/0.log"
Sep 10 16:04:25.357394 ip-10-0-128-241 kubenswrapper[2567]: I0910 16:04:25.357363    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_postgres-69fb4594cf-tcnpw_c17c97c0-957e-46eb-be34-41e3f644284a/postgres/0.log"
Sep 10 16:04:26.145732 ip-10-0-128-241 kubenswrapper[2567]: I0910 16:04:26.145696    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-557cbbbb8c-hn8vs_82379701-1582-41e4-b444-129c64f9ee97/authorino/0.log"
Sep 10 16:04:27.545866 ip-10-0-128-241 kubenswrapper[2567]: I0910 16:04:27.545824    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-785ff899f9-7scqc_db9bca7a-dea3-4901-a6ce-b6b85cf40b32/kube-auth-proxy/0.log"
Sep 10 16:04:27.917687 ip-10-0-128-241 kubenswrapper[2567]: I0910 16:04:27.917607    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-kbxnm_66268cd1-5727-4ec6-8832-873f3132d1e2/payload-pre-processing/0.log"
Sep 10 16:04:28.511413 ip-10-0-128-241 kubenswrapper[2567]: I0910 16:04:28.511385    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-67d8d5d6d-vq88d_d61c9899-c724-4ebe-8a03-f90000c3b568/main/0.log"
Sep 10 16:04:28.625707 ip-10-0-128-241 kubenswrapper[2567]: I0910 16:04:28.625669    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-59975d69f9-nxp96_01606484-2402-43a3-98de-8bff3c7cdbc9/main/0.log"
Sep 10 16:04:28.852116 ip-10-0-128-241 kubenswrapper[2567]: I0910 16:04:28.852033    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-598466548d-b6999_623a26a6-a6e2-4939-ac5c-a6e847393d10/main/0.log"
Sep 10 16:04:29.563583 ip-10-0-128-241 kubenswrapper[2567]: I0910 16:04:29.563547    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 16:04:34.730047 ip-10-0-128-241 kubenswrapper[2567]: I0910 16:04:34.730012    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-cs24j/must-gather-2hxbk"]
Sep 10 16:04:34.730436 ip-10-0-128-241 kubenswrapper[2567]: I0910 16:04:34.730420    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5cb4993c-6ce8-49d6-9f34-8aa7c46200f3" containerName="maas-api"
Sep 10 16:04:34.730436 ip-10-0-128-241 kubenswrapper[2567]: I0910 16:04:34.730432    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="5cb4993c-6ce8-49d6-9f34-8aa7c46200f3" containerName="maas-api"
Sep 10 16:04:34.730513 ip-10-0-128-241 kubenswrapper[2567]: I0910 16:04:34.730484    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="5cb4993c-6ce8-49d6-9f34-8aa7c46200f3" containerName="maas-api"
Sep 10 16:04:34.733410 ip-10-0-128-241 kubenswrapper[2567]: I0910 16:04:34.733394    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-cs24j/must-gather-2hxbk"
Sep 10 16:04:34.735993 ip-10-0-128-241 kubenswrapper[2567]: I0910 16:04:34.735973    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-cs24j\"/\"default-dockercfg-sr2kk\""
Sep 10 16:04:34.735993 ip-10-0-128-241 kubenswrapper[2567]: I0910 16:04:34.735985    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-cs24j\"/\"openshift-service-ca.crt\""
Sep 10 16:04:34.736164 ip-10-0-128-241 kubenswrapper[2567]: I0910 16:04:34.735985    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-cs24j\"/\"kube-root-ca.crt\""
Sep 10 16:04:34.744534 ip-10-0-128-241 kubenswrapper[2567]: I0910 16:04:34.744514    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 16:04:34.748379 ip-10-0-128-241 kubenswrapper[2567]: I0910 16:04:34.748359    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-cs24j/must-gather-2hxbk"]
Sep 10 16:04:34.837184 ip-10-0-128-241 kubenswrapper[2567]: I0910 16:04:34.837137    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log" limit=1024
Sep 10 16:04:34.860286 ip-10-0-128-241 kubenswrapper[2567]: I0910 16:04:34.860219    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5d9tt\" (UniqueName: \"kubernetes.io/projected/4b7af9b1-7ecc-4146-af20-6a40e9c67d2a-kube-api-access-5d9tt\") pod \"must-gather-2hxbk\" (UID: \"4b7af9b1-7ecc-4146-af20-6a40e9c67d2a\") " pod="openshift-must-gather-cs24j/must-gather-2hxbk"
Sep 10 16:04:34.860436 ip-10-0-128-241 kubenswrapper[2567]: I0910 16:04:34.860346    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/4b7af9b1-7ecc-4146-af20-6a40e9c67d2a-must-gather-output\") pod \"must-gather-2hxbk\" (UID: \"4b7af9b1-7ecc-4146-af20-6a40e9c67d2a\") " pod="openshift-must-gather-cs24j/must-gather-2hxbk"
Sep 10 16:04:34.961379 ip-10-0-128-241 kubenswrapper[2567]: I0910 16:04:34.961343    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5d9tt\" (UniqueName: \"kubernetes.io/projected/4b7af9b1-7ecc-4146-af20-6a40e9c67d2a-kube-api-access-5d9tt\") pod \"must-gather-2hxbk\" (UID: \"4b7af9b1-7ecc-4146-af20-6a40e9c67d2a\") " pod="openshift-must-gather-cs24j/must-gather-2hxbk"
Sep 10 16:04:34.961543 ip-10-0-128-241 kubenswrapper[2567]: I0910 16:04:34.961397    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/4b7af9b1-7ecc-4146-af20-6a40e9c67d2a-must-gather-output\") pod \"must-gather-2hxbk\" (UID: \"4b7af9b1-7ecc-4146-af20-6a40e9c67d2a\") " pod="openshift-must-gather-cs24j/must-gather-2hxbk"
Sep 10 16:04:34.961760 ip-10-0-128-241 kubenswrapper[2567]: I0910 16:04:34.961735    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/4b7af9b1-7ecc-4146-af20-6a40e9c67d2a-must-gather-output\") pod \"must-gather-2hxbk\" (UID: \"4b7af9b1-7ecc-4146-af20-6a40e9c67d2a\") " pod="openshift-must-gather-cs24j/must-gather-2hxbk"
Sep 10 16:04:34.970390 ip-10-0-128-241 kubenswrapper[2567]: I0910 16:04:34.970368    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5d9tt\" (UniqueName: \"kubernetes.io/projected/4b7af9b1-7ecc-4146-af20-6a40e9c67d2a-kube-api-access-5d9tt\") pod \"must-gather-2hxbk\" (UID: \"4b7af9b1-7ecc-4146-af20-6a40e9c67d2a\") " pod="openshift-must-gather-cs24j/must-gather-2hxbk"
Sep 10 16:04:35.042392 ip-10-0-128-241 kubenswrapper[2567]: I0910 16:04:35.042307    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-cs24j/must-gather-2hxbk"
Sep 10 16:04:35.313687 ip-10-0-128-241 kubenswrapper[2567]: I0910 16:04:35.313661    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-cs24j/must-gather-2hxbk"]
Sep 10 16:04:35.315616 ip-10-0-128-241 kubenswrapper[2567]: W0910 16:04:35.315590    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4b7af9b1_7ecc_4146_af20_6a40e9c67d2a.slice/crio-7a9025b16b492ab091b5060b6f6233d4cc1176849d98eeceaee58673d8841b96 WatchSource:0}: Error finding container 7a9025b16b492ab091b5060b6f6233d4cc1176849d98eeceaee58673d8841b96: Status 404 returned error can't find the container with id 7a9025b16b492ab091b5060b6f6233d4cc1176849d98eeceaee58673d8841b96
Sep 10 16:04:35.317558 ip-10-0-128-241 kubenswrapper[2567]: I0910 16:04:35.317541    2567 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 10 16:04:35.381335 ip-10-0-128-241 kubenswrapper[2567]: I0910 16:04:35.381302    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-cs24j/must-gather-2hxbk" event={"ID":"4b7af9b1-7ecc-4146-af20-6a40e9c67d2a","Type":"ContainerStarted","Data":"7a9025b16b492ab091b5060b6f6233d4cc1176849d98eeceaee58673d8841b96"}
Sep 10 16:04:36.389443 ip-10-0-128-241 kubenswrapper[2567]: I0910 16:04:36.389335    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-cs24j/must-gather-2hxbk" event={"ID":"4b7af9b1-7ecc-4146-af20-6a40e9c67d2a","Type":"ContainerStarted","Data":"8952e47e411e3ea884fd659ee75900dd87d4a21d2fdc3745cee15c6e6ebfb709"}
Sep 10 16:04:36.389443 ip-10-0-128-241 kubenswrapper[2567]: I0910 16:04:36.389396    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-cs24j/must-gather-2hxbk" event={"ID":"4b7af9b1-7ecc-4146-af20-6a40e9c67d2a","Type":"ContainerStarted","Data":"21b1afbfe375159f4c6e5fe66adc3d050575aa806005fa6f86b99592921a25e2"}
Sep 10 16:04:36.430281 ip-10-0-128-241 kubenswrapper[2567]: I0910 16:04:36.430200    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-cs24j/must-gather-2hxbk" podStartSLOduration=1.661217228 podStartE2EDuration="2.430175887s" totalImagesPullingTime="768.958659ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 16:04:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 16:04:36.429870297 +0000 UTC m=+2261.998102027" watchObservedRunningTime="2026-09-10 16:04:36.430175887 +0000 UTC m=+2261.998407618"
Sep 10 16:04:37.932422 ip-10-0-128-241 kubenswrapper[2567]: I0910 16:04:37.932379    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-fgx9b_49bbf894-04df-427c-b71a-c108fbf25054/global-pull-secret-syncer/0.log"
Sep 10 16:04:37.993633 ip-10-0-128-241 kubenswrapper[2567]: I0910 16:04:37.993601    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-pzrzz_aa974d12-9161-4bc9-a1c7-0846241a3206/konnectivity-agent/0.log"
Sep 10 16:04:38.107609 ip-10-0-128-241 kubenswrapper[2567]: I0910 16:04:38.107577    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-128-241.ec2.internal_7cc72b205a190f6d5d54c7ca1053da96/haproxy/0.log"
Sep 10 16:04:42.414254 ip-10-0-128-241 kubenswrapper[2567]: I0910 16:04:42.413534    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-557cbbbb8c-hn8vs_82379701-1582-41e4-b444-129c64f9ee97/authorino/0.log"
Sep 10 16:04:44.516404 ip-10-0-128-241 kubenswrapper[2567]: I0910 16:04:44.516369    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-74cb57df7-pncp7_64105509-967c-440a-948f-8ff2b09eec87/kube-state-metrics/0.log"
Sep 10 16:04:44.543812 ip-10-0-128-241 kubenswrapper[2567]: I0910 16:04:44.543782    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-74cb57df7-pncp7_64105509-967c-440a-948f-8ff2b09eec87/kube-rbac-proxy-main/0.log"
Sep 10 16:04:44.570982 ip-10-0-128-241 kubenswrapper[2567]: I0910 16:04:44.570890    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-74cb57df7-pncp7_64105509-967c-440a-948f-8ff2b09eec87/kube-rbac-proxy-self/0.log"
Sep 10 16:04:44.933778 ip-10-0-128-241 kubenswrapper[2567]: I0910 16:04:44.933180    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-tz22l_6cabb203-0adb-478d-92b6-9dd54c84e8e2/node-exporter/0.log"
Sep 10 16:04:44.967183 ip-10-0-128-241 kubenswrapper[2567]: I0910 16:04:44.967108    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-tz22l_6cabb203-0adb-478d-92b6-9dd54c84e8e2/kube-rbac-proxy/0.log"
Sep 10 16:04:45.001354 ip-10-0-128-241 kubenswrapper[2567]: I0910 16:04:45.001323    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-tz22l_6cabb203-0adb-478d-92b6-9dd54c84e8e2/init-textfile/0.log"
Sep 10 16:04:45.191383 ip-10-0-128-241 kubenswrapper[2567]: I0910 16:04:45.191212    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_65aca3d0-4b3b-4f36-b3b3-456394ada2a3/prometheus/0.log"
Sep 10 16:04:45.242546 ip-10-0-128-241 kubenswrapper[2567]: I0910 16:04:45.242475    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_65aca3d0-4b3b-4f36-b3b3-456394ada2a3/config-reloader/0.log"
Sep 10 16:04:45.289613 ip-10-0-128-241 kubenswrapper[2567]: I0910 16:04:45.289582    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_65aca3d0-4b3b-4f36-b3b3-456394ada2a3/thanos-sidecar/0.log"
Sep 10 16:04:45.323076 ip-10-0-128-241 kubenswrapper[2567]: I0910 16:04:45.323050    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_65aca3d0-4b3b-4f36-b3b3-456394ada2a3/kube-rbac-proxy-web/0.log"
Sep 10 16:04:45.353864 ip-10-0-128-241 kubenswrapper[2567]: I0910 16:04:45.353831    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_65aca3d0-4b3b-4f36-b3b3-456394ada2a3/kube-rbac-proxy/0.log"
Sep 10 16:04:45.391378 ip-10-0-128-241 kubenswrapper[2567]: I0910 16:04:45.391343    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_65aca3d0-4b3b-4f36-b3b3-456394ada2a3/kube-rbac-proxy-thanos/0.log"
Sep 10 16:04:45.417951 ip-10-0-128-241 kubenswrapper[2567]: I0910 16:04:45.417919    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_65aca3d0-4b3b-4f36-b3b3-456394ada2a3/init-config-reloader/0.log"
Sep 10 16:04:47.962459 ip-10-0-128-241 kubenswrapper[2567]: I0910 16:04:47.962421    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-t96pm_0b9d2cda-73e3-4e7c-9e17-1a3d903d95cf/console-operator/2.log"
Sep 10 16:04:47.971768 ip-10-0-128-241 kubenswrapper[2567]: I0910 16:04:47.971732    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-t96pm_0b9d2cda-73e3-4e7c-9e17-1a3d903d95cf/console-operator/3.log"
Sep 10 16:04:48.216597 ip-10-0-128-241 kubenswrapper[2567]: I0910 16:04:48.216514    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-cs24j/perf-node-gather-daemonset-rzn8d"]
Sep 10 16:04:48.222527 ip-10-0-128-241 kubenswrapper[2567]: I0910 16:04:48.222491    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-cs24j/perf-node-gather-daemonset-rzn8d"
Sep 10 16:04:48.231512 ip-10-0-128-241 kubenswrapper[2567]: I0910 16:04:48.231480    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-cs24j/perf-node-gather-daemonset-rzn8d"]
Sep 10 16:04:48.307479 ip-10-0-128-241 kubenswrapper[2567]: I0910 16:04:48.307429    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/de9b62b1-f9a4-4d9f-a2a1-a52098a86725-sys\") pod \"perf-node-gather-daemonset-rzn8d\" (UID: \"de9b62b1-f9a4-4d9f-a2a1-a52098a86725\") " pod="openshift-must-gather-cs24j/perf-node-gather-daemonset-rzn8d"
Sep 10 16:04:48.307479 ip-10-0-128-241 kubenswrapper[2567]: I0910 16:04:48.307485    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/de9b62b1-f9a4-4d9f-a2a1-a52098a86725-podres\") pod \"perf-node-gather-daemonset-rzn8d\" (UID: \"de9b62b1-f9a4-4d9f-a2a1-a52098a86725\") " pod="openshift-must-gather-cs24j/perf-node-gather-daemonset-rzn8d"
Sep 10 16:04:48.307732 ip-10-0-128-241 kubenswrapper[2567]: I0910 16:04:48.307518    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/de9b62b1-f9a4-4d9f-a2a1-a52098a86725-proc\") pod \"perf-node-gather-daemonset-rzn8d\" (UID: \"de9b62b1-f9a4-4d9f-a2a1-a52098a86725\") " pod="openshift-must-gather-cs24j/perf-node-gather-daemonset-rzn8d"
Sep 10 16:04:48.307732 ip-10-0-128-241 kubenswrapper[2567]: I0910 16:04:48.307572    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jdb5c\" (UniqueName: \"kubernetes.io/projected/de9b62b1-f9a4-4d9f-a2a1-a52098a86725-kube-api-access-jdb5c\") pod \"perf-node-gather-daemonset-rzn8d\" (UID: \"de9b62b1-f9a4-4d9f-a2a1-a52098a86725\") " pod="openshift-must-gather-cs24j/perf-node-gather-daemonset-rzn8d"
Sep 10 16:04:48.307732 ip-10-0-128-241 kubenswrapper[2567]: I0910 16:04:48.307643    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/de9b62b1-f9a4-4d9f-a2a1-a52098a86725-lib-modules\") pod \"perf-node-gather-daemonset-rzn8d\" (UID: \"de9b62b1-f9a4-4d9f-a2a1-a52098a86725\") " pod="openshift-must-gather-cs24j/perf-node-gather-daemonset-rzn8d"
Sep 10 16:04:48.408456 ip-10-0-128-241 kubenswrapper[2567]: I0910 16:04:48.408411    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/de9b62b1-f9a4-4d9f-a2a1-a52098a86725-sys\") pod \"perf-node-gather-daemonset-rzn8d\" (UID: \"de9b62b1-f9a4-4d9f-a2a1-a52098a86725\") " pod="openshift-must-gather-cs24j/perf-node-gather-daemonset-rzn8d"
Sep 10 16:04:48.408648 ip-10-0-128-241 kubenswrapper[2567]: I0910 16:04:48.408466    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/de9b62b1-f9a4-4d9f-a2a1-a52098a86725-podres\") pod \"perf-node-gather-daemonset-rzn8d\" (UID: \"de9b62b1-f9a4-4d9f-a2a1-a52098a86725\") " pod="openshift-must-gather-cs24j/perf-node-gather-daemonset-rzn8d"
Sep 10 16:04:48.408648 ip-10-0-128-241 kubenswrapper[2567]: I0910 16:04:48.408505    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/de9b62b1-f9a4-4d9f-a2a1-a52098a86725-proc\") pod \"perf-node-gather-daemonset-rzn8d\" (UID: \"de9b62b1-f9a4-4d9f-a2a1-a52098a86725\") " pod="openshift-must-gather-cs24j/perf-node-gather-daemonset-rzn8d"
Sep 10 16:04:48.408648 ip-10-0-128-241 kubenswrapper[2567]: I0910 16:04:48.408558    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jdb5c\" (UniqueName: \"kubernetes.io/projected/de9b62b1-f9a4-4d9f-a2a1-a52098a86725-kube-api-access-jdb5c\") pod \"perf-node-gather-daemonset-rzn8d\" (UID: \"de9b62b1-f9a4-4d9f-a2a1-a52098a86725\") " pod="openshift-must-gather-cs24j/perf-node-gather-daemonset-rzn8d"
Sep 10 16:04:48.408648 ip-10-0-128-241 kubenswrapper[2567]: I0910 16:04:48.408623    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/de9b62b1-f9a4-4d9f-a2a1-a52098a86725-lib-modules\") pod \"perf-node-gather-daemonset-rzn8d\" (UID: \"de9b62b1-f9a4-4d9f-a2a1-a52098a86725\") " pod="openshift-must-gather-cs24j/perf-node-gather-daemonset-rzn8d"
Sep 10 16:04:48.408911 ip-10-0-128-241 kubenswrapper[2567]: I0910 16:04:48.408794    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/de9b62b1-f9a4-4d9f-a2a1-a52098a86725-lib-modules\") pod \"perf-node-gather-daemonset-rzn8d\" (UID: \"de9b62b1-f9a4-4d9f-a2a1-a52098a86725\") " pod="openshift-must-gather-cs24j/perf-node-gather-daemonset-rzn8d"
Sep 10 16:04:48.408911 ip-10-0-128-241 kubenswrapper[2567]: I0910 16:04:48.408857    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/de9b62b1-f9a4-4d9f-a2a1-a52098a86725-sys\") pod \"perf-node-gather-daemonset-rzn8d\" (UID: \"de9b62b1-f9a4-4d9f-a2a1-a52098a86725\") " pod="openshift-must-gather-cs24j/perf-node-gather-daemonset-rzn8d"
Sep 10 16:04:48.409033 ip-10-0-128-241 kubenswrapper[2567]: I0910 16:04:48.408930    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/de9b62b1-f9a4-4d9f-a2a1-a52098a86725-podres\") pod \"perf-node-gather-daemonset-rzn8d\" (UID: \"de9b62b1-f9a4-4d9f-a2a1-a52098a86725\") " pod="openshift-must-gather-cs24j/perf-node-gather-daemonset-rzn8d"
Sep 10 16:04:48.409033 ip-10-0-128-241 kubenswrapper[2567]: I0910 16:04:48.408975    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/de9b62b1-f9a4-4d9f-a2a1-a52098a86725-proc\") pod \"perf-node-gather-daemonset-rzn8d\" (UID: \"de9b62b1-f9a4-4d9f-a2a1-a52098a86725\") " pod="openshift-must-gather-cs24j/perf-node-gather-daemonset-rzn8d"
Sep 10 16:04:48.419700 ip-10-0-128-241 kubenswrapper[2567]: I0910 16:04:48.419664    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jdb5c\" (UniqueName: \"kubernetes.io/projected/de9b62b1-f9a4-4d9f-a2a1-a52098a86725-kube-api-access-jdb5c\") pod \"perf-node-gather-daemonset-rzn8d\" (UID: \"de9b62b1-f9a4-4d9f-a2a1-a52098a86725\") " pod="openshift-must-gather-cs24j/perf-node-gather-daemonset-rzn8d"
Sep 10 16:04:48.534826 ip-10-0-128-241 kubenswrapper[2567]: I0910 16:04:48.534728    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-cs24j/perf-node-gather-daemonset-rzn8d"
Sep 10 16:04:48.844527 ip-10-0-128-241 kubenswrapper[2567]: I0910 16:04:48.843820    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-cs24j/perf-node-gather-daemonset-rzn8d"]
Sep 10 16:04:48.845927 ip-10-0-128-241 kubenswrapper[2567]: W0910 16:04:48.845893    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podde9b62b1_f9a4_4d9f_a2a1_a52098a86725.slice/crio-5f759d9f621dad902bc2eedb5f932422cc06dd6f57ea0aff403b527498ff1172 WatchSource:0}: Error finding container 5f759d9f621dad902bc2eedb5f932422cc06dd6f57ea0aff403b527498ff1172: Status 404 returned error can't find the container with id 5f759d9f621dad902bc2eedb5f932422cc06dd6f57ea0aff403b527498ff1172
Sep 10 16:04:49.500777 ip-10-0-128-241 kubenswrapper[2567]: I0910 16:04:49.500733    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-cs24j/perf-node-gather-daemonset-rzn8d" event={"ID":"de9b62b1-f9a4-4d9f-a2a1-a52098a86725","Type":"ContainerStarted","Data":"9cefddfc39a4cac27483f43a311d92e4733f65177b90d468012016c9f81dbbfc"}
Sep 10 16:04:49.501315 ip-10-0-128-241 kubenswrapper[2567]: I0910 16:04:49.500840    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-cs24j/perf-node-gather-daemonset-rzn8d" event={"ID":"de9b62b1-f9a4-4d9f-a2a1-a52098a86725","Type":"ContainerStarted","Data":"5f759d9f621dad902bc2eedb5f932422cc06dd6f57ea0aff403b527498ff1172"}
Sep 10 16:04:49.501315 ip-10-0-128-241 kubenswrapper[2567]: I0910 16:04:49.501134    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-cs24j/perf-node-gather-daemonset-rzn8d"
Sep 10 16:04:49.524375 ip-10-0-128-241 kubenswrapper[2567]: I0910 16:04:49.524315    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-cs24j/perf-node-gather-daemonset-rzn8d" podStartSLOduration=1.524294014 podStartE2EDuration="1.524294014s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 16:04:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 16:04:49.523815705 +0000 UTC m=+2275.092047436" watchObservedRunningTime="2026-09-10 16:04:49.524294014 +0000 UTC m=+2275.092525742"
Sep 10 16:04:50.790437 ip-10-0-128-241 kubenswrapper[2567]: I0910 16:04:50.790406    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-4k8rd_9818ae98-700a-4e5c-a4a5-24c0fc624542/dns/0.log"
Sep 10 16:04:50.823602 ip-10-0-128-241 kubenswrapper[2567]: I0910 16:04:50.823568    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-4k8rd_9818ae98-700a-4e5c-a4a5-24c0fc624542/kube-rbac-proxy/0.log"
Sep 10 16:04:51.094807 ip-10-0-128-241 kubenswrapper[2567]: I0910 16:04:51.094723    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-rctzd_4dd1b47c-e497-45d2-a404-4ea3a2d8ec94/dns-node-resolver/0.log"
Sep 10 16:04:51.768708 ip-10-0-128-241 kubenswrapper[2567]: I0910 16:04:51.768667    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-g8rkv_e9b09c08-aec9-41db-8386-e62731150ccc/node-ca/0.log"
Sep 10 16:04:53.588937 ip-10-0-128-241 kubenswrapper[2567]: I0910 16:04:53.588907    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-785ff899f9-7scqc_db9bca7a-dea3-4901-a6ce-b6b85cf40b32/kube-auth-proxy/0.log"
Sep 10 16:04:53.756870 ip-10-0-128-241 kubenswrapper[2567]: I0910 16:04:53.756826    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-kbxnm_66268cd1-5727-4ec6-8832-873f3132d1e2/payload-pre-processing/0.log"
Sep 10 16:04:54.843342 ip-10-0-128-241 kubenswrapper[2567]: I0910 16:04:54.843292    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-h8h5g_b9ea6621-b91a-44bd-a38f-666177f4d26b/serve-healthcheck-canary/0.log"
Sep 10 16:04:55.482143 ip-10-0-128-241 kubenswrapper[2567]: I0910 16:04:55.482112    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-5vhfm_66247a82-0118-4ced-87d2-5f65c49557dc/kube-rbac-proxy/0.log"
Sep 10 16:04:55.508675 ip-10-0-128-241 kubenswrapper[2567]: I0910 16:04:55.508582    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-5vhfm_66247a82-0118-4ced-87d2-5f65c49557dc/exporter/0.log"
Sep 10 16:04:55.516808 ip-10-0-128-241 kubenswrapper[2567]: I0910 16:04:55.516778    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-cs24j/perf-node-gather-daemonset-rzn8d"
Sep 10 16:04:55.554488 ip-10-0-128-241 kubenswrapper[2567]: I0910 16:04:55.554451    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-5vhfm_66247a82-0118-4ced-87d2-5f65c49557dc/extractor/0.log"
Sep 10 16:04:58.651866 ip-10-0-128-241 kubenswrapper[2567]: I0910 16:04:58.651800    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-68569f6669-xb22r_286af2a2-bbd2-415c-96dc-fa96daa937b4/manager/0.log"
Sep 10 16:04:59.041873 ip-10-0-128-241 kubenswrapper[2567]: I0910 16:04:59.041832    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-797ffb6d9c-sws7t_7fb99e31-3fe0-462b-af8e-cc5d038b6377/manager/0.log"
Sep 10 16:04:59.063928 ip-10-0-128-241 kubenswrapper[2567]: I0910 16:04:59.063896    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-797ffb6d9c-sws7t_7fb99e31-3fe0-462b-af8e-cc5d038b6377/copy-manifests/0.log"
Sep 10 16:04:59.236520 ip-10-0-128-241 kubenswrapper[2567]: I0910 16:04:59.236489    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-785ff899f9-7scqc_db9bca7a-dea3-4901-a6ce-b6b85cf40b32/kube-auth-proxy/0.log"
Sep 10 16:04:59.355141 ip-10-0-128-241 kubenswrapper[2567]: I0910 16:04:59.355056    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-kbxnm_66268cd1-5727-4ec6-8832-873f3132d1e2/payload-pre-processing/0.log"
Sep 10 16:05:00.589146 ip-10-0-128-241 kubenswrapper[2567]: I0910 16:05:00.589045    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-d87857f9b-hjfmt_0c234d4b-0010-4600-820b-5fb1645b411c/manager/0.log"
Sep 10 16:05:01.690640 ip-10-0-128-241 kubenswrapper[2567]: I0910 16:05:01.690611    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-785ff899f9-7scqc_db9bca7a-dea3-4901-a6ce-b6b85cf40b32/kube-auth-proxy/0.log"
Sep 10 16:05:01.909823 ip-10-0-128-241 kubenswrapper[2567]: I0910 16:05:01.909786    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-kbxnm_66268cd1-5727-4ec6-8832-873f3132d1e2/payload-pre-processing/0.log"